JP6270624B2 - Outside environment recognition device - Google Patents

Description

  The present invention relates to a vehicle environment recognition device that recognizes an environment outside the host vehicle, and more particularly to a vehicle environment recognition device that is suitable for identifying a traffic light.

  Conventionally, a specific object such as a vehicle or a traffic light located in front of the host vehicle is detected to avoid a collision with the preceding vehicle (collision avoidance control), or the distance between the preceding vehicle and the signal color of the traffic signal is recognized. There is known a technique (cruise control) for controlling the vehicle so as to keep it at a safe distance (for example, Patent Document 1).

  Such a specific object is extracted from an image obtained by capturing an environment outside the vehicle ahead of the host vehicle based on the luminance and distance. For example, color images outside the vehicle are captured, adjacent pixels are grouped together, and light emitters (lamps) such as traffic lights are recognized based on distance, size, height, and position of the host vehicle with respect to the traveling path. The technique to do is known (for example, patent document 2).

Japanese Patent No. 3349060 JP 2010-224925 A

  By the way, some of the traffic signals are provided with a traffic signal for a streetcar provided for a streetcar, for example, in addition to a traffic signal for a vehicle provided for a vehicle. In such a traffic light, the light emitters of the car traffic light and the streetcar traffic light each emit light simultaneously. Therefore, if the signal colors of the light emitters of the car traffic signal and the tramway traffic signal are simply specified as specific objects, the technique as in Patent Document 1 will automatically be based on the signal color of the tramway traffic signal. The vehicle will be controlled. Therefore, it is desired to accurately specify only the signal color of the automobile traffic signal even when a traffic signal other than the automobile traffic signal is provided.

  In view of such a problem, an object of the present invention is to provide a vehicle environment recognition apparatus that can increase the accuracy of identifying a traffic light.

  In order to solve the above-described problems, an external environment recognition apparatus according to the present invention includes a light emitter of a traffic light, a traffic light identifying unit that identifies a signal color of the light emitter, and a position information generating unit that derives the position of the light emitter. A plurality of the light emitters and at least one of the light emitters has a red signal color, the relative distance is the same as the light emitter of the red signal color on the basis of the light emitter of the red signal color. A target that excludes the red signal light emitter from the control target object, provided that there is another light emitter of the signal color above the light emitter of the red signal color. An object exclusion unit.

  In addition, the traffic signal includes a traffic signal for a vehicle for a vehicle and a traffic signal for a moving body other than the vehicle, and the traffic signal for the vehicle may be disposed above a traffic signal for a moving body other than the vehicle. .

  According to the present invention, it is possible to increase the accuracy of identifying a traffic light.

It is the block diagram which showed the connection relation of the environment recognition system. It is the functional block diagram which showed the schematic function of the external environment recognition apparatus. It is explanatory drawing for demonstrating a luminance image and a distance image. It is explanatory drawing for demonstrating operation | movement of a traffic signal specific | specification part. It is a figure which shows the traffic signal with which the traffic signal for motor vehicles and the traffic signal for streetcars were attached. It is a flowchart for demonstrating a target object exclusion process.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

  In recent years, an in-vehicle camera mounted on a vehicle images a road environment ahead of the host vehicle, identifies an object such as a preceding vehicle based on color information and position information in the image, and collides with the identified object. Vehicles equipped with a so-called anti-collision function, which avoids the above-mentioned problem and keeps the inter-vehicle distance from the preceding vehicle at a safe distance (ACC: Adaptive Cruise Control), are becoming popular.

  In a vehicle equipped with such a vehicle environment recognition device for recognizing the vehicle environment, it is conceivable to control the traveling state of the host vehicle based on the signal color of a traffic light located in front of the vehicle. For example, if the signal color of the traffic light ahead is red while traveling in ACC, it may be possible to brake the vehicle and shift to a stopped state.

  However, some of the traffic lights are provided with a traffic signal for a streetcar provided for a streetcar, for example, in addition to a traffic signal for an automobile provided for a vehicle. In such a traffic light, the light emitters (lamps) of either the traffic light for automobiles and the traffic light for streetcars may emit light simultaneously. Therefore, when a plurality of light emitters are recognized, the vehicle exterior environment recognition device identifies a signal color of a traffic signal for an automobile provided for the automobile from the plurality of light emitters, and the host vehicle is based on the signal color. It is necessary to control the running state. Therefore, in the present embodiment, an outside environment recognition device is provided that can improve the accuracy of identifying a traffic signal by excluding the signal color of the traffic signal provided for another moving body (tram) from the control target. To do.

(Environment recognition system 100)
FIG. 1 is a block diagram showing a connection relationship of the environment recognition system 100. The environment recognition system 100 includes an imaging device 110, a vehicle exterior environment recognition device 120, and a vehicle control device (ECU: Engine Control Unit) 130 provided in the host vehicle 1.

  The imaging device 110 includes an imaging element such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), captures an environment corresponding to the front of the host vehicle 1, and captures three hues (R ( A color image or a monochrome image composed of (red), G (green), and B (blue) can be generated. Here, the color image picked up by the image pickup device 110 is called a luminance image, and is distinguished from a distance image described later.

  In addition, the imaging devices 110 are arranged in a substantially horizontal direction so that the optical axes of the two imaging devices 110 are substantially parallel on the traveling direction side of the host vehicle 1. The imaging device 110 continuously generates image data obtained by imaging an object existing in the detection area in front of the host vehicle 1, for example, every 1/60 second frame (60 fps). Here, the object to be recognized is not only a three-dimensional object that exists independently, such as a vehicle, a pedestrian, a traffic light, a road (traveling path), and a guardrail, but also one of three-dimensional objects such as taillights, blinkers, and lighting parts of traffic lights. The thing which can be specified as a part is also included. Each functional unit in the following embodiment performs each process for each frame in response to such update of the image data.

  The vehicle exterior environment recognition device 120 acquires image data from each of the two imaging devices 110, derives parallax using so-called pattern matching, and associates the derived parallax information (corresponding to a relative distance described later) with the image data. Generate a distance image. The luminance image and the distance image will be described in detail later. Further, the outside environment recognition device 120 uses the luminance based on the luminance image and the relative distance from the own vehicle 1 based on the distance image to which specific object the target in the detection area in front of the own vehicle 1 corresponds. Is identified.

  When the outside environment recognition apparatus 120 identifies a specific object, the vehicle control apparatus 130 instructs the vehicle control apparatus 130 to permit the travel of the host vehicle 1 or stop the host vehicle 1 based on the specific object (for example, the signal color of the traffic light). On the other hand, information indicating that is output.

  The vehicle control device 130 receives a driver's operation input through the steering wheel 132, the accelerator pedal 134, and the brake pedal 136, and controls the host vehicle 1 by transmitting it to the steering mechanism 142, the drive mechanism 144, and the brake mechanism 146. In addition, the vehicle control device 130 controls the drive mechanism 144 and the braking mechanism 146 in accordance with instructions from the outside environment recognition device 120.

  Hereinafter, the configuration of the outside environment recognition device 120 will be described in detail. Here, the procedure for specifying a traffic signal as a control input, which is characteristic of the present embodiment, and the travel trajectory that is the trajectory of the host vehicle 1 that is the premise thereof will be described in detail, and are irrelevant to the features of the present embodiment. The description of the configuration is omitted.

(Vehicle environment recognition device 120)
FIG. 2 is a functional block diagram showing a schematic function of the outside environment recognition device 120. As shown in FIG. 2, the vehicle exterior environment recognition device 120 includes an I / F unit 150, a data holding unit 152, and a central control unit 154.

  The I / F unit 150 is an interface for performing bidirectional information exchange with the imaging device 110 and the vehicle control device 130. The data holding unit 152 includes a RAM, a flash memory, an HDD, and the like. The data holding unit 152 holds various pieces of information necessary for the processing of each function unit described below, and temporarily holds image data received from the imaging device 110. To do.

  The central control unit 154 is configured by a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing a program, a RAM as a work area, and the like, and through the system bus 156, an I / F unit 150, a data holding unit 152 and the like are controlled. In the present embodiment, the central control unit 154 also functions as an image processing unit 160, a three-dimensional position information generation unit 162, an object specifying unit 164, an object excluding unit 166, and a control instruction output unit 168. In the following, detailed operations of such functional units will be described in the order of image processing, target object specifying process, target object excluding process, and control instruction output process based on the general purpose.

(Image processing)
The image processing unit 160 acquires image data from each of the two imaging devices 110, and selects a block corresponding to a block arbitrarily extracted from one image data (for example, an array of horizontal 4 pixels × vertical 4 pixels) as the other image data. The parallax is derived using so-called pattern matching that is searched from the above. Here, “horizontal” indicates the horizontal direction of the captured luminance image, and “vertical” indicates the vertical direction of the captured luminance image.

  As this pattern matching, it is conceivable to compare the luminance (Y color difference signal) in units of blocks indicating an arbitrary image position between two pieces of image data. For example, SAD (Sum of Absolute Difference) that takes the difference in luminance, SSD (Sum of Squared Intensity Difference) that uses the difference squared, or NCC that takes the similarity of the variance value obtained by subtracting the average value from the luminance of each pixel There are methods such as (Normalized Cross Correlation). The image processing unit 160 performs such a block-unit parallax derivation process for all blocks displayed in the detection area (for example, horizontal 600 pixels × vertical 180 pixels). Here, the block is assumed to be horizontal 4 pixels × vertical 4 pixels, but the number of pixels in the block can be arbitrarily set.

  However, the image processing unit 160 can derive the parallax for each block, which is a unit of detection resolution, but cannot recognize what kind of target object the block is. Accordingly, the disparity information is derived independently not in units of objects but in units of detection resolution (for example, blocks) in the detection region. Here, an image in which the parallax information derived in this way (corresponding to a relative distance described later) is associated with image data is referred to as a distance image.

  FIG. 3 is an explanatory diagram for explaining the luminance image 210 and the distance image 212. For example, it is assumed that a luminance image (image data) 210 as illustrated in FIG. 3A is generated for the detection region 214 through the two imaging devices 110. However, only one of the two luminance images 210 is schematically shown here for easy understanding. In the present embodiment, the image processing unit 160 obtains a parallax for each block from such a luminance image 210 and forms a distance image 212 as shown in FIG. Each block in the distance image 212 is associated with the parallax of the block. Here, for convenience of description, blocks from which parallax is derived are represented by black dots.

  Returning to FIG. 2, the three-dimensional position information generation unit 162 uses the so-called stereo method to calculate disparity information for each block in the detection region 214 based on the distance image 212 generated by the image processing unit 160. Convert into 3D position information including horizontal distance, height and relative distance. Here, the stereo method is a method of deriving a relative distance of the target object from the imaging device 110 from the parallax of the target object by using a triangulation method. At this time, the three-dimensional position information generation unit 162 determines the target based on the relative distance of the target part and the detected distance on the distance image 212 between the point on the road surface and the target part at the same relative distance as the target part. The height of the part from the road surface is derived. Since various known techniques can be applied to the relative distance deriving process and the three-dimensional position specifying process, description thereof is omitted here.

(Object identification processing)
The target object specifying unit 164 specifies which target object (pixel or block) in the detection region 214 corresponds to using the luminance based on the luminance image 210 and the three-dimensional position information based on the distance image 212. . The object specifying unit 164 functions as various specifying units depending on the object to be specified. For example, in the present embodiment, the light emitters (lamps) of one or more traffic lights located in front of the host vehicle 1 and the signal colors (red signal color, yellow signal color, blue signal) of each of the one or more light emitters. It functions as a traffic light identifying unit that identifies the color.

  FIG. 4 is an explanatory diagram for explaining the operation of the traffic light specifying unit. Here, the procedure for specifying the red signal color of the traffic light by the traffic signal specifying unit will be described as an example and the specifying procedure will be described. First, the traffic light identifying unit determines that the luminance of an arbitrary target part in the luminance image 210 is a luminance range (for example, a reference value is luminance (R) and luminance (G) is a reference). It is determined whether the value (R) is 0.5 times or less and the luminance (B) is 0.38 times or less of the reference value (R). And if it is contained in the brightness | luminance range used as object, the identification number which shows the said target object will be attached | subjected to the object site | part. Here, as shown in the enlarged view of FIG. 4, the identification number “1” is given to the target portion corresponding to the target object (light emitting body of red signal color).

  Next, the traffic signal specifying unit uses an arbitrary target part as a base point, and a difference between the target part and a horizontal distance and a height difference (which may further include a relative distance difference) are within a predetermined range. The target parts that are considered to correspond to the same target object (with the same identification number) are grouped, and the target parts are also made into an integrated target part group. Here, the predetermined range is represented by a distance in the real space, and can be set to an arbitrary value (for example, 1.0 m). In addition, the traffic light identifying unit also has a target object (a red signal color of a red signal color) that has a horizontal distance difference and a height difference within a predetermined range with respect to a target part newly added by grouping. Target regions having the same illuminant are grouped. As a result, if the distances between the target parts with the same identification number are within a predetermined range, all the target parts are grouped. Here, as shown in the enlarged view of FIG. 4, the target part group 220 is formed by grouping the target parts with the identification number “1”.

  Subsequently, the traffic signal specifying unit includes a group of target region group 220 in which a height range (for example, 4.5 to 7.0 m) and a width range (for example, 0.05 to 0.00) associated with the target object. 2m), a depth range (for example, 60 m to 120 m), and the like are determined. If the predetermined condition is satisfied, the grouped target part group 220 is determined as a target (light emitting body of red signal color). The reason why the depth range is determined is that the time difference from when the object (light emitting body of red signal color) is specified until the own vehicle 1 is controlled is considered. This is because if the object is specified for the first time after approaching the traffic light, the control of the host vehicle 1 may be delayed. In addition, here, an example in which a red signal color is specified as an object has been described, but it goes without saying that a traffic light specifying unit can also specify a yellow signal color, a blue signal color, and the like.

  Moreover, when the target part group 220 has the characteristic peculiar to the target object, it may be determined as a target object on the condition. For example, when the light emitter of the traffic light is configured by an LED (Light Emitting Diode), the light emitter blinks at a cycle (for example, 100 Hz) that cannot be grasped by human eyes. Therefore, the traffic light specifying unit can determine the target (red signal color) based on the change in the time direction of the luminance of the target portion of the luminance image 210 acquired asynchronously with the blinking timing of the LED.

(Object exclusion process)
FIG. 5 is a diagram showing a traffic signal in which an automobile traffic signal and a tramway traffic signal are provided. As shown in FIG. 5, in the traffic light 300, for example, a traffic signal 320 for a streetcar provided for a streetcar is provided in addition to a traffic signal 310 for an automobile. In such a traffic light 300, an automobile traffic signal 310 is generally installed above (vertically above) a streetcar traffic signal 320. The vehicle traffic signal 310 is provided with a blue signal color light emitter 311, a yellow signal color light emitter 312 and a red signal color light emitter 313. The streetcar traffic signal 320 includes, for example, an orange signal color light emitter 321 that has an arrow shape facing left, a red signal color light emitter 322 that has an “x” shape indicating a stop instruction, and a right direction. A light-emitting body 323 having an orange signal color in the shape of an arrow facing the direction is provided. However, the number of light emitters of the streetcar traffic signal 320 and the relative position in the horizontal direction with respect to the automobile traffic signal 310 are not limited to this, and various modes are generally employed.

  Here, when the light emitter 322 of the traffic light 320 for the streetcar is lit, the object specifying unit 164 outputs the signal color of any light emitter of the traffic light 310 for the vehicle and the light emitter 322 of the traffic light 320 for the streetcar. The red signal color is specified as an object. If it does so, in the conventional vehicle control apparatus, there exists a possibility that the own vehicle 1 may be controlled based on the red signal color of the light-emitting body 322 of the traffic signal 320 for streetcars. In addition, the method of specifying the signal color of the traffic light with respect to the own vehicle 1 by specifying the shape of the light emitter 322 is also conceivable. However, in order to specify the shape of the light emitter 322, the host vehicle 1 must approach the light emitter 322 to some extent depending on the resolution and the imaging environment of the imaging device 110, and the above-described depth range (for example, 60 m) ~ 120 m), it is difficult to specify the shape of the light emitter 322. Although it is conceivable to bring the depth range closer to the own vehicle 1, in that case, as described above, the control on the own vehicle 1 may be delayed or the vehicle control may not be performed.

  Therefore, when the predetermined condition is satisfied, for example, when the light emitters of the plurality of traffic lights are specified, and the light emitter of the red signal color is specified, the object excluder 166 emits the red signal color. And a red signal color light emitter from a target that can be controlled, provided that there is a light signal emitter of another signal color above the red signal light emitter. exclude. Hereinafter, specific specific object exclusion processing will be described.

  For example, in FIG. 5, if the light emitter 311 of the automobile traffic light 310 and the light emitter 322 of the streetcar traffic light 320 are turned on, the object specifying unit 164 displays the blue signal color of the light emitter 311, and The red signal color of the light emitter 322 is specified. In such a case, in the specific object exclusion process, the relative distance between the light emitter 311 and the light emitter 322 is the same distance, and the light emitter 311 is located above the light emitter 322. 322 is excluded from objects that can be controlled.

  FIG. 6 is a flowchart for explaining the object exclusion process by the object exclusion unit 166. Referring to FIG. 6, the object excluding unit 166 first checks the number of light emitters (signal colors) specified by the object specifying unit 164 (S400).

  And the target object exclusion part 166 determines whether the number of confirmed light-emitting bodies is 1 or less (S402). As a result, if it is determined that the number of confirmed light emitters is 1 or less (YES in S402), target object excluding unit 166 ends the object excluding process.

  On the other hand, if it is determined that the number of confirmed illuminants is not 1 or less (NO in S402), the object excluding unit 166 confirms the signal color of the illuminant identified by the object identifying unit 164 (S404). ).

  Then, the object exclusion unit 166 determines whether there is at least one red signal color among the confirmed signal colors (S406). As a result, if it is determined that there is no red signal color among the confirmed signal colors (NO in S406), object exclusion unit 166 ends the object exclusion process.

  On the other hand, if it is determined that there is a red signal color among the confirmed signal colors (YES in S406), object exclusion unit 166 is based on the three-dimensional position derived by three-dimensional position information generation unit 162. The position of each of the other light emitters within a predetermined range is searched using one light signal light emitter of one or a plurality of red signal colors as a reference (S408). The predetermined range is, for example, 1 m in the horizontal direction with reference to the red signal light emitter.

  Subsequently, the object excluding unit 166 has a relative distance that is the same as the light emitting body of the reference red signal color based on the searched position of the other light emitting body and has the red signal color of the reference. It is determined whether there is a light emitter located above the light emitter (S410). Note that the same relative distance is, for example, within 1 m in the front-rear direction from the reference red signal color.

  As a result, if there is a light emitter that has the same relative distance as the light emitter of the reference red signal color and is located above the light emitter of the reference red signal color (YES in S410). The object excluding unit 166 excludes the illuminant of the reference red signal color from the objects that can be controlled (S412). On the other hand, if there is no illuminant having the same relative distance as the illuminant of the reference red signal color, or there is no illuminant located above the illuminant of the reference red signal color (S410). In step S414, the target object excluding unit 166 maintains the reference red light-emitting body as it is without excluding it from the target object that can be controlled.

  Next, the target object exclusion unit 166 determines whether or not there are a plurality of confirmed red light-emitting illuminants, and the processes of Steps S408 to S414 have been performed for all of them (S416). If there is a light-emitting body with a red signal color that has not been subjected to the processing in steps S408 to S414 (YES in S416), the object excluding unit 166 returns to the processing in step S408 and performs the processing in steps S408 to S414. If there is no illuminant having a red signal color that has not been performed (NO in S416), the object excluding unit 166 ends the object excluding process.

  In this way, the object exclusion unit 166 has a relative distance that is the same as the red signal color light emitter when a plurality of light emitters are specified and a red signal color light emitter is specified, and On the condition that there is a light emitter located above the light emitter of the red signal color, the light emitter of the red signal color is excluded from the object. Thus, like the traffic light 300 shown in FIG. 5, the car traffic signal 310 and the streetcar traffic signal 320 are provided side by side, and even if the light emitter 322 of the streetcar traffic light 320 is lit, the red signal color Can be excluded from the object. Thereby, in the control instruction output process described later, it is possible to prevent the instruction information for stopping the host vehicle 1 from being output to the vehicle control device 130 based on the light emitting body 322 of the red signal color.

(Control instruction output processing)
The control instruction output unit 168 controls the host vehicle 1 based on the specific object, with the signal color of the illuminant specified by the object specifying unit 164 and not excluded by the object excluding unit 166 as the specific object. Instruction information is output to the vehicle control device 130. For example, when the signal color of the light emitter specified by the object specifying unit 164 is a red signal color, instruction information for stopping the host vehicle 1 is output to the vehicle control device 130, and the host vehicle 1 is stopped. Let

  As described above, in the outside environment recognition device 120 according to the present embodiment, even if the traffic signal for a car and the traffic signal for a streetcar are provided side by side, a light emitter with a red signal color and a light emitter with another signal color. Are in the same relative distance and there is a light emitter of another signal color above the light emitter of the red signal color, the light emitter of the red signal color is excluded from the object. Accordingly, it is possible to prevent the signal color of the traffic light for the tram from being erroneously specified as an object, and it is possible to further increase the accuracy of specifying the traffic light.

  Note that some of the traffic lights for automobiles are provided with arrow-shaped light emitters that indicate the direction in which they can travel below the light emitters of the blue signal color, the yellow signal color, and the red signal color. The type of illuminant emits light in a blue signal color, and the illuminant of red signal color is not excluded by the above-described object exclusion process.

  In addition, in a traffic signal for an automobile, a blue signal color light emitter having an arrow shape that indicates a travelable direction is provided above the blue signal color, yellow signal color, and red signal color light emitters. is there. In such a case, the red signal color illuminant and the arrow-shaped blue signal color illuminant are turned on at the same time, so the red signal color illuminant is excluded by the object exclusion process described above. However, the vehicle 1 can be controlled by specifying the blue signal color of the arrow-shaped light emitter.

  Further, even when the pedestrian traffic light is disposed at a position where the relative distance is the same as that of the automotive traffic signal, the pedestrian traffic light is disposed below the automotive traffic signal. Therefore, even if the red signal color of the pedestrian traffic light is specified, the red signal color is excluded from the target object by the above-described target object exclusion process, and the car traffic signal is specified. It becomes possible to raise.

  Also provided are a program that causes a computer to function as the vehicle exterior environment recognition device 120 and a computer-readable storage medium that stores the program, such as a flexible disk, magneto-optical disk, ROM, CD, DVD, and BD. Here, the program refers to data processing means described in an arbitrary language or description method.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  It should be noted that each step of the object exclusion process described above does not necessarily have to be processed in time series in the order described as a flowchart, and may include processes in parallel or by a subroutine.

  The present invention relates to a vehicle environment recognition device that recognizes the environment outside the host vehicle, and can be used particularly for a vehicle environment recognition device that is suitable for identifying a traffic light.

DESCRIPTION OF SYMBOLS 1 Vehicle 120 Outside-vehicle environment recognition apparatus 162 Three-dimensional position information generation part 164 Target object specific part (traffic signal specific part)
166 Object exclusion part

Claims (2)

A light emitter of a traffic light, and a traffic light specifying unit for specifying a signal color of the light emitter;
A position information generator for deriving the position of the light emitter;
When a plurality of the light emitters are detected and at least one of the light emitters has a red signal color, the relative distance is the same as the light emitter of the red signal color with reference to the light emitter of the red signal color. Object exclusion that excludes the red signal color light emitter from the object to be controlled, provided that there is another light signal light emitter above the red signal color light emitter at a distance And
A vehicle exterior environment recognition device comprising: The traffic signal includes a traffic signal for an automobile and a traffic signal for a moving body other than the automobile,
The vehicle exterior environment recognition device according to claim 1, wherein the vehicle traffic signal is disposed above a traffic signal for a moving body other than the vehicle.