JP2006040123A - Recognition support device for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make a driver properly recognize presence of an object relatively at high risk to his/her own vehicle. <P>SOLUTION: An object image segmentation part 23 segments an image of the object extracted by an object extraction part 21 from image data to be inputted from an outside sensor 12 to generate a segmented image. A segmented image display control part 32 calculates the presence direction of the object viewed from the driver based on information on the presence direction of the object calculated by an object direction calculation part 29 and a detection signal regarding the sense of the driver's head to be outputted from a directional sensor 18. The segmented image display control part 32 outputs the segmented image generated by the object image segmentation part 23 and the information on the presence direction of the object viewed from the driver to a sub-display device 17 at least when it is judged that danger of visibility calculated by a visual inspection danger judgment part 25 exceeds predetermined danger or when it is judged that risk calculated by a risk judgment part 28 exceeds predetermined risk. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a vehicle recognition support apparatus.

Conventionally, for example, a camera or a radar detects an object existing outside the host vehicle, and based on the detection result, the driver sees, for example, a transmissive liquid crystal display device mounted on the front windshield or on the driver's head. There is known an information display device that displays a bright spot at a position superimposed on an object (see, for example, Patent Document 1).
Conventionally, for example, a blind spot area of a driver of the host vehicle is photographed, and a captured image of the blind spot area is displayed on a display device, and an alarm sound output and warning information are superimposed on the display device, thereby A vehicle periphery monitoring device that alerts a driver to a captured image is known (for example, see Patent Document 2).
Japanese Patent Laid-Open No. 10-246640 JP 2001-18717 A

By the way, in the device according to the above-described prior art, when the driver's line-of-sight position is not in an object outside the host vehicle or the blind spot area, simply performing a bright spot display and an alarm output to alert the driver, For example, there is a risk that excessive bright spot display or warning output may be executed on a road with a relatively heavy traffic. Further, on the front windshield or on the display screen, if only bright spots are displayed at positions around an object that can be an obstacle to the host vehicle, for example, a plurality of points are superimposed so as to be superimposed along the driver's line of sight. When there is an object, bright spots are displayed on these multiple objects, and the driver can instantly recognize which of the multiple objects is more dangerous than the host vehicle. The problem that cannot be done.
The present invention has been made in view of the above circumstances, and provides a vehicle recognition support device that can allow a driver to appropriately recognize the presence of an object having a relatively high degree of danger with respect to the host vehicle. Objective.

  In order to solve the above-described problems and achieve the object, the vehicle recognition support apparatus according to the first aspect of the present invention is a photographing means for photographing the outside of the own vehicle (for example, an outside-field photographing camera in the embodiment). 12a), and a display device (for example, the front display device 15 and the rear display device 16 in the embodiment) provided in the own vehicle and a photographed image obtained by photographing by the photographing means are displayed on the display device. An object recognition support device for a vehicle, the object extracting means for extracting an object from a photographed image obtained by photographing by the photographing means (for example, the object extracting unit 21 in the embodiment), and the object extracting means Image cutting means (for example, the object image cutting unit 23 in the embodiment) that cuts out the image of the object extracted in step S1 from the captured image, and the risk of determining the risk of the object with respect to the host vehicle Degree determination means ( For example, the risk determination unit 28) in the embodiment and a sub display device (for example, the sub display device 17 in the embodiment) provided at a position different from the display device, the risk determination When the degree of risk determined by the means exceeds a predetermined value set in advance, an image of the object cut out by the image cut-out means is displayed on the sub-display device.

  According to the vehicle recognition support device having the above-described configuration, in addition to the display device that displays the captured image obtained by the photographing of the photographing unit, the auxiliary display device is provided, and when the degree of danger exceeds a predetermined value, the object extraction unit By displaying the cut-out image of the object extracted in step 3 on the sub-display device, it is possible to prevent the driver from overlooking or mistaken the object and to recognize the object by the driver. Can be supported accurately.

  Furthermore, in the vehicle recognition support device according to the second aspect of the present invention, the auxiliary display device is mounted on a driver's head.

  According to the vehicle recognition support device having the above-described configuration, even if the orientation of the driver's head changes, the display content of the sub-display device can always be visually recognized by the driver.

  Furthermore, the vehicle recognition support apparatus according to the third aspect of the present invention is directed to direction detection means for detecting the direction in which the object is present (for example, the direction sensor 18 and the object direction calculation unit 29 in the embodiment, and the switching unit). And a direction detected by the direction detection unit together with the image of the object cut out by the image cutout unit.

  According to the vehicle recognition support apparatus having the above-described configuration, even if the relative position of the target object with respect to the host vehicle changes, the driver can accurately recognize the actual position of the target object.

  Furthermore, the vehicle recognition support apparatus according to the fourth aspect of the present invention is an enlarged image acquisition unit (for example, an enlarged image acquisition unit according to an embodiment) capable of enlarging an image of the object or enlarging an image of the object. 22).

  According to the vehicle recognition support apparatus configured as described above, the driver may miss an object by displaying an enlarged image of the object extracted by the object extraction unit on the display device or the sub display device. Thus, it is possible to prevent the driver from making a mistake and to accurately support the recognition operation of the object by the driver.

According to the vehicle recognition support apparatus of the present invention, the driver can overlook the object or make a mistake by displaying the cut-out image of the object extracted by the object extraction unit on the sub-display device. It is possible to accurately support the driver's recognition of the object.
Furthermore, according to the vehicle recognition support device of the invention described in claim 2, even if the orientation of the driver's head changes, the driver can always make the display content of the sub-display device visible. it can.
Furthermore, according to the vehicle recognition support device of the invention described in claim 3, even if the relative position of the object with respect to the host vehicle changes, the driver can accurately recognize the actual position of the object. be able to.
Furthermore, according to the vehicle recognition support device of the invention described in claim 4, it is possible to prevent the driver from overlooking the object or to make a mistake, thereby accurately recognizing the object by the driver. Can help.

Hereinafter, a vehicle recognition support apparatus according to an embodiment of the present invention will be described with reference to the accompanying drawings.
For example, as shown in FIG. 1, the vehicle recognition support device 10 according to the present embodiment includes a processing device 11 configured by an electronic circuit including a CPU and the like, and an external sensor 12 for detecting the external environment of the host vehicle, The object sensor 13, the vehicle state quantity sensor 14, the front display device 15, the rear display device 16, the sub display device 17, and the direction sensor 18 are configured.

The external sensor 12 includes an external camera 12a and an image processing unit 12b, which are composed of, for example, a CCD camera or a C-MOS camera that can capture images in the visible light region or the infrared region.
The external shooting camera 12a is, for example, a camera for front shooting that captures the outside of a predetermined detection range in front and side of the traveling direction of the host vehicle through the front window, and rear and side of the traveling direction of the host vehicle through the rear window, for example. The camera is configured to include a rear-facing camera that captures the outside of the predetermined detection range.
The image processing unit 12b performs predetermined image processing such as filtering and binarization processing on the captured image obtained by capturing with the external imaging camera 12a, and generates image data including a two-dimensional array of pixels. To the processing device 11.

The object sensor 13 includes, for example, an object photographing camera 13a and an image processing unit 13b made up of a CCD camera, a C-MOS camera, or the like that can image in the visible light region or the infrared region, and is input from the processing device 11. The detection range can be changed according to the control signal, and the object within the detection range can be photographed at an appropriate magnification.
The object photographing camera 13a photographs an appropriate object existing in the external environment of the host vehicle through, for example, a front window or a rear window.
The image processing unit 13b performs predetermined image processing such as filtering and binarization processing on a captured image obtained by capturing with the object photographing camera 13a, and generates image data including pixels in a two-dimensional array. And output to the processing device 11.

The vehicle state quantity sensor 14 is a positioning signal such as a GPS (Global Positioning Systems) signal for measuring the position of the host vehicle using an artificial satellite, or a position signal transmitted from an information transmitter outside the host vehicle. A position sensor that detects the current position of the host vehicle by reception, for example, a magnetic action with a base point marker disposed on the road, or the like, a sensor that detects a vehicle state quantity related to the traveling state of the host vehicle, such as a vehicle speed sensor, a rudder A detection signal output from each device and sensor is output to the processing device 11.
The vehicle speed sensor detects, for example, the vehicle travel distance per predetermined unit processing time, that is, the speed (vehicle speed) of the host vehicle based on, for example, the rotational speed of the wheels, and the steering angle sensor is, for example, a steering shaft (not shown). It consists of a provided rotary encoder or the like, and detects the direction and magnitude of the steering angle input by the driver. The gyro sensor indicates the direction of the host vehicle in a horizontal plane, a tilt angle with respect to the vertical direction (for example, a yaw angle that is a rotation angle about the vertical axis of the center of gravity of the vehicle), a change amount of the tilt angle (for example, a yaw rate), and the like. To detect.

For example, as shown in FIG. 2, the front display device 15 is a liquid crystal display device or the like disposed on the upper part of the instrument panel in the vehicle interior, and among the image data obtained by photographing with the external photography camera 12 a, In addition to displaying image data of the outside world in a predetermined detection range forward and sideward in the traveling direction of the host vehicle, as described later, it was obtained by photographing with the object photographing camera 13a in accordance with control processing in the processing device 11. Display image data.
For example, as shown in FIG. 2, the rear / side display device 16 is a liquid crystal display device or the like disposed at a position near the room mirror on the vehicle interior side of the front portion of the roof panel. Among the obtained image data, in particular, the image data of the outside of the predetermined detection range behind and in the direction of travel of the host vehicle is displayed, and an object according to the control processing in the processing device 11 as described later. The image data obtained by photographing with the photographing camera 13a is displayed.
Note that the rear / side display device 16 is provided with a camera for rearward shooting of the external camera 12a, assuming a state equivalent to a state in which the driver visually recognizes the rear of the host vehicle through the rearview mirror. An image obtained by performing mirror conversion on the captured image obtained by photographing is displayed.

For example, as shown in FIG. 3, the sub display device 17 is a liquid crystal display device or the like disposed at the tip of a headset 19 that can be mounted on the head of the driver D. Of the image data obtained by shooting 12a, the cut image cut out by the processing device 11 is displayed.
The headset 19 is provided with a direction sensor 18 including, for example, a gyro sensor, and a detection signal output from the direction sensor 18 is output to the processing device 11. This detection signal is used in processing for detecting the direction of the head of the driver D wearing the headset 19.

  The processing device 11 includes an object extraction unit 21, an enlarged image acquisition unit 22, an object image cutout unit 23, a sign determination unit 24, a visual difficulty determination unit 25, and a host vehicle movement state calculation unit 26. The object movement state calculation unit 27, the risk determination unit 28, the object direction calculation unit 29, the composite image generation unit 30, the image display control unit 31, and the cut-out image display control unit 32 are provided. It is configured.

The object extraction unit 21 performs recognition processing such as feature amount calculation and shape determination on the image data input from the external sensor 12 using, for example, another vehicle, a pedestrian, or a sign as a detection object.
In the image data recognition processing, in the feature amount calculation processing, for example, extraction and labeling of detection objects based on pixel continuity is performed on the image data after binarization processing, and the extracted detection objects The center of gravity and area of the image, the aspect ratio of the circumscribed rectangle, and the like are calculated. In the shape determination process, for example, a search on image data is performed based on a predetermined pattern (for example, an outline) stored in advance, and a detection target is extracted according to the similarity to the predetermined pattern.

The magnified image acquisition unit 22 acquires a magnified image of the target object extracted by the target object extraction unit 21, and a control signal that instructs the shooting direction and the shooting magnification of the target shooting camera 13 a of the target sensor 13. Is output to the object sensor 13, and image data obtained by photographing with the object photographing camera 13 a, that is, an enlarged image of the object is acquired and output to the image display control unit 31.
The object image cutout unit 23 cuts out the image of the object extracted by the object extraction unit 21 from the image data input from the external sensor 12 to generate a cutout image, and the cutout image display control unit 32. Output to.

The sign determination unit 24 determines whether or not the object extracted by the object extraction unit 21 is a road sign composed of signs such as guidance, warning, regulation, instruction, assistance, and the like. Is output to the image display control unit 31.
The visual recognition difficulty level determination unit 25 detects illuminance or color contrast in the image data input from the external sensor 12 or between the target object extracted by the target object extraction unit 21 and the surrounding area of the target object. The complexity of the shape of the edge image of the contour portion is detected, and the difficulty level of visibility is calculated according to the detection results.
For example, the visibility difficulty level determination unit 25 may change the visibility difficulty level to increase as the illuminance or color contrast decreases or as the complexity of the shape of the edge image increases. Set. Then, the visual recognition difficulty level determination unit 25 determines whether the calculated visibility difficulty level exceeds a predetermined difficulty level, and outputs the determination result to the image display control unit 31.

The own vehicle movement state calculation unit 26 performs a calculation process based on a positioning signal output from the vehicle state quantity sensor 14, and an autonomous navigation calculation process based on detection signals output from the vehicle speed sensor and the gyro sensor of the vehicle state quantity sensor 14. To calculate the current position of the vehicle. Further, the host vehicle movement state calculation unit 26 estimates the traveling locus of the host vehicle based on detection signals (for example, vehicle speed and yaw rate) output from the vehicle speed sensor and the gyro sensor of the vehicle state quantity sensor 14, and the degree of risk. It outputs to the determination part 28.
The object movement state calculation unit 27 is based on the change in the position of the object in the image data extracted by the object extraction unit 21 and the current position of the host vehicle calculated by the host vehicle movement state calculation unit 26. Then, the relative position and the relative speed with respect to the host vehicle are calculated as the moving state of the object, and the traveling locus of the object is estimated and output to the risk determination unit 28.

The degree-of-risk determination unit 28 estimates the current position and travel locus of the host vehicle input from the host vehicle movement state calculation unit 26, and estimates of the target travel track input from the object movement state calculation unit 27. On the basis of the relative position and the relative speed, the degree to which the target object interferes with the traveling of the host vehicle, that is, the risk level is calculated. For example, the risk determination unit 28 sets the risk to change in an increasing tendency as the relative speed of the object increases.
Then, the risk determination unit 28 determines whether or not the calculated risk exceeds a predetermined risk, and outputs the determination result to the image display control unit 31 and the cut image display control unit 32.

  The target object direction calculation unit 29 calculates the presence direction of the target object viewed from the own vehicle based on the position of the target object extracted by the target object extraction unit 21 in the image data, and the cut image display control unit To 32.

The composite image generation unit 30 is provided in, for example, the image display control unit 31 and generates a composite image by combining the image data input from the external sensor 12 with the enlarged image of the object input from the enlarged image acquisition unit 22. .
For example, as shown in FIG. 4, a pedestrian A1 trying to cross the traveling path R of the own vehicle or a stopped vehicle C1 as an object in the image data P ahead of the traveling direction of the own vehicle input from the external sensor 12 A pedestrian A2 who is about to enter the travel path R of the own vehicle from behind the vehicle, another vehicle A3 who is about to enter the travel path R of the own vehicle from behind the building C2, and a travel path of the own vehicle When a regulation sign A4 arranged in front of R and a guide sign A5 arranged in front of the traveling path R of the host vehicle are extracted, enlarged images B1,..., B5 of the objects A1,. It is synthesized at an appropriate position in the image data P, for example, a position that does not hinder recognition of the traveling path R.

  Then, the image display control unit 31 determines that at least the visibility difficulty calculated by the visual recognition difficulty determination unit 25 exceeds the predetermined difficulty, or the risk calculated by the risk determination unit 28. When it is determined that the degree exceeds the predetermined risk level, the composite image generated by the composite image generation unit 30 is output to the front display device 15 or the rear display device 16. On the other hand, when it is determined that the visibility difficulty calculated by the visibility difficulty determination unit 25 is equal to or less than the predetermined difficulty and the risk calculated by the risk determination unit 28 is equal to or less than the predetermined risk. Only the image data input from the external sensor 12 is output to the front display device 15 or the rear display device 16.

  For example, in FIG. 4, when the travel trajectory α estimated for the host vehicle and the travel trajectories β1, β2, β3 estimated for the objects A1, A2, A3 intersect, The risk calculated for the objects A1, A2 and A3 exceeds a predetermined risk, and the illuminance and color of the surrounding area of the object during night driving, for example, the object A1, When the contrast is low, or by being blocked by other objects such as the objects A2 and A3, only a part of the shape of the object such as a pedestrian or another vehicle is detected, and the shape of the edge image of the contour portion The complexity of visibility calculated for the objects A1, A2, A3 exceeds the predetermined difficulty, and the objects A1, A1 are placed at appropriate positions in the image data P. Enlarged images B1, B2, B of A2, A3 There synthesized image synthesized is displayed on the front display unit 15 or the rear display unit 16.

For example, when it is determined that the visibility difficulty exceeds a predetermined difficulty due to low illuminance and color contrast with respect to the surrounding area as in the object A1, the periphery of the object A1 is appropriately processed. An enlarged image B1 having a high illuminance and color contrast with respect to the region is combined with the image data P.
Further, for example, when it is determined that the risk calculated for each of the objects A1, A2, and A3 exceeds a predetermined risk, each of the objects A1, A2, and A3 is set according to the risk of each object. The display states of the outer peripheral portions D1, D2, and D3 of the enlarged images B1, B2, and B3 are changed. For example, when the danger level is relatively low, the colors of the outer peripheral portions D1, D2, and D3 are yellow. When the degree of danger is relatively medium, the color of each outer peripheral part D1, D2, D3 is set to red. For example, when the degree of danger is relatively high, each outer peripheral part D1, D2, The color of D3 is set to red and is set to blink at a predetermined cycle.

Further, the image display control unit 31 determines that the object extracted by the object extraction unit 21 in the sign determination unit 24 is a road sign made up of signs such as guidance, warning, regulation, instruction, and assistance. If it is, the composite image generated by the composite image generation unit 30 is output to the front display device 15 or the rear display device 16, and even after the host vehicle has passed the position of the road sign, The output of the composite image is continued for a predetermined time.
For example, in FIG. 4, when the target object A4 of the restriction sign and the target object A5 of the guide sign are detected at a position ahead of the traveling path R of the host vehicle, an appropriate position in the image data P is detected. A combined image obtained by combining the enlarged images B4 and B5 of the objects A4 and A5 is displayed on the front display device 15 or the rear display device 16. The display of the composite image is continued for a predetermined time after the host vehicle passes through the objects A4 and A5.

  The clipped image display control unit 32 determines that the visibility difficulty calculated by the visual difficulty determination unit 25 exceeds a predetermined difficulty, or the risk calculated by the risk determination unit 28. When it is determined that the degree exceeds the predetermined risk level, the cut image generated by the object image cutout unit 23 is output to the sub display device 17. Here, the cut-out image display control unit 32 detects information related to the direction of the presence of the object calculated by the object direction calculation unit 29 and the direction of the head of the driver D output from the direction sensor 18. Based on the signal, the presence direction of the object viewed from the driver D is calculated, and information on the presence direction is output to the sub display device 17.

For example, as shown in FIGS. 4 and 5, when the traveling locus α estimated for the host vehicle and the traveling locus β1 estimated for the object A1 intersect, The calculated risk level exceeds the predetermined risk level, and when the illuminance or color contrast with respect to the surrounding area of the target object is low, such as when the target object A1 is running at night, the target object The visibility difficulty calculated for A1 exceeds a predetermined difficulty, and the cut-out image E1 of the object A1 cut out from the image data P indicates the direction of presence of the object A1 as seen from the driver D. It is displayed on the sub display device 17 together with the arrow image F1 shown.
For example, when it is determined that the visibility difficulty exceeds a predetermined difficulty due to low illuminance and color contrast with respect to the surrounding area as in the object A1, the periphery of the object A1 is appropriately processed. A cutout image E <b> 1 with high illuminance and color contrast for the region is output to the sub display device 17.

For example, as shown in FIG. 6, there is a pedestrian G1 that crosses a traveling path R on which the host vehicle V travels after a left turn in a region where the driver's blind spot is caused by the front pillar FP of the host vehicle V that turns left at an intersection. In this case, it is determined that the visibility difficulty calculated by the visual difficulty determination unit 25 for the pedestrian G1 exceeds a predetermined difficulty, and the risk calculated by the risk determination unit 28 It is determined that the degree exceeds the predetermined risk level. In the front display device 15, an image J1 of the pedestrian G1 is displayed in the vicinity of a position that intersects a direction substantially equivalent to the line-of-sight direction H1 from the driver toward the pedestrian G1. Further, the sub-display device 17 displays a cutout image K1 of the pedestrian G1 and an arrow image L1 indicating the direction of the pedestrian G1 as seen from the driver.
In this case, for example, if it is determined that the pedestrian G1 has a relatively moderate risk, the color of the outer periphery of the image J1 of the pedestrian G1 is set to red, and the direction in which the pedestrian G1 exists Is set to red.

For example, as shown in FIG. 7, when there is another vehicle VA that enters the intersection in the area where the driver's blind spot is caused by the front pillar FP of the host vehicle V that enters the intersection, the other vehicle VA On the other hand, it is determined that the visibility difficulty level calculated by the visibility difficulty level determination unit 25 exceeds the predetermined difficulty level, and the risk level calculated by the risk level determination unit 28 exceeds the predetermined risk level. It is determined that Then, in the front display device 15, an image J2 of the other vehicle VA is displayed in the vicinity of a position that intersects a direction substantially equal to the line-of-sight direction H2 from the driver toward the other vehicle VA. Further, the sub-display device 17 displays a cut-out image K2 of the other vehicle VA and an arrow image L2 indicating the direction in which the other vehicle VA is viewed from the driver.
In this case, the danger level of the other vehicle VA is determined according to, for example, the magnitude of the relative speed with respect to the host vehicle V. For example, when the danger level is relatively low, the color of the outer peripheral portion of the image J2 of the other vehicle VA. And the color of the arrow image L2 indicating the direction in which the other vehicle VA exists is set to yellow, and when the degree of danger is relatively medium, the color of the outer periphery of the image J2 and the color of the arrow image L2 are set to red. For example, when the degree of risk is relatively high, the color of the outer peripheral portion of the image J2 and the color of the arrow image L2 are set to red and set to blink at a predetermined cycle.

For example, as shown in FIG. 8, when the host vehicle V performs a lane change on a one-side multiple lane travel path, the host vehicle V is traveling in a lane behind the host vehicle V after the lane change. When the other vehicle VB exists, it is determined that the visibility difficulty calculated by the visibility difficulty determination unit 25 for the other vehicle VB exceeds the predetermined difficulty, and the risk determination unit It is determined that the risk calculated at 28 exceeds the predetermined risk. Then, in the rear display device 16, for example, when the rear display device 16 is used as a rearview mirror, the image J3 of the other vehicle VB is displayed in the vicinity of the position where the other vehicle VB exists. Further, the sub-display device 17 displays a cutout image K3 of the other vehicle VB and an arrow image L3 indicating the direction in which the other vehicle VB is seen from the driver.
In this case, the danger level of the other vehicle VB is determined according to, for example, the magnitude of the relative speed with respect to the host vehicle V. For example, when the danger level is relatively low, the color of the outer periphery of the image J2 of the other vehicle VB. The color of the arrow image L3 indicating the direction in which the other vehicle VB is present is set to yellow.

  The vehicle recognition support apparatus 10 according to the present embodiment has the above-described configuration. Next, an operation of the vehicle recognition support apparatus 10, particularly an operation of displaying a cut-out image on the sub display device 17 will be described.

First, for example, in step S01 shown in FIG. 9, image data of a photographed image obtained by photographing with the external camera 12a of the external sensor 12 is acquired.
Next, in step S02, the acquired image data of the captured image is displayed on the front display device 15 or the rear display device 16.
In step S03, the image data input from the external sensor 12 is subjected to recognition processing such as feature amount calculation and shape discrimination using, for example, another vehicle, a pedestrian, or a sign as a detection target. It is determined whether or not an object exists in the outside world of the vehicle.
When the determination result is “NO”, the series of processes is terminated.
On the other hand, if this determination is “YES”, the flow proceeds to step S 04.

In step S04, it is determined whether or not the object extracted from the image data is a road sign composed of signs such as guidance, warning, regulation, instruction, and assistance.
If this determination is “NO”, the flow proceeds to step S 11 described later.
On the other hand, if this determination is “YES”, the flow proceeds to step S 05.
In step S05, the extracted road sign image is cut out from the image data input from the external sensor 12 to obtain a cut-out image.
Next, in step S06, the cut image is displayed on the sub display device 17.

Next, in step S07, a relative position between the host vehicle and the road sign extracted as the object is calculated.
Next, in step S08, it is determined whether or not the host vehicle has passed the installation position of the road sign.
If this determination is “NO”, the flow returns to step S 06 described above.
On the other hand, if this determination is “YES”, the flow proceeds to step S 09.
In step S09, the cut image is continuously displayed on the sub display device 17 for a predetermined time.
In step S10, the display of the cut image on the sub display device 17 is terminated, and the series of processes is terminated.

In step S11, an image of an object other than the extracted road sign (for example, a pedestrian or another vehicle) is cut out from the image data input from the external sensor 12, and a cut image is acquired.
Next, in step S <b> 12, information on the direction of presence of the object viewed from the host vehicle calculated by the object direction calculation unit 29 and detection of the direction of the driver's head output from the direction sensor 18. Based on the signal, the presence direction of the object viewed from the driver is detected.
Next, in step S13, the movement state of the target object (for example, the relative speed with respect to the host vehicle or the estimated value of the traveling locus) is calculated.
Next, in step S14, based on the estimated value of the traveling locus of the host vehicle, the estimated value of the traveling locus of the target object, and the relative speed, the degree to which the target object hinders the traveling of the host vehicle, that is, the risk level is determined. calculate.
Next, in step S15, illuminance or color contrast is detected in the image data input from the external sensor 12, or the edge image of the contour portion between the extracted object and the peripheral area of the object is detected. The complexity of the shape is detected, and the difficulty level of visibility is calculated according to these detection results.

Next, in step S <b> 16, it is determined whether the calculated visibility difficulty level exceeds a predetermined difficulty level, or whether the calculated risk level exceeds a predetermined risk level.
If the determination result is “YES”, the process proceeds to step S17, where the cut-out image of the object and the arrow image indicating the direction of the object viewed from the driver are displayed on the sub display device 17, It returns to step S12 mentioned above.
On the other hand, if this determination is “NO”, the flow proceeds to step S 18, and if the clipped image is being displayed at this time, display of the clipped image and the arrow image on the secondary display device 17 is stopped, and a series of Terminate the process.

As described above, according to the vehicle recognition support device 10 according to the present embodiment, it is extracted at least when the degree of visibility exceeds a predetermined value or when the degree of risk exceeds a predetermined value. By combining the enlarged image of the object with the image data input from the external sensor 12 and displaying it, it is possible to prevent the driver from overlooking the object or making a mistake. It can accurately support the recognition of objects.
Furthermore, in addition to the front display device 15 and the rear display device 16 that display the image data input from the external sensor 12, a sub display device 17 is provided, and at least when the degree of visibility exceeds a predetermined value or a danger When the degree exceeds a predetermined value, a clipped image of the extracted object is displayed on the sub display device 17 to prevent the driver from overlooking or mistaken the object. The driver can accurately support the recognition of the object by the driver.

  Further, when the extracted object is a road sign, the front display device 15 or the rear display device 16 continues for a predetermined time after the passage even after the host vehicle passes the position of the road sign. By continuing to display the composite image at the sub-display and the cut-out image on the sub-display device 17, the driver can be prevented from making a mistake even when the display content is complicated on the road sign. Thus, it is possible to accurately support the recognition operation of the road sign by the driver.

In addition, by providing the headset 19 that can be mounted on the driver's head with the sub display device 17, even if the orientation of the driver's head changes, the display contents of the sub display device 17 can be operated. Can always be visually recognized.
Furthermore, the headset 19 is provided with a direction sensor 18 that detects the direction of the driver's head, and the object direction calculation unit 29 calculates the direction in the presence direction of the object as viewed from the host vehicle based on the image data. In consideration of the detection signal related to the direction of the head of the driver output from the sensor 18, the presence direction of the object viewed from the driver is calculated, and an arrow image indicating the existence direction is extracted from the target object. By displaying the image on the sub display device 17 together with the image, the driver can accurately recognize the actual position of the object even when the relative position of the object with respect to the host vehicle changes.

  In the above-described embodiment, an enlarged image of the object is obtained by optical zoom that instructs the shooting direction and magnification of the object shooting camera 13a of the object sensor 13 and changes the focal length of the object shooting camera 13a. However, the present invention is not limited to this. For example, the object sensor 13 is omitted, and among the image data input from the external sensor 12, the image area including the object is complemented and enlarged by image processing. An enlarged image may be acquired.

  In the embodiment described above, the risk determination unit 28 determines whether or not the calculated risk exceeds a predetermined risk. However, the present invention is not limited to this. For example, as shown in FIG. A determination threshold value that changes in a decreasing tendency as the relative speed increases may be set, and it may be determined whether or not the calculated risk level exceeds a determination threshold value corresponding to the relative speed.

1 is a configuration diagram of a vehicle recognition support apparatus according to an embodiment of the present invention. It is a figure which shows the inside of the vehicle carrying the front display apparatus and rear display apparatus which are shown in FIG. It is a figure which shows the subdisplay apparatus shown in FIG. It is a figure which shows an example of the image data ahead of the advancing direction of the own vehicle displayed on a front display apparatus. It is a figure which shows an example of the cut-out image and arrow image which are displayed on a subdisplay apparatus. It is a figure which shows an example of the image data ahead of the advancing direction of the own vehicle displayed on a front display apparatus, and the cut-out image and arrow image displayed on a sub display apparatus. It is a figure which shows an example of the image data ahead of the advancing direction of the own vehicle displayed on a front display apparatus, and the cut-out image and arrow image displayed on a sub display apparatus. It is a figure which shows an example of the image data ahead of the advancing direction of the own vehicle displayed on a front display apparatus, and the cut-out image and arrow image displayed on a sub display apparatus. It is a flowchart which shows operation | movement of the recognition assistance apparatus of the vehicle shown in FIG. It is a graph which shows an example of the relationship between the relative speed of the own vehicle and a target object, and the determination threshold value concerning the danger level of a target object.

Explanation of symbols

10. Vehicle recognition support device 12a External camera (photographing means)
15 Front display device (display device)
16 Rear display device (display device)
17 Sub display device 18 Direction sensor (direction detection means)
21. Object extraction unit (object extraction means)
22 Enlarged image acquisition unit (enlarged image acquisition means)
23 Object image cutout part (image cutout means)
28 Risk determination unit (Danger determination means)
29 Object direction calculation unit (direction detection means)
32 Cutout image display control unit (direction detection means)

Claims (4)

Photographing means for photographing the outside of the host vehicle, a display device provided inside the host vehicle,
A vehicle recognition support device for displaying a photographed image obtained by photographing by the photographing means on the display device,
An object extraction means for extracting an object from a photographed image obtained by photographing by the photographing means;
Image cutout means for cutting out the image of the object extracted by the object extraction means from the captured image;
A risk determination means for determining a risk of the object with respect to the host vehicle;
A sub-display device provided at a position different from the display device,
When the risk determined by the risk determination means exceeds a predetermined value set in advance, an image of the object cut out by the image cut-out means is displayed on the sub display device. Vehicle recognition support device. 2. The vehicle recognition support device according to claim 1, wherein the sub display device is mounted on a driver's head. Provided with direction detecting means for detecting the direction in which the object exists,
The vehicle recognition support device according to claim 1, wherein a direction detected by the direction detection unit is displayed on the sub display device together with an image of the object cut out by the image cutout unit. The vehicle recognition support device according to claim 1, further comprising an enlarged image acquisition unit capable of enlarging the object and enlarging an image of the object.

Images