JP5004903B2 - Vehicle periphery monitoring device - Google Patents

Description

  The present invention relates to a vehicle periphery monitoring device that monitors the periphery of a vehicle using a captured image obtained by an imaging unit mounted on the vehicle.

A region corresponding to the pedestrian's head, which is an object, is extracted from the infrared image captured by the imaging device, and the region corresponding to the pedestrian's torso is extracted based on the head region. A technique for recognizing the presence of a pedestrian and informing the driver of the vehicle of the presence of the pedestrian is known (see Patent Document 1).
Japanese Patent Laid-Open No. 11-328364

  However, there are cases where not only pedestrians (human beings) but also quadruped animals such as deers are present around the host vehicle. In this case, it is difficult to determine whether or not the object is a quadruped animal even if a method for determining whether or not the object is a human is employed. For this reason, it becomes difficult to inform the driver of the vehicle of the presence of a quadruped animal, and to drive the vehicle so that the driver avoids contact with the object.

  Then, an object of this invention is to provide the vehicle periphery monitoring apparatus which can recognize with high precision whether a target object is a quadruped animal.

A vehicle periphery monitoring device according to a first aspect of the present invention is a vehicle periphery monitoring device that monitors the periphery of a vehicle using a captured image obtained by an imaging unit mounted on the vehicle, and is a first device that represents an object from the captured image . Extracting a first object region constituted by a second high-intensity region having a luminance equal to or higher than a second threshold value included in the first high-luminance region having a luminance equal to or higher than a threshold value; A target area extracting means for extracting a second target area, which is an area where a part of the target object below exists, as a target area; and the first target area extracted by the target area extracting means . The ratio of the horizontal width to the vertical width is 1 or more , and extends from the object area extracted by the object area extracting means in the vertical direction or the direction inclined with respect to the vertical direction and downward. As you head A first element region group consisting of two pairs of element regions arranged side by side so as to widen the gap, and a pair of element regions arranged side by side so that the gap increases toward the bottom, and And a target classification unit that determines that the target object is a quadruped animal on the condition that a second group of element areas including one element area arranged side by side is extracted at different times. It is characterized by.

  According to the vehicle periphery monitoring device of the first aspect of the present invention, an object extracted from a captured image (in addition to an image captured by the imaging unit, an image obtained by applying a processing to the image is included). On the condition that the first and second element region groups are extracted at different times from the object region as the region where the object exists, it is determined that the object corresponds to a quadruped animal. The “quadruped animal” does not mean all the quadruped animals, but means a specific quadruped animal that can be classified by the method of the present invention. The first element region group is highly likely to correspond to the four legs of a laterally-facing quadruped that shows the “first posture” in which the front legs and the rear legs are open. The second element region group is highly likely to correspond to four legs of a laterally-facing quadruped that shows a “second posture” in which one of the front legs and the rear legs is open and the other is substantially aligned. . Further, when a quadruped animal that is walking or running is observed from the side, the “first posture” and the “second posture” may be confirmed at different times as the posture of the quadruped animal. high. In view of these points, it is possible to recognize with high accuracy whether or not the object represented by the high luminance region extracted from the captured image by the determination method as described above is a quadruped.

  The vehicle periphery monitoring device according to a second aspect of the present invention is the vehicle periphery monitoring device according to the first aspect, wherein the object classification means is reversed left and right compared to the second element region group and the second element region group. As a further requirement that the pair of side-by-side element regions and the third element region group consisting of the one element region arranged so that the interval is widened toward the lower side are extracted at different times. It is determined that the object corresponds to the quadruped animal.

  According to the vehicle periphery monitoring device of the second invention, the object corresponds to a quadruped animal, with the further requirement that the second element area group and the third element area group are extracted from the object area at different times. Then, it is determined. Compared to the second posture, the third element region group has a left and right arrangement of a pair of open legs of the front and rear legs and a pair of substantially aligned legs (front and rear for a quadruped animal). ) There is a high probability that it corresponds to the four legs of a sideways quadruped animal showing the "third posture" which is reversed. Further, when a quadruped animal walking or running is observed from the side, it is highly likely that the “second posture” and the “third posture” are confirmed at different times as the posture of the quadruped animal. . In view of these points, it is possible to recognize with high accuracy whether or not the object represented by the high luminance region extracted from the captured image by the determination method as described above is a quadruped.

  The vehicle periphery monitoring device according to a third aspect of the present invention is the vehicle periphery monitoring device according to the second aspect of the present invention, wherein the object classifying means performs the second operation within a first predetermined time after the first element region group is extracted. Extracting an element region group; extracting the third element region group within a second predetermined time longer than the first predetermined time after the second element region group is extracted; As a further requirement, it is determined that the object corresponds to the quadruped animal.

  According to the vehicle periphery monitoring apparatus of the third aspect of the invention, the second element region group is extracted within the first predetermined time after the first element region group is extracted, and the second element region group is On the condition that the third element region group is extracted within a second predetermined time longer than the first predetermined time after the extraction, it is determined that the object corresponds to a quadruped animal. When a walking or running quadruped is observed from the side, the quadruped's posture is the second higher than the time required for the quadruped's posture to transition from the first posture to the second posture. There is a high possibility that the time required for the transition from the first posture to the third posture will be longer. Therefore, as described above, the second predetermined time is set longer than the first predetermined time, so that the third element region group cannot be extracted after the second element region group is extracted. It can be avoided that the object is mistakenly identified as not being a quadruped at an early stage. In other words, whether or not the object is a quadruped animal can be recognized with high accuracy by the above-described determination method.

A vehicle periphery monitoring device according to a fourth aspect of the present invention is a vehicle periphery monitoring device that monitors the periphery of a vehicle using a captured image obtained by an imaging unit mounted on the vehicle, and is a first device that represents an object from the captured image . Extracting a first object region constituted by a second high-intensity region having a luminance equal to or higher than a second threshold value included in the first high-luminance region having a luminance equal to or higher than a threshold value; A target area extracting means for extracting a second target area, which is an area where a part of the target object below exists, as a target area; and the first target area extracted by the target area extracting means . The ratio of the horizontal width to the vertical width is 1 or more , and extends from the object area extracted by the object area extracting means in the vertical direction or the direction inclined with respect to the vertical direction and downward. As you head A second element region group composed of a pair of side-by-side element regions arranged so as to be widened, and one element region arranged next to the pair of element regions, and the second element region The object is a quadruped animal, provided that the pair of element regions arranged in the left-right direction compared to the group and the third element region group consisting of the one element region are extracted at different times. It is characterized by having an object classification means which determines that it corresponds to.

  According to the vehicle periphery monitoring device of the fourth invention, for the same reason as in the second invention, whether or not the object represented by the high-intensity region extracted from the captured image by the determination method as described above is a quadruped animal. Can be recognized with high accuracy.

The vehicle periphery monitoring device according to a fifth aspect of the present invention is the vehicle periphery monitoring device according to the fourth aspect, wherein the object classification means extracts the third element region group after extracting the second element region group. In the meantime, as a further requirement that a first element region group consisting of two pairs of element regions arranged side by side so as to increase in distance from the second object region downward is extracted, It is determined that the object corresponds to the quadruped animal.

According to the vehicle periphery monitoring device of the fifth aspect of the present invention, it is further required that the first element region group is extracted while the third element region group is extracted after the second element region group is extracted. It is determined that the object corresponds to a quadruped animal . The first element region group is highly likely to correspond to the four legs of a laterally-facing quadruped that shows the “first posture” in which the front legs and the rear legs are open. Further, when a quadruped animal that is walking or running is observed from the side, “second posture”, “first posture”, and “third posture” are sequentially changed at different times as the posture of the quadruped animal. It is likely to be confirmed. In view of these points, it is possible to recognize with high accuracy whether or not the object represented by the high luminance region extracted from the captured image by the determination method as described above is a quadruped.

  An embodiment of a vehicle periphery monitoring device of the present invention will be described. First, the structure of the vehicle periphery monitoring apparatus of this embodiment is demonstrated. The vehicle periphery monitoring device includes an image processing unit 1 shown in FIGS. 1 and 2. The image processing unit 1 is connected with two infrared cameras 2R and 2L as imaging means for capturing an image in front of the host vehicle 10, and as a sensor for detecting the traveling state of the host vehicle 10, the image processing unit 1 A yaw rate sensor 3 that detects the yaw rate, a vehicle speed sensor 4 that detects the traveling speed (vehicle speed) of the host vehicle 10, and a brake sensor 5 that detects the presence or absence of a brake operation of the host vehicle 10 are connected. Further, the image processing unit 1 includes a speaker 6 for outputting auditory notification information such as voice, and a display device for displaying captured images and visual notification information captured by the infrared cameras 2R and 2L. 7 is connected. The infrared cameras 2R and 2L correspond to the image pickup means in the present invention. When the distance to the object is measured by a radar, only one infrared camera may be mounted on the vehicle 10.

  Although not shown in detail, the image processing unit 1 includes electronic circuits including an A / D conversion circuit, a microcomputer (having a CPU, a RAM, a ROM, and the like), an image memory, and the like. The infrared cameras 2R, 2L, Analog signals output from the yaw rate sensor 3, the vehicle speed sensor 4, and the brake sensor 5 are converted into digital data by the A / D conversion circuit and input to the microcomputer. Then, the microcomputer detects an object such as a person (pedestrian or person riding a bicycle) based on the input data, and the speaker 6 if the detected object satisfies a predetermined notification requirement. And the process which issues a report to a driver by the display device 7 is executed. The image processing unit 1 has functions as an object extraction unit and an object classification unit.

  As shown in FIG. 2, the infrared cameras 2 </ b> R and 2 </ b> L are attached to the front portion (the front grill portion in the figure) of the host vehicle 10 in order to image the front of the host vehicle 10. In this case, the infrared cameras 2R and 2L are respectively disposed at a position on the right side and a position on the left side of the center of the host vehicle 10 in the vehicle width direction. These positions are symmetrical with respect to the center of the vehicle 10 in the vehicle width direction. The infrared cameras 2R and 2L are fixed so that their optical axes extend in parallel in the front-rear direction of the vehicle 10, and the heights of the respective optical axes from the road surface are equal to each other. The infrared cameras 2R and 2L have sensitivity in the far-infrared region, and have a characteristic that the higher the temperature of an object to be imaged, the higher the level of the output video signal (the higher the luminance of the video signal). is doing.

  In the present embodiment, the display device 7 includes a head-up display 7a (hereinafter referred to as HUD 7a) that displays image information on the front window of the host vehicle 10. In addition, as a display device 7, instead of the HUD 7a, or together with the HUD 7a, a display provided integrally with a meter for displaying a traveling state such as the vehicle speed of the host vehicle 10 or a display provided in an in-vehicle navigation device is provided. It may be used.

  Next, basic functions of the vehicle periphery monitoring apparatus having the above-described configuration will be described with reference to the flowchart of FIG. The basic processing contents of the flowchart of FIG. 3 are the same as the processing contents described in FIG. 3 of Japanese Patent Application Laid-Open No. 2001-6096 and FIG. 3 of Japanese Patent Application Laid-Open No. 2007-310705, for example.

  Specifically, first, infrared image signals from the infrared cameras 2R and 2L are input to the image processing unit 1 (FIG. 3 / STEP 11). Next, the image processing unit 1 A / D converts the output signals of the infrared cameras 2R and 2L (FIG. 3 / STEP 12). Then, the image processing unit 1 acquires a gray scale image from the A / D converted infrared image (FIG. 3 / STEP 13). Thereafter, the image processing unit 1 binarizes the reference image (right image) (FIG. 3 / STEP 14). Next, the image processing unit 1 extracts an area where the object exists from the binarized image as the object area S. Specifically, the pixel group constituting the high brightness area of the binarized image is converted into run length data (FIG. 3 / STEP 15). Next, a label (identifier) is attached to each line group that overlaps in the vertical direction of the reference image (FIG. 3 / STEP 16). Then, each line group is extracted as an object (FIG. 3 / STEP 17). Thereby, for example, as shown in FIGS. 6A to 6C, an area defined so as to surround the entire object by the rectangular double frame is extracted as the object area S. It should be noted that not all of the target object (high luminance region) but part of the target object, particularly when the target object is a quadruped animal, the lower part of the target object where the leg is likely to exist is included. The object region S may be set in Thereby, for example, the region surrounded by the alternate long and short dash line located in the lower portion of the region surrounded by the double line shown in FIGS. 6A to 6C may be extracted as the object region S. .

  Thereafter, the image processing unit 1 calculates the position of the center of gravity of each object extracted as described above (position on the reference image), the area, and the aspect ratio of the circumscribed square (FIG. 3 / STEP 18). Next, the image processing unit 1 traces the object for the time, that is, determines the identity of the object for each arithmetic processing period of the image processing unit 1 (FIG. 3 / STEP 19). Then, the image processing unit 1 reads the outputs (the detected vehicle speed value and the detected yaw rate value) of the vehicle speed sensor 4 and the yaw rate sensor 5 (FIG. 3 / STEP 20).

  On the other hand, in parallel with the calculation of the aspect ratio of the circumscribed rectangle and the tracking of the target object during the time, the image processing unit 1 firstly, in the reference image, an area corresponding to each target object (for example, the circumscribed square of the target object). (Region) is extracted as a search image (FIG. 3 / STEP 31). Next, a search area for searching for an image (corresponding image) corresponding to the search image from the left image is set, and a correlation operation is executed to extract the corresponding image (FIG. 3 / STEP 32). And the distance from the own vehicle 10 of the target object (distance in the front-back direction of the own vehicle 10) is calculated (FIG. 3 / STEP33).

  Next, the image processing unit 1 calculates a real space position that is a position (relative position with respect to the host vehicle 10) of each object in the real space (FIG. 3 / STEP 21). Then, the image processing unit 1 corrects the position X in the X direction of the real space position (X, Y, Z) of the object according to the time-series data of the turning angle obtained in STEP 20 (FIG. 3 / (Step 22). Thereafter, the image processing unit 1 estimates a relative movement vector of the object with respect to the host vehicle 10 (FIG. 3 / STEP 23). Next, when the relative movement vector is obtained in STEP 23, the possibility of contact with the detected object is determined, and a notification determination process for issuing a notification when the possibility is high is executed (FIG. 3 / STEP 24). When it is determined that the object satisfies the notification determination requirement (FIG. 3 / STEP 24... YES), a notification output determination process is performed to determine whether or not an actual notification regarding the object that satisfies the notification determination requirement should be performed. (FIG. 3 / STEP 25). Thereafter, when the image processing unit 1 determines that the notification should be performed (FIG. 3 / STEP 25... YES), the image processing unit 1 notifies the driver that the possibility of contact with the host vehicle 10 is high (FIG. 3). 3 / STEP 26). Specifically, the situation is reported by outputting sound through the speaker 6. The situation is also reported by highlighting the object on the HUD 7a. The above is the overall operation of the periphery monitoring device of this embodiment. In addition, the structure which performs the process of STEP11-18 by the image processing unit 1 is equivalent to the 1st object area | region extraction means of this invention.

  Next, the notification determination process in STEP 24 of the flowchart shown in FIG. 3 will be described using the flowchart of FIG. The basic processing content of the flowchart of FIG. 4 is the same as the processing content described in FIG. 4 of Japanese Patent Laid-Open No. 2001-6096 and FIG.

  Specifically, first, the image processing unit 1 performs a contact determination process (FIG. 4 / STEP 41). The contact determination process determines that there is a possibility of contact when the distance between the host vehicle 10 and the object is equal to or less than a value obtained by multiplying the relative speed Vs and the margin time T. Next, when there is a possibility that the host vehicle and the object come into contact within the margin time T (FIG. 4 / STEP 41... YES), the image forming unit 1 determines whether or not the object exists in the approach determination area. The determination is made (FIG. 4 / STEP 42). If the object does not exist in the approach determination area (FIG. 4 / STEP 42... NO), the image forming unit 1 may enter the approach determination area and come into contact with the host vehicle 10. It is determined whether or not (FIG. 4 / STEP 43). Specifically, when the object is present in the intrusion determination area and the movement vector of the object determined in STEP 23 is toward the approach determination area, it is determined that the possibility of contact is high.

  On the other hand, when the object exists in the approach determination area (FIG. 4 / STEP 42... YES), or when the object may enter the approach determination area and come into contact with the host vehicle 10 (FIG. 4). / STEP43... YES), the image processing unit 1 performs an artificial structure determination process for determining whether or not the object is an artificial structure (FIG. 4 / STEP44). In the artificial structure determination process, when a feature that is impossible for a pedestrian is detected, the object is determined to be an artificial structure. Next, when it is determined that the object is not an artificial structure (FIG. 4 / STEP 44... NO), it is determined whether or not the object corresponds to a pedestrian (FIG. 4 / STEP 45). Specifically, it is determined whether or not the object corresponds to a pedestrian from the characteristics such as the shape and size of the image area of the object on the gray scale image and luminance dispersion. Then, when it is determined that the object corresponds to a pedestrian (FIG. 4 / STEP 45... YES), the detected object is determined as a notification or alerting target (FIG. 4 / STEP 46).

  When it is determined that the object is an artificial structure (FIG. 4 / STEP 44... YES) or when the object is determined not to be a pedestrian (FIG. 4 / STEP 45. It is determined whether or not it corresponds to a paw animal (FIG. 4 / STEP 47). When it is determined that the target object corresponds to a quadruped animal (FIG. 4 / STEP 47... YES), the image processing unit 1 determines the detected target object as a report target (FIG. 4 / STEP 46). On the other hand, when it is determined that the object does not correspond to a quadruped animal (FIG. 4 / STEP 47... NO), the image processing unit 1 excludes the detected object from the report target (FIG. 4 / STEP 48). .

  Next, an animal determination process that is a main function of the vehicle periphery monitoring device of the present invention will be described with reference to the flowchart of FIG. It is determined whether or not an element region group is extracted from the object region S extracted by the object extraction means (see FIG. 3 / STEPs 11 to 18). Specifically, depending on whether or not a correlation value between a template stored or stored in advance in a memory and a part or all of the high-luminance image area in the object area S exceeds a reference value, the high-luminance It is determined whether a part or all of the image area corresponds to any one of the first to third element area groups. As templates, a first type template T1 for extracting the first element region group E1, a second type template T2 for extracting the second element region group E2, and a third element region group A third type of template T3 for extracting E3 is prepared. In addition, each element area group E1-E3 may be extracted based on the shape and arrangement | positioning aspect of a thinning image obtained after the high-intensity area | region extracted from the target object area | region S was thinned.

  As shown in FIG. 6A, the first object region group E1 has a pair of side-by-side element regions (in the up-down direction or the up-down direction) that are arranged so that the distance increases as they go downward. (The same applies hereinafter.) E1a and E1b, and another pair of elements arranged side by side so as to be spaced apart from each other. It consists of areas E1c and E1d. As shown in FIG. 6B, the second element region group E2 is composed of one element region E2a and a pair of element regions E2b arranged so that the distance between the second element region group E2 increases toward the lower side. And E2c. As shown in FIG. 6C, the third element region group E3 includes a pair of element regions E3a and E3b that are arranged so that the distance between the third element region group E3 and the element region extends downward. E3c. The element region group shown in FIG. 6B is defined as a third element region group E3, and the element region group shown in FIG. 6C is defined as a second element region group E2. May be.

  As shown in FIG. 7A, the first type of template T1 is a pair of side-by-side reference elements (inclined with respect to the vertical direction or the vertical direction) that are arranged so that the distance increases as they go downward. (The same applies hereinafter.) T1a and T1b, and another pair of reference elements T1c arranged side by side so as to increase in distance as they go downward. And T1d. As shown in FIG. 7 (b), the second type of template T2 includes one reference element T2a and a pair of reference elements T2b arranged so that the distance increases in the downward direction. T2c. As shown in FIG. 7C, the third type template T3 has a pair of reference elements T3a and T3b that are arranged so that the distance between the third type template T3 and the reference element extends in the downward direction. T3c. Note that the template shown in FIG. 7B may be defined as the third type template T3, and the template shown in FIG. 7C may be defined as the second type template T2. As various templates, templates of a plurality of patterns in which some or all of the shapes, sizes, and arrangements (intervals) of the reference elements are different may be prepared. The template may be created based on an image of a specific kind of quadruped animal such as a sideways dog, deer, or horse imaged by the infrared cameras 2R and 2L.

  First, it is determined whether or not the first element region group E1 is extracted from the object region S (FIG. 5 / STEP 102). When it is determined that the first element region group E1 has been extracted (FIG. 5 / STEP102... YES), it is determined whether or not the second element region group E2 has been extracted from the object region S (FIG. 5 / STEP104). . Note that after determining whether or not the second element region group E2 is extracted in STEP102, it may be determined whether or not the first element region group E1 is extracted in STEP104. If it is determined that the second element region group E2 has not been extracted (FIG. 5 / STEP 104... NO), has the first predetermined time elapsed since it was determined that the first element region group E1 was extracted? It is determined whether or not (FIG. 5 / STEP 110). The first predetermined time is set based on the standard walking cycle of the quadruped animal, such as about 0.25 times the standard walking cycle of the quadruped animal. When the determination result is negative (FIG. 5 / STEP 110... NO), it is determined again whether the second element region group E2 is extracted from the object region S (FIG. 5 / STEP 104).

  When it is determined that the second element region group E2 has been extracted (FIG. 5 / STEP 104... YES), it is determined whether or not the third element region group E3 has been extracted from the object region S (FIG. 5 / (STEP 106). If it is determined that the third element region group E3 has not been extracted (FIG. 5 / STEP 106... NO), has the second predetermined time elapsed since it was determined that the second element region group E2 was extracted? It is determined whether or not (FIG. 5 / STEP 112). The second predetermined time is set to be longer than the first predetermined time, such as being set to about twice the first predetermined time. If it is determined in STEP 102 whether or not the second element region group E2 has been extracted, and if it is determined in STEP 104 whether or not the first element region group E1 has been extracted, the second predetermined time is the first predetermined time. It is set based on the standard walking cycle of a quadruped animal as well as the predetermined time. If the determination result is negative (FIG. 5 / STEP 112... NO), it is determined again whether the third element region group E3 is extracted from the object region S (FIG. 5 / STEP 106).

  When it is determined that the third element region group E3 has been extracted (FIG. 5 / STEP 106... YES), it is determined that the object corresponds to a quadruped animal (FIG. 5 / STEP 114). When it is determined that the first element region group E1 has been extracted and the first predetermined time has passed without the second element region group E2 being extracted (FIG. 5 / STEP104... NO, STEP110... YES). It is determined that the object does not correspond to a quadruped animal (FIG. 5 / STEP 116). In addition, when it is determined that the second element region group E2 has been extracted and the second predetermined time has passed without the third element region group E3 being extracted (FIG. 5 / STEP106..., STEP112... YES). It is determined that the object does not correspond to a quadruped animal (FIG. 5 / STEP 116).

  In the image processing unit 1, the configuration (CPU and associated RAM, ROM, etc.) for executing the animal determination process (STEP 100 to 116, STEP 200 to 208) corresponds to the object classification means of the present invention.

  According to the vehicle periphery monitoring device that exhibits the function, the first element region group E1 and the second element region group E2 are extracted at different times from the object region S extracted as the region where the object exists. And the fact that the second element region group E2 and the third element region group E3 are extracted at different times is determined to be a quadruped animal (see FIG. 5 / STEP 102, 104, 106, 114). The first element region group E1 is highly likely to correspond to four legs of a laterally-facing quadruped that indicates the “first posture” in which the front legs and the rear legs are open (see FIG. 6A). ). The second element region group E2 has a probability corresponding to four legs of a laterally-facing quadruped that indicates a “second posture” in which one of the front legs and the rear legs is open and the other is substantially aligned. High (see FIG. 6B). Compared to the second posture, the third element region group E3 has a left and right arrangement of a pair of open legs and a pair of substantially aligned legs (for a quadruped animal). There is a high probability that it corresponds to the four legs of a laterally facing quadruped animal showing the “third posture” which is reversed (see FIG. 6C). Furthermore, when a walking or running quadruped animal is observed from the side, the “first posture”, “second posture”, and “third posture” are confirmed at different times as the posture of the quadruped animal. There is a high possibility of being. In view of these points, it is possible to recognize with high accuracy whether or not the object represented by the high luminance region extracted from the captured image by the determination method as described above is a quadruped.

  Further, the second element region group E2 is extracted within the first predetermined time after the first element region group E1 is extracted, and the first element region group E2 is extracted after the first element region group E2 is extracted. With the further requirement that the third element region group E3 be extracted within a second predetermined time longer than the predetermined time, it is determined that the object corresponds to a quadruped animal (FIG. 5 / STEP 110, 112, 114). When a walking or running quadruped is observed from the side, the quadruped's posture is the second higher than the time required for the quadruped's posture to transition from the first posture to the second posture. There is a high possibility that the time required for the transition from the first posture to the third posture will be longer. Therefore, as described above, the second predetermined time is set longer than the first predetermined time, so that the third element region group cannot be extracted after the second element region group is extracted. It can be avoided that the object is mistakenly identified as not being a quadruped at an early stage. In other words, whether or not the object is a quadruped animal can be recognized with high accuracy by the above-described determination method.

  Note that it may be determined that the object corresponds to a quadruped animal on the condition that the first element region group E1 and the second element region group E2 are extracted at different times (regardless of the previous or later) ( (See FIG. 5 / STEPs 102, 104, and 114). At this time, it may be determined that the target object corresponds to a quadruped animal as a further requirement that the interval between the different times is within a first predetermined time as necessary (see FIG. 5 / STEP 110). Only if the second element region group E2 and the third element region group E3 are extracted at different times (regardless of the previous or later), it may be determined that the object corresponds to a quadruped (see FIG. 5 / STEP 104, 106, 114). At this time, if necessary, it may be determined that the object corresponds to a quadruped animal on the further requirement that the interval between the different times is within a second predetermined time (see FIG. 5 / STEP 112). It may be determined that the object corresponds to a quadruped animal only on the condition that the second element region group E2, the first element region group E1, and the third element region group E3 are sequentially extracted (FIG. 5 / STEP 102, 104, 106, 114). At this time, the time from when the second element region group E2 is extracted until the first element region group E1 is extracted, and after the first element region group E1 is extracted, the third element region group E3. It may be determined that the object corresponds to a quadruped animal on the further requirement that both the time until extraction is within the first predetermined time (see FIG. 5 / STEP 110).

Note that the object region S may be extracted as follows. First, a binarized image obtained by binarizing a grayscale image (see FIG. 3 / STEP 13) obtained from an infrared image captured by the infrared cameras 2R and 2L and A / D converted (see FIG. 3). A region including the high luminance region (object) extracted from 3 / STEP 14) is extracted as the first object region S1 (FIG. 8 / STEP 202). As a result, as shown in FIG. 9A, the high-intensity region of the object (the second higher than the first threshold included in the first high-intensity region having a luminance equal to or higher than the first threshold representing the object). A hatched rectangular area surrounding the second high-luminance area having a luminance equal to or higher than the threshold is extracted as the first object area S1. The aspect ratio W / H, which is a value obtained by dividing the width W of the first object region S1 by the height H, is 1 or more, and is an additional requirement for determining that the object corresponds to a quadruped animal. It may be. The first object region S1 may correspond to a torso of a sideways quadruped animal.

  Subsequently, a reference area A is extracted with reference to the first object area S1 (FIG. 8 / STEP 204). As a result, a region surrounded by a broken line as shown in FIG. 9B is extracted as the reference region A. The upper end position of the reference area A coincides with the upper end position of the first object area S1. The height of the reference area A is set to three times the height H of the first object area S1. The left end position of the reference area A is left by a predetermined margin MARGIN_W from the left end position of the first object area S1, and the right end position of the reference area A is a predetermined margin MARGIN_W ( Only margin is on the right side. Note that the upper end position of the reference area A may be located below the upper end position of the first object area S1.

  Further, the second object area S2 is extracted as the object area S from the reference area A (FIG. 8 / STEP 206). Specifically, edge pixels are extracted in the reference area A with the vertical direction and the horizontal direction as scanning directions. The edge pixel is a portion where the luminance in the binarized image changes from “0” to “1”. As shown in FIG. 9C, the positions of a series of vertical edge pixel groups of a predetermined number or more arranged in the uppermost horizontal direction or almost horizontal direction (deviation by 2 to 3 pixels) in the reference area A are shown. Extracted as the upper end position of the leg. Further, the positions of a series of vertical edge pixel groups of a predetermined number or more arranged in the lowest horizontal direction or almost horizontal direction (displacement of 2 to 3 pixels in the vertical direction) in the reference area A are shown in FIG. ) Is extracted as the leg lower end position. Further, in the reference area A, the positions of a series of a plurality of horizontal edge pixel groups of a predetermined number or more arranged in the leftmost or rightmost vertical direction or substantially vertical direction (displacement by 2 to 3 pixels is allowed) are shown in FIG. As shown in (c), the leg left and right end positions are extracted. Then, as shown in FIG. 9 (c), below the first object region S1, a pair of horizontal lines indicating the leg lower end position and the leg upper end position, and a pair of vertical indicating the leg left and right end positions. A region surrounded by the line is extracted as the second object region S2. Note that a second target area S2 having a predetermined shape stored in the memory may be extracted below the first target area S1.

  Thereafter, as in the above-described embodiment, whether or not the object corresponds to a quadruped animal is determined by determining whether or not an element region group is extracted from the object region S extracted as the second object region S2. It can be determined whether or not (see FIG. 5 / STEPs 102 to 114, FIG. 6 and FIG. 7). In this case as well, in the same manner as in the above-described embodiment, whether or not the target object is a quadruped animal can be recognized with high accuracy in view of the posture change mode of the lateral quadruped during walking or running.

The figure which shows the whole structure of one Embodiment of the vehicle periphery monitoring apparatus of this invention. Perspective view of a vehicle equipped with a vehicle periphery monitoring device Flow chart showing functions of image processing unit constituting vehicle periphery monitoring device Flow chart showing functions of image processing unit constituting vehicle periphery monitoring device The flowchart which shows the object classification determination process in this embodiment Illustration of element regions Illustration about templates Flowchart showing leg region extraction processing in this embodiment The figure for demonstrating the process of an image processing unit

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image processing unit (object extraction means, object determination means), 2R, 2L ... Infrared camera (imaging means)

Claims (5)

A vehicle periphery monitoring device that monitors the periphery of a vehicle using a captured image obtained by an imaging means mounted on the vehicle,
From the captured image , a first object configured by a second high-brightness area having a luminance equal to or higher than the first threshold and included in a first high-brightness area having a luminance equal to or higher than the first threshold representing the object. An object region extracting means for extracting a region, and extracting a second object region, which is a region where a part of the object exists below the first object region, as an object region;
The ratio of the width to the vertical width of the first object area extracted by the object area extraction means is 1 or more, and from the object area extracted by the object area extraction means, the vertical direction or A first element region group composed of two pairs of element regions arranged side by side so as to extend in a direction inclined with respect to the up-down direction and to extend downward, and as it goes downward The requirement is that a second element region group consisting of a pair of side-by-side element regions arranged so that the interval is widened and one element region side by side is extracted at different times. A vehicle periphery monitoring device comprising: object classification means for determining that the object corresponds to a quadruped animal. The vehicle periphery monitoring device according to claim 1,
The object classifying means is arranged in such a way that the second element region group and the pair of side by side are arranged so that the interval is widened toward the lower side as compared with the second element region group. The vehicle periphery characterized in that it is determined that the object corresponds to the quadruped animal, with the further requirement that an element region and a third element region group consisting of the one element region are extracted at different times Monitoring device. In the vehicle periphery monitoring device according to claim 2,
The object classification means extracts the second element region group within a first predetermined time after the first element region group is extracted, and the second element region group is extracted. The third element region group is extracted within a second predetermined time that is longer than the first predetermined time, and it is determined that the object corresponds to the quadruped animal. A vehicle periphery monitoring device. A vehicle periphery monitoring device that monitors the periphery of a vehicle using a captured image obtained by an imaging means mounted on the vehicle,
From the captured image , a first object configured by a second high-brightness area having a luminance equal to or higher than the first threshold and included in a first high-brightness area having a luminance equal to or higher than the first threshold representing the object. An object region extracting means for extracting a region, and extracting a second object region, which is a region where a part of the object exists below the first object region, as an object region;
The ratio of the width to the vertical width of the first object area extracted by the object area extraction means is 1 or more, and from the object area extracted by the object area extraction means, the vertical direction or A pair of element regions arranged side by side so as to extend in a direction inclined with respect to the up-down direction and to be spaced downward, and one of the element regions arranged next to the pair of element regions A second element region group consisting of element regions, and a third element region group consisting of the pair of element regions and the one element region arranged in the left-right direction in comparison with the second element region group. A vehicle periphery monitoring apparatus, comprising: an object classification unit that determines that the object corresponds to a quadruped animal on the condition that the object is extracted at different times. In the vehicle periphery monitoring device according to claim 4,
The object classification means is arranged such that the interval increases from the second object area downward while the third element area group is extracted after the second element area group is extracted. The vehicle periphery monitoring characterized in that it is determined that the object corresponds to the quadruped animal, on the further requirement that a first element region group consisting of two pairs of the element regions arranged side by side is extracted. apparatus.

Images