JP2003284057A - Vehicle periphery monitoring unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle periphery monitoring unit which stably extracts a walker by exactly deciding an unshaped binarized object to be extracted from an image imaged by a camera. <P>SOLUTION: An image processing unit calculates a binarized object shape feature quantities in the real space by utilizing the center of gravity, area, of the binarized object, distance from its own vehicle, further aspect ratio, a height and a width of a circumscribed rectangular shape of the object and the coordinate values of the center of gravity (S41). Then, the unit obtains the height of the object on an object for a gray scale including the binarized object (S42), and sets a plurality of mask regions in the object for the gray scale and calculates a luminance mean value and a luminance change (dispersion) of each mask (S43). The unit decides whether the height, width, presence height, luminance means value, luminance dispersion of the binarized object are values within suitable ranges as the walker or not (S44 to S48). If any is decided not to be in the suitable range as a pedestrian, it is decided that the object is not a pedestrian (S49). <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle periphery monitoring device for extracting an object by binarizing an image captured by an infrared camera.

[0002]

2. Description of the Related Art Conventionally, an object around a vehicle such as a pedestrian having a possibility of collision with a vehicle is extracted from an image around the vehicle captured by an infrared camera, and the information is provided to a driver of the vehicle. Monitoring devices have been proposed. In this system, the possibility of collision with an object such as a pedestrian is determined based on the relative distance between the vehicle and the object or the relative speed.

Further, for example, a vehicle periphery monitoring device for extracting an object such as a pedestrian having a possibility of collision with the vehicle from an image of the periphery of the vehicle captured by the infrared camera as described below is as follows. There is something like this. More specifically, this device binarizes an infrared image to search for a region where bright parts are concentrated, and uses the aspect ratio and sufficiency ratio of this region, and also the actual area and the position of the center of gravity on the screen. By calculating the distance by using the distance, it is determined whether or not this area is the head of the pedestrian. Then, if the area of the pedestrian's head can be determined, the height of the pedestrian on the image is calculated from the determined distance from the camera of the head area and the average height of the adult to determine the pedestrian's body. Areas are set, and these areas are displayed separately from other areas. Thereby, the position of the entire body of the pedestrian on the infrared image is specified, and this information is displayed to the driver of the vehicle, whereby more effective visual assistance can be performed (for example, Patent Document 1). reference.).

[0004]

[Patent Document 1] Japanese Patent Laid-Open No. 11-328364

[0005]

However, the pedestrian on the infrared image has only the head or a part of the head due to the binarization processing depending on the influence of the cap, clothing, and the existence environment of the pedestrian himself / herself. The binarized shape is indefinite such that only the upper body, only the lower body, and the entire body are extracted. Further, in general, when the vehicle is traveling, there is a change in the shape of the front road surface and the influence of the pitching of the vehicle, and a pedestrian can be detected from a child to an adult (adult) with a height different from the original height. Therefore, since the barycentric coordinates on the screen of the object cannot be fixed with respect to the distance, when the pedestrian is extracted only by the shape determination as in the conventional device described above, only the pedestrian is stably extracted. There was a possibility that I could not.

The present invention has been made in view of the above problems, and is a vehicle for accurately determining an indeterminate binarized object extracted from an image taken by a camera and stably extracting pedestrians. An object is to provide a peripheral monitoring device.

[0007]

In order to solve the above problems, a vehicle periphery monitoring device according to the invention of claim 1 is a vehicle for recognizing a pedestrian by using images captured by two infrared cameras. A surrounding monitoring device, which is a binarized object extracting means for extracting a binarized object from the grayscale image by binarizing the grayscale image of the image (for example, step S1 of the embodiment). ~ Step S13) and a grayscale object extraction means (for example, in the embodiment) that extracts a grayscale object in a range including the binarized object from the grayscale image by changing the brightness of the grayscale image. Steps S41 to S42) and a search area (for example, an implementation) in the area of the grayscale object (for example, the area AREA0 in the embodiment). The form of a mask area AREA1, AR
EA2, AREA3) is set, and a pedestrian discrimination means for recognizing a pedestrian in the grayscale image based on the luminance distribution of the search area (for example, step S43 of the embodiment).
Up to step S80).

The vehicle periphery monitoring device having the above-mentioned configuration is
First, the position of the binarized object is recognized on the grayscale image by the binarized object extraction means. Then, the grayscale object extracting means sets a grayscale object in a range including the binarized object, and the pedestrian discriminating means calculates the luminance dispersion for each search area set on the grayscale object. Then, from the characteristics of the brightness distribution of the search area, 2
It is possible to determine whether the object to be valued is a pedestrian.

According to a second aspect of the invention, there is provided the vehicle periphery monitoring device according to the first aspect, wherein the pedestrian discrimination means sets the lateral size of the search area to 2
The width of the binarized object image is set, and the vertical size of the search area is set as the height of the grayscale object image. The vehicle periphery monitoring device having the above-described configuration uses the horizontal width of the search area on the grayscale image for which the luminance dispersion is obtained as the horizontal width of the binarized object to recognize the pedestrian in the horizontal width of the search area. Set to an appropriate width 2
It is possible to determine whether the object to be valued is a pedestrian.

According to a third aspect of the present invention, there is provided a vehicle periphery monitoring device according to the first aspect, wherein the pedestrian discrimination means uses the upper end of the grayscale object as a reference for the search area. , A head region having a size corresponding to the head of a pedestrian is set. In the vehicle periphery monitoring device having the above configuration, the binarized object is set as a binarized object by setting the search area on the grayscale image for obtaining the luminance dispersion to an appropriate size for recognizing the head of the pedestrian. It can be determined whether or not.

According to a fourth aspect of the invention, there is provided the vehicle periphery monitoring device according to the first aspect, wherein the pedestrian discrimination means uses the upper end of the grayscale object as a reference for the search area. A head region having a size corresponding to the head of the pedestrian and a torso region larger than the head region corresponding to the torso of the pedestrian below the head region. . The vehicle surroundings monitoring device having the above configuration, on the grayscale image for which the brightness dispersion is obtained, a search area of a size suitable for recognizing the head of a pedestrian,
It is possible to determine whether or not the binarized object is a pedestrian by setting a search area having an appropriate size for recognizing the torso of the pedestrian.

According to a fifth aspect of the present invention, there is provided a vehicle surroundings monitoring device for detecting an object existing around the vehicle from an infrared image taken by an infrared camera, wherein infrared rays are detected from the infrared image. Object extracting means for extracting the object to be emitted (eg, step S of the embodiment)
1 to step S13) and a heat storage body extracting means for extracting a heat storage body that stores only heat given from outside without generating heat from the target object extracted by the target object extracting means (for example, the embodiment. Steps S45, S46, S4
7, S52, S53, S53-1, S60, S61, S
62) and pedestrian recognition means for recognizing a pedestrian from the objects excluding the heat storage body extracted by the heat storage body extraction means (for example, step S48 to step S50, step S54 to step S59 of the embodiment). , Step S63 to step S80).

A vehicle periphery monitoring device having the above-mentioned configuration is
By the heat storage body extraction means, walking as far as possible by removing the heat storage body from the object that has a different property from the pedestrian but may be mistakenly recognized as a pedestrian due to the influence of the external environment. Whether or not it is a pedestrian can be determined from among objects that are similar in nature to the person.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a vehicle periphery monitoring device according to an embodiment of the present invention. In FIG. 1, reference numeral 1 is an image processing unit including a CPU (central processing unit) for controlling the vehicle periphery monitoring device of the present embodiment, and two infrared cameras 2R and 2L capable of detecting far infrared rays. A yaw rate sensor 3 for detecting a yaw rate of the vehicle, a vehicle speed sensor 4 for detecting a traveling speed (vehicle speed) of the vehicle, and a brake sensor 5 for detecting a brake operation are connected. As a result, the image processing unit 1 detects a pedestrian or an animal in front of the vehicle from the infrared image of the periphery of the vehicle and a signal indicating the traveling state of the vehicle, and issues an alarm when it is determined that the possibility of collision is high. Emit.

Further, the image processing unit 1 includes a speaker 6 for issuing a sound alarm and infrared cameras 2R, 2R.
For example, a display integrated with a meter or a display integrated with a meter that displays a running state of the vehicle by a numerical value for displaying an image captured by L so that the driver of the vehicle recognizes an object having a high risk of collision. NAVIDisplay installed on the console of the vehicle, and HUD that displays information in a position that does not obstruct the driver's forward view of the front window
An image display device 7 including (Head Up Display) 7a and the like is connected.

The image processing unit 1 also includes an A / D conversion circuit for converting an input analog signal into a digital signal, an image memory for storing a digitized image signal, a CPU (central processing unit) for performing various arithmetic processes, and a CPU. RAM (Random A) used to store data during calculation
ccess Memory), ROM (Read Only Memor) that stores programs executed by the CPU, tables, maps, etc.
y), an output circuit for outputting a driving signal for the speaker 6, a display signal for the HUD 7a, etc., and the infrared camera 2
The output signals of the R, 2L, yaw rate sensor 3, vehicle speed sensor 4, and brake sensor 5 are converted into digital signals and input to the CPU.

Further, as shown in FIG. 2, the infrared camera 2
R and 2L are arranged in the front part of the host vehicle 10 at positions substantially symmetrical to the center of the host vehicle 10 in the vehicle width direction.
The optical axes of the two infrared cameras 2R and 2L are parallel to each other, and they are fixed so that their heights from the road surface are equal. The infrared cameras 2R and 2L have a characteristic that the output signal level thereof increases (luminance increases) as the temperature of the object increases. Also, HUD7a
Is provided so that the display screen is displayed at a position in the front window of the host vehicle 10 that does not hinder the driver's forward visibility.

Next, the operation of this embodiment will be described with reference to the drawings. FIG. 3 is a flowchart showing an operation of detecting and warning an object such as a pedestrian in the image processing unit 1 of the vehicle periphery monitoring device according to the present embodiment. First, the image processing unit 1 acquires an infrared image which is an output signal of the infrared cameras 2R and 2L (step S1), and A /
D conversion is performed (step S2), and the grayscale image is stored in the image memory (step S3). Here, the right image is obtained by the infrared camera 2R,
The left image is obtained by L. Further, in the right image and the left image, the horizontal position of the same target object on the display screen is displaced, and thus the distance to the target object can be calculated from this displacement (parallax).

When a grayscale image is obtained in step S3, the right image obtained by the infrared camera 2R is used as a reference image, and the image signal is binarized, that is, an area brighter than the luminance threshold value ITH is ". 1 ”(white)
Then, processing for setting the dark area to “0” (black) is performed (step S4). FIG. 4A shows a grayscale image obtained by the infrared camera 2R, and by binarizing the grayscale image, an image as shown in FIG. 4B is obtained.
Note that, in FIG. 4B, for example, an object surrounded by a frame of P1 to P4 is an object displayed as white on the display screen (hereinafter, referred to as “high brightness area”). When binarized image data is acquired from the infrared image, a process of converting the binarized image data into run length data is performed (step S5). The line represented by the run length data is a pixel level representation of a region that has become white due to binarization. Each line has a width of 1 pixel in the y direction, and has a width of 1 pixel in the x direction. It has the length of the pixel which comprises run length data.

Next, the object is labeled from the image data converted into the run length data (step S).
By performing 6), the process of extracting the object is performed (step S7). That is, of the lines converted into run-length data, the line having a portion overlapping in the y direction is regarded as one target, so that, for example, the high-brightness areas P1 to P4 shown in FIG. Target object)
Will be understood as. When the extraction of the object is completed, the center of gravity G, the area S, and the aspect ratio ASPECT of the circumscribed quadrangle of the extracted object are calculated (step S).
8).

Here, as for the area S, the run length data of the object of the label A is (x [i], y [i], run.
[I], A) (i = 0, 1, 2, ... N-1), the run length data length (run [i] -1) is the same object (N run length data). ) Is added to calculate. Further, the coordinates (xc, yc) of the center of gravity G of the object A are the lengths (r
un [i] -1) and coordinates x of each run length data
It is calculated by multiplying by [i] or y [i], respectively, and further summing these for the same object and dividing by the area S. Furthermore, the aspect ratio ASPECT is
It is calculated as the ratio Dy / Dx of the vertical length Dy and the horizontal length Dx of the circumscribed quadrangle of the object. Since the run length data is indicated by the number of pixels (the number of coordinates) (= run [i]), the actual length needs to be “−1” (1 is subtracted) (= run [i] ] -1). Further, the position of the center of gravity G may be replaced by the position of the center of gravity of the circumscribing quadrangle.

After the center of gravity of the object, the area, and the aspect ratio of the circumscribing quadrangle can be calculated, the object is time-tracked, that is, the same object is recognized for each sampling cycle (step S9). In the time tracking, the time t as an analog quantity is discretized in a sampling cycle, and the time is set to k. For example, when the objects A and B are extracted at time k, the objects C and D extracted at time (k + 1) and the object are extracted. The identity with the objects A and B is determined. Then, when it is determined that the objects A and B are the same as the objects C and D, the objects C and D are set to the objects A and B, respectively.
By changing the label to B, time tracking is performed. Further, the position coordinates (of the center of gravity) of each object recognized in this way are stored in the memory as time-series position data and used in the subsequent arithmetic processing.

The processes of steps S4 to S9 described above are executed for the binarized reference image (the right image in this embodiment). Next, the turning speed θr of the host vehicle 10 is calculated by reading the vehicle speed VCAR detected by the vehicle speed sensor 4 and the yaw rate YR detected by the yaw rate sensor 3 and integrating the yaw rate YR with time (step S10).

On the other hand, in parallel with the processing in steps S9 and S10, in steps S11 to S13, processing for calculating the distance z between the object and the vehicle 10 is performed. Since this calculation requires a longer time than steps S9 and S10, it is executed at a cycle longer than steps S9 and S11 (for example, a cycle about three times the execution cycle of steps S1 to S10). First, by selecting one of the objects tracked by the binarized image of the reference image (right image), the search image R1 (here, the entire area surrounded by the circumscribed rectangle is searched image from the right image). Is extracted) (step S11).

Next, a search area for searching the image corresponding to the search image R1 (hereinafter referred to as "corresponding image") in the left image is set, and the correlation calculation is executed to extract the corresponding image (step S12). Specifically, the search area R2 is set in the left image according to the vertex coordinates of the search image R1, and the brightness difference sum value C (a) indicating the height of the correlation with the search image R1 in the search area R2 is set. , B) is calculated, and the total value C (a, b) is calculated.
The area with the minimum is extracted as the corresponding image. Note that this correlation calculation is performed using a grayscale image instead of a binarized image. If there is past position data for the same object, a region R2a narrower than the search region R2 is set as the search region based on the position data.

Since the search image R1 is extracted in the reference image (right image) and the corresponding image R4 corresponding to this object is extracted in the left image by the processing in step S12, the center of gravity of the search image R1 is next extracted. The position, the position of the center of gravity of the corresponding image R4, and the parallax amount Δd (the number of pixels) are obtained, and the distance z between the vehicle 10 and the object is calculated from this (step S13). Next, calculation of the turning angle θr in step S10 and step S1
When the calculation of the distance to the object in 3 is completed, the coordinates (x, y) in the image and the distance z are converted to real space coordinates (X, Y, Z).
(Step S14). Here, the real space coordinates (X, Y, Z) are the infrared camera 2 as shown in FIG.
The position of the midpoint of the mounting position of R, 2L (the position fixed to the vehicle 10) is set as the origin O, and the coordinates are determined as shown in the figure.
The vertical direction is defined as y.

Further, when the real space coordinates are obtained, the turning angle correction for correcting the positional deviation on the image due to the turning of the vehicle 10 is performed (step S15). The turning angle correction is performed by the vehicle 10 during the period from time k to (k + 1).
When, for example, the head is turned leftward by the turning angle θr, the range of the image shifts by Δx in the x direction on the image obtained by the camera, and this is a process of correcting this. In the following description, the coordinates after the turning angle correction are displayed as (X, Y, Z).

After the turning angle correction for the real space coordinates is completed, then, for the same object, N pieces (for example, N = 10) after the turning angle correction are obtained within the monitoring period of ΔT.
The approximate straight line LMV corresponding to the relative movement vector between the object and the own vehicle 10 is obtained from the real space position data of, that is, the time series data. Then, the latest position coordinate P (0) =
(X (0), Y (0), Z (0)) and position coordinates P (N-1) = (X) before (N-1) samples (before time ΔT).
(N-1), Y (N-1), Z (N-1)) are corrected to the positions on the approximate straight line LMV, and the corrected position coordinates Pv (0)
= (Xv (0), Yv (0), Zv (0)) and Pv
(N-1) = (Xv (N-1), Yv (N-1), Zv
(N-1)) is calculated.

As a result, a relative movement vector is obtained as a vector from the position coordinates Pv (N-1) to Pv (0) (step S16). In this way, from the plurality (N pieces) of data within the monitoring period ΔT, the target vehicle 10
By calculating an approximate straight line that approximates the relative movement trajectory with respect to the relative movement vector to obtain the relative movement vector, the influence of the position detection error can be reduced and the possibility of collision with the object can be predicted more accurately.

When the relative movement vector is obtained in step S16, next, an alarm determination process is performed to determine the possibility of collision with the detected object (step S).
17). The details of the alarm determination process will be described later. When it is determined in step S17 that there is no possibility of collision between the vehicle 10 and the detected object (NO in step S17), the process returns to step S1 and the above-described processing is repeated. In step S17, the own vehicle 10
When it is determined that there is a possibility of collision with the detected object (YES in step S17), the process proceeds to the alarm output determination process in step S18.

In step S18, it is determined from the output BR of the brake sensor 5 whether or not the driver of the vehicle 10 is performing a brake operation, thereby performing an alarm output determination process, that is, determination of whether or not to output an alarm. Is performed (step S18). If the driver of the host vehicle 10 is performing a brake operation, the acceleration Gs (the deceleration direction is positive) generated thereby is calculated, and this acceleration Gs is calculated.
Is greater than the predetermined threshold value GTH, it is determined that the collision is avoided by the brake operation, the alarm output determination process is ended (NO in step S18), the process returns to step S1, and the above process is repeated. As a result, when an appropriate brake operation is performed, an alarm is not issued, and it is possible to prevent the driver from getting annoyed.

Further, when the acceleration Gs is less than or equal to the predetermined threshold value GTH, or when the driver of the vehicle 10 is not performing the brake operation, the process immediately proceeds to step S19 (YES in step S18) to contact the object. Therefore, an alarm is issued by a voice through the speaker 6 (step S19), and an image obtained by, for example, the infrared camera 2R is output to the image display device 7 to detect an approaching object. The image is displayed as a highlighted image for the driver of the vehicle 10 (step S20). Note that the predetermined threshold value GTH is the distance Zv between the object and the host vehicle 10 when the acceleration Gs during the brake operation is maintained as it is.
It is a value corresponding to the condition that the vehicle 10 stops at the travel distance of (0) or less.

The above is the object detection / warning operation in the image processing unit 1 of the vehicle periphery monitoring device of the present embodiment. Next, referring to the flow chart shown in FIG.
The alarm determination process in step S17 of the flowchart shown in FIG. 3 will be described in more detail. FIG. 5 is a flowchart showing the alarm determination processing operation of this embodiment. The warning determination process includes the following collision determination process, determination process of whether or not the vehicle is in the approach determination region, approach collision determination process, pedestrian determination process, and artificial structure determination process.
This is a process of determining the possibility of collision with an object detected as 0. Hereinafter, as shown in FIG. 6, an example will be described in which there is an object 20 moving at a speed Vp from a direction substantially 90 ° with respect to the traveling direction of the host vehicle 10.

In FIG. 5, first, the image processing unit 1
Performs a collision determination process (step S31). In FIG. 6, the collision determination process is performed when the target object 20 approaches the distance Zv (0) from the distance Zv (N−1) during the time ΔT.
A process of determining a relative velocity Vs in the Z direction with the host vehicle 10 and determining whether the two collide with each other within the allowance time T, assuming that the two move within the height H while maintaining the relative velocity Vs. Is. Here, the margin time T is intended to determine the possibility of a collision by the time T before the predicted collision time. Therefore, the allowance time T is set to, for example, about 2 to 5 seconds. H is a predetermined height that defines the range in the height direction, and is set to, for example, about twice the vehicle height of the host vehicle 10.

Next, in step S31, when there is a possibility that the vehicle 10 and the object collide with each other within the allowance time T (YES in step S31), the image processing unit further increases the reliability of the determination. 1 performs a determination process as to whether or not the target object exists within the approach determination area (step S
32). As shown in FIG. 7, assuming that the area that can be monitored by the infrared cameras 2R and 2L is an outer triangular area AR0 indicated by a thick solid line, Z1 = A region AR1 that is closer to the host vehicle 10 than Vs × T and corresponds to a range in which the object has a margin β (for example, about 50 to 100 cm) added to both sides of the vehicle width α of the host vehicle 10, that is, the target This is a process of determining whether or not an object exists in the approach determination area AR1 in which the possibility of collision with the vehicle 10 is extremely high if the object continues to exist. The approach determination area AR1 also has the predetermined height H.

Furthermore, in step S32, when the object does not exist in the approach determination area (N in step S32).
O), the image processing unit 1 performs the approach collision determination process for determining whether or not the object may enter the approach determination area and collide with the host vehicle 10 (step S33). In the approach collision determination process, areas AR2 and AR3 having a larger absolute value of the X coordinate than the approach determination area AR1 (laterally outside the approach determination area) are referred to as approach determination areas, and an object in this area is By moving, the approach determination area AR1
Is a process of determining whether or not to collide with the host vehicle 10 while entering the vehicle. The entry determination areas AR2 and AR3 also have the predetermined height H.

On the other hand, in step S32, when the object is present in the approach determination area (YES in step S32), the image processing unit 1 determines whether the object may be a pedestrian. Pedestrian determination processing is performed (step S34). The details of the pedestrian determination process will be described later. If it is determined in step S34 that the object may be a pedestrian (step S3
4), the artificial structure determination process for determining whether or not the object is an artificial structure is performed to further improve the reliability of the determination (step S35). The artificial structure determination process is a process of determining an object as an artificial structure and excluding it from an alarm target when a characteristic that is not possible for a pedestrian such as the following is detected in the object image. Is. (1) When the image of the target object includes a portion indicating a straight edge. (2) When the corners of the image of the object are right angles. (3) When the image of the target object includes a plurality of objects having the same shape. (4) When the image of the object matches the shape of the artificial structure registered in advance.

Therefore, in step S33 described above,
When the target object may enter the approach determination region and collide with the host vehicle 10 (YES in step S33), and in step S35, the target object that is determined to be a pedestrian is an artificial structure. If not a thing (step S
No of 35), the image processing unit 1 determines that there is a possibility of collision between the vehicle 10 and the detected object (target of alarm) (step S36), and YES of step S17 shown in FIG. Then, the process proceeds to step S18, and an alarm output determination process (step S18) is performed.

On the other hand, in step S31 described above, if there is no possibility of collision between the host vehicle 10 and the object within the allowance time T (NO in step S31), or in step S33, the object falls within the approach determination area. When there is no possibility of entering the vehicle and colliding with the host vehicle 10 (step S3
3), or when it is determined in step S34 that the object is not a pedestrian (step S).
If the target object that is determined to be a pedestrian in step S35 is an artificial structure (YES in step S35), the image processing unit 1 indicates that there is no possibility of collision between the host vehicle 10 and the detected object (not the target of warning)
(Step S37), and step S1 shown in FIG.
If NO in step 7, the process returns to step S1, and the object detection / warning operation of a pedestrian or the like is repeated.

Next, the pedestrian determination processing in step S34 of the flowchart shown in FIG. 5 will be described in more detail with reference to the flowcharts shown in FIGS. 8 to 13 are flowcharts showing the pedestrian determination processing operation of the present embodiment. In FIG. 8, first, the image processing unit 1 performs the center of gravity G (xc, yc) of the binarized object calculated in step S8 of the flowchart shown in FIG. 3 (the center of gravity of the binarized object shown in FIG. 14). G100), area S (binary object area S shown in FIG. 14)
101), and the aspect ratio ASPEC of the circumscribed rectangle of the object
T, and own vehicle 10 calculated in step S13
In addition to the distance z between the object and the object, the height hb and width wb of the circumscribed quadrangle of the binarized object shown in FIG. 14, and the circumscribed quadrangle barycentric coordinates (xb, yb) (circumscribed quadrangle barycenter 1 shown in FIG.
02) is used to calculate the binarized object shape feature amount indicating the shape feature of the binarized object in the real space (step S41). The binary object shape feature amount to be obtained is the camera base length D [m] and the camera focal length f.
[M], pixel pitch p [m / pixel], and parallax amount Δd [pixel] calculated by correlation calculation of left and right images.
l] is used for the calculation.

Specifically, the rate Rate of the circumscribed quadrangle and the object area is Rate = S / (hb × wb) (1) Asp, which represents the aspect ratio ASPECT of the circumscribed quadrangle, is Asp = hb / wb. (2) The distance z between the host vehicle 10 and the target object is expressed as z = (f × D) / (Δd × p) (3), so it is a binarization target in the real space. Object width ΔW
b and the height ΔHb are ΔWb = wb × z × p / f ΔHb = hb × z × p / f (4)

The barycentric coordinates of the binarized object (Xc, Yc, Z
c) is Xc = xc × z × p / f Yc = yc × z × p / f Zc = z (5) The barycenter coordinates of the circumscribed quadrangle of the object (Xb, Yb, Zb) are Xb = xb × z × p / f Yb = yb × z × p / f Zb = z (6) The upper end position coordinates (Xt, Yt, Zt) of the binarized object are: Xt = xb × z × p / f It can be calculated by Yt = yb × z × p / f−ΔHb / 2 Zt = z (7).

Next, using the grayscale image acquired in step S3 of the flowchart shown in FIG. 3, the height of the object on the grayscale image including the binarized object extracted in step S7 is calculated. Obtain (step S42). To determine the height of the object on the grayscale image, set a mask area of a predetermined size from the upper end of the binarized object circumscribing rectangle and set it on the grayscale image. The change is large (the object and the background image are included), the degree of correlation of the mask areas between the left and right images is high (there are no two or more objects in the mask area), and the value is further binary. An area including a mask area having the same distance (parallax) as the object to be converted is extracted as an area of the grayscale object. Then, the height of the area of the grayscale object on the image Height (p
ixel) is calculated, and the height ΔHg of the grayscale object is calculated by the equation (8). ΔHg = z × Height × p / f (8)

As shown in FIG. 15, the area of the grayscale object on the image is AREA0, and mask areas AREA1, AREA2, and AREA3 are set therein, and the average brightness value and the brightness change of each mask are set. (Dispersion) is calculated (step S43). Here, the average luminance value of AREA1 is Ave_A1, and the luminance variance of AREA2 is Var_.
The luminance dispersion of A2 and AREA3 is Var_A3. In the following processing, AREA1 is used to determine the presence of the head of the target object, AREA2 is used to determine the presence of the body of the target object, and AREA3 is used to determine the presence of a shape change from the head to the lower body. The AREA 3 is a heat storage body that does not generate heat by itself, such as a wall, and stores only the heat given from the outside, and a part of the object showing a monotonous change in brightness is extracted by the binarization process. If it is given, it will also be used to identify it as a pedestrian. In addition, FIG. 15 schematically shows a pedestrian captured by a camera, and a hatched region is a region of an object captured by binarization, and a region surrounded by a dotted line is binarization. Although not captured, it represents the part of the object where the presence of the object can be confirmed against the background in the grayscale image. The dimensions of each part shown in FIG. 15 are examples of the dimensions of each part in the real space.

More specifically, the pedestrian determination based on the shape of the binarized object and the pedestrian determination using the luminance dispersion of each mask area of the grayscale image will be described below with reference to the flowcharts shown in FIGS. 8 to 13. explain. First, the image processing unit 1 determines whether or not the height, width, existing height, luminance average value, and luminance dispersion of the binarized object are within a range suitable for a pedestrian. Specifically, since the target is a pedestrian, the width ΔWb of the binarized object is the threshold TH1.
It is determined whether or not it is TH2 or less (appropriate value as the width of a pedestrian) or more (step S44). In step S44, when the width ΔWb of the binarized object is greater than or equal to the threshold value TH1 and less than or equal to TH2 (YES in step S44), the height ΔHb of the binarized object is the threshold value TH3 (suitable as the height of a pedestrian. Value) and height of grayscale object ΔH
It is determined whether or not g is less than a threshold value TH4 (an appropriate value as the height of a pedestrian) (step S45).

When the height ΔHb of the binarized object is less than the threshold TH3 and the height ΔHg of the grayscale object is less than the threshold TH4 in step S45 (YES in step S45), the object from the road surface is detected. It is determined whether or not the upper end height position Yt of the object is less than a threshold value TH5 (an appropriate value as the height of a pedestrian) (step S46). In step S46, the upper end height position Y of the object from the road surface is
When t is less than the threshold value TH5 (Y in step S46)
ES), luminance distribution Var_A3 of mask area AREA3
Is greater than the threshold TH6 (step S
47). This processing is performed in the mask area AREA when the object of FIG. 16 is a part or the whole of a pedestrian or a wall.
This will be described with reference to the figure showing the luminance dispersion of No. 3.

Specifically, by setting the area width of the mask area AREA3 as the binarization object width, only the head of the pedestrian is extracted by the binarization processing as shown in FIG. 16 (a). In the case of lighting, there is a difference in brightness from the lower body part. Also, FIG.
As shown in 6 (b), when at least the upper body or the whole body of the pedestrian is extracted by the binarization process, a brightness difference from the background area (image) occurs. On the other hand, FIG.
As shown in Fig. 3, in the case of an object such as a wall where the temperature difference of the entire object is small, the difference in brightness between the binarized extraction part and the part that is not so is small, and the object is composed of a linear part like AREA3. Has been done. Therefore, the luminance dispersion Var_A3 of AREA3 shows a high value for a pedestrian and a low value for an object such as a wall. Therefore, step S4
7, the luminance distribution Var_A3 of the mask area AREA3
Is greater than the threshold value TH6, it is determined whether the object is a pedestrian.

Further, in step S47, when the luminance variance Var_A3 of the mask area AREA3 is larger than the threshold value TH6 (YES in step S47), the pedestrian determination is performed based on the temporal change of the object shape. Specifically, since the target is a pedestrian binarized object, it is considered that the binarized object shape does not change significantly with time.
Therefore, the maximum value Max_Rate, which is the area ratio of the circumscribed quadrangle to the area of the binarized object within the specified time, is Max_Rate.
And the minimum value Min_Rate is less than a threshold TH7 (step S48).

On the other hand, if the width ΔWb of the binarized object is less than the threshold value TH1 or greater than TH2 in step S44 (NO in step S44), or the height ΔHb of the binarized object in step S45. Is greater than or equal to the threshold TH3, or the height Δ of the grayscale object
If Hg is greater than or equal to the threshold value TH4 (NO in step S45), or if the upper end height position Yt of the object from the road surface is greater than or equal to the threshold value TH5 in step S46 (NO in step S46), or step S47.
, The luminance distribution of the mask area AREA3 is the threshold value TH.
If it is 6 or less (NO in step S47), further in step S48, the maximum value M of Rate, which is the area ratio of the circumscribed quadrangle and the area of the binarized object within the specified time, is M.
Difference between ax_Rate and minimum value Min_Rate (Ma
If x_Rate-Min_Rate is equal to or greater than the threshold value TH7 (NO in step S48), it is determined that the object captured in the area AREA0 is not a pedestrian (step S49). The determination process is terminated, and if NO in step S34 shown in FIG.
In step S37, it is determined that the target object is not the alarm target.

In step S48, the maximum value Max_Rate and the minimum value Min_R of Rate, which is the area ratio of the circumscribed quadrangle and the area of the binarized object within the specified time, are determined.
When the difference in ate is less than the threshold value TH7 (YES in step S48), the image processing unit 1 then performs a pedestrian determination for each shape of the extracted target object in more detail.
Specifically, first, the upper end height position Y of the object from the road surface
It is determined whether or not t is larger than a threshold value TH8 (an appropriate value as a height capable of distinguishing the upper half body and the lower half body of a pedestrian) (step S50). In step S50, when the upper end height position Yt of the object from the road surface is less than or equal to the threshold value TH8 (NO in step S50), the process proceeds to step S51 in FIG. 9 and is the lower half of the pedestrian or the pedestrian who sits down. As
It is determined whether or not the width ΔWb of the binarized object is less than or equal to a threshold value TH9 (a value suitable as the width of the body of a pedestrian) (step S5).
1).

FIG. 9 shows a processing procedure for identifying a pedestrian who has a lower body extracted by the binarization process or is sitting. In step S51, the width ΔWb of the binarization target is a threshold value. If it is equal to or lower than TH9 (YES in step S51), the height ΔHg of the grayscale object is determined in order to determine whether the object is a pedestrian sitting down.
Is less than the threshold value TH10 (appropriate value as the height of a pedestrian) (step S52).

In step S52, if the height ΔHg of the grayscale object is greater than or equal to the threshold value TH10 (NO in step S52), it is assumed that this object corresponds to the torso or the lower half of the body of the pedestrian, and the head is placed above the body. In order to determine whether there is a copy, the average brightness value Ave_A1 of the upper mask area AREA1 shown in FIG.
It is determined whether or not it is larger (step S53). In step S53, when the average brightness value Ave_A1 of the mask area AREA1 is larger than the threshold value TH11 (YES in step S53), the body part may be less likely to radiate heat due to the influence of clothes, so that the brightness on the grayscale image is reduced. As an object with a pattern, the mask area A
It is determined whether the luminance variance Var_A2 of REA2 is larger than the threshold TH18 (step S53-1). Then, in step S53-1, the mask area AREA
If the luminance variance Var_A2 of 2 is larger than the threshold TH18 (YES in step S53-1), the area AREA
It is determined that the target object captured by 0 is a pedestrian (step S54), the pedestrian determination process is terminated, the determination in step S34 shown in FIG. 5 is YES, the process proceeds to step S35 in FIG. 5, and the artificial structure determination is performed. To do.

On the other hand, in step S51, when the width ΔWb of the binarized object is larger than the threshold value TH9 (NO in step S51), or in step S53,
When the average brightness value Ave_A1 of the mask area AREA1 is less than or equal to the threshold value TH11 (NO in step S53),
Further, in step S53-1, the mask area ARE
If the luminance variance Var_A2 of A2 is equal to or less than the threshold TH18 (NO in step S53-1), it is determined that the target object captured in the area AREA0 is not a pedestrian (step S55). The pedestrian determination process is ended, the process proceeds to step S37 in FIG. 5 with NO in step S34 shown in FIG. 5, and it is determined that the object is not a warning target.

If the height ΔHg of the grayscale object is less than the threshold value TH10 in step S52 (YES in step S52), this object is regarded as a sitting pedestrian and the binarized object is considered. It is determined whether or not the upper-end height position Yt of the object from the road surface is larger than a threshold value TH12 (a suitable value as a height capable of distinguishing a sitting pedestrian from a standing pedestrian) (step S56). In step S56, when the upper end height position Yt of the object from the road surface of the binarized object is larger than the threshold value TH12 (YES in step S56), the aspect ratio ASPECT of the circumscribed quadrangle of the binarized object is represented. Asp is the threshold TH13
It is determined whether or not the value is TH14 or less (a value appropriate for a pedestrian) (step S57).

In step S57, Asp representing the aspect ratio ASPECT of the circumscribed quadrangle of the binarized object is the threshold value T.
When it is H13 or more and TH14 or less (step S57
), The center of gravity 102 of the circumscribed quadrangle represented by the equation (9)
And the distance Di in the real space between the center of gravity G100 of the binarized object and
It is determined whether s_c is less than a threshold TH15 (a value suitable for a pedestrian) (step S58). Dis_c = SQRT in ^{((Xb-Xc) 2 +} (Yb-Yc) 2) ··· (9) step S58, the distance Dis_c threshold TH1
If it is less than 5 (YES in step S58), for example, an object having a ΔWb of 1.0 m or less and a ΔHg of less than 1.0 m is an object other than a pedestrian, specifically, a front portion of the vehicle. Since it is included, the upper mask area AR of the binarized object
In EA1, it is determined whether or not there is a site having a high degree of correlation with the head pattern registered in advance (step S5).
9).

In step S59, if there is a portion having a high degree of correlation with the head pattern registered in advance in the upper mask area AREA1 of the binarized object (Y in step S59).
ES), the target object captured in the area AREA0 is determined to be a pedestrian (step S54), the pedestrian determination process is terminated, and YES is determined in step S34 shown in FIG. 5, the process proceeds to step S35 in FIG. Determine the structure.

On the other hand, at step S56, the upper end height position Yt of the binarized object from the road surface is the threshold value TH.
If it is 12 or less (NO in step S56), or in step S57, Asp representing the aspect ratio ASPECT of the circumscribed quadrangle of the binarization target is less than the threshold TH13 or greater than TH14 (step S56).
57) or the distance Dis_c is greater than or equal to the threshold value TH15 in step S58 (NO in step S58), and further in step S59, the head pattern registered in advance in the upper mask area AREA1 of the binarized object. If there is no part having a high degree of correlation with (NO in step S59), the area AR
It is determined that the target object captured by EA0 is not a pedestrian (step S55), the pedestrian determination process is terminated, the process proceeds to step S37 in FIG. 5 as NO in step S34 shown in FIG. 5, and the target object is a warning target. Determine not.

Further, in step S50 of FIG. 8, the upper end height position Yt of the object from the road surface is a threshold value TH8 (appropriate value as a height capable of distinguishing the upper half and the lower half of the pedestrian).
If it is larger (YES in step S50), the process shown in FIG.
In step S60 of 0, the height ΔHg of the grayscale object is set to the threshold value TH16 (the above-mentioned threshold value TH16) in order to determine whether the object is an object floating in the air (for example, an object such as a curved mirror). It is determined whether or not it is larger than the threshold TH8) (step S60).

FIG. 10 shows a processing procedure for identifying a pedestrian whose head or upper body is extracted by the binarization processing. In step S60, the height ΔHg of the grayscale object is a threshold value TH16. If it is larger (YES in step S60), the target object is not an object floating in the air, so the target object area (AREA0) is next.
It is determined whether the head part or the body part exists at the upper end part of the. Specifically, first, since the head is exposed, the average luminance value Ave_ of the mask area AREA1 is obtained.
It is determined whether A1 is larger than the threshold TH17 (step S61).

In step S61, the mask area AR
When the average brightness value Ave_A1 of EA1 is larger than the threshold value TH17 (YES in step S61), the body part may not easily dissipate heat due to the influence of clothes, so that as a target object having a brightness pattern on the grayscale image,
The luminance distribution Var_A2 of the mask area AREA2 is the threshold value T.
It is determined whether or not it is larger than H18 (step S6).
2). In step S62, the mask area AR
When the luminance variance Var_A2 of EA2 is larger than the threshold value TH18 (YES in step S62), first, the width ΔWb of the binarization target is determined in order to determine the pedestrian whose head or upper body is extracted by the binarization process. Is the threshold TH19
It is determined whether or not (a suitable value as a width capable of distinguishing the pedestrian's head or the upper half of the body) or less (step S63).

Next, when the width ΔWb of the binarized object is larger than the threshold value TH19 in step S63 (NO in step S63), at least the upper half of the pedestrian or the whole body is walked by the binarization process. In order to determine the person, it is determined whether or not the width ΔWb of the binarized object is less than or equal to a threshold value TH9 (a suitable value as the body width of a pedestrian) (step S64). Further, in step S64, when the width ΔWb of the binarized object is larger than the threshold value TH9 (NO in step S64), binarization is performed to determine whether or not a plurality of pedestrians are performing parallel walking. The width ΔWb of the object is the threshold value TH2 (a suitable value as the width of the body of a pedestrian)
It is determined whether or not the following (step S65).

Further, in the above determination, in step S60, the height ΔHg of the grayscale object is the threshold value TH.
If it is 16 or less (NO in step S60), or in step S61, the average brightness value Ave_A1 of the mask area AREA1 is less than or equal to the threshold TH17 (NO in step S61), or in step S62.
, The luminance distribution Var_A of the mask area AREA2
When 2 is less than or equal to the threshold value TH18 (NO in step S62), and further in step S65, when the width ΔWb of the binarized object is larger than the threshold value TH2 (NO in step S65). Is the area ARE
It is determined that the target object captured by A0 is not a pedestrian (step S66), the pedestrian determination process is terminated, the process proceeds to step S37 of FIG. 5 with NO in step S34 shown in FIG. 5, and the target object is a warning target. Determine not.

On the other hand, in step S63, when the width ΔWb of the binarized object is less than or equal to the threshold value TH19 (YES in step S63), the object is a gait whose head or upper body is extracted by the binarization process. 11, the process proceeds to step S67 in FIG. 11, and Asp representing the aspect ratio ASPECT of the circumscribed quadrangle of the binarization target is the threshold TH20.
It is determined whether or not the value is TH21 or less (a value appropriate for a pedestrian's head or upper body) (step S67).

FIG. 11 shows a processing procedure for identifying a pedestrian whose head or upper body is extracted by the binarization processing. In step S67, the aspect ratio of the circumscribed quadrangle of the binarized object is shown. Asp representing ASPECT is the threshold TH
If 20 or more and TH21 or less (YES in step S67), the distance Dis_c in the real space between the circumscribed quadrangle centroid 102 and the centroid G100 of the binarized object is the threshold T.
It is determined whether it is less than H15 (step S68). In step S68, when the distance Dis_c is less than the threshold value TH15 (YES in step S68), the area AR
It is determined that the target object captured by EA0 is a pedestrian (step S69), the pedestrian determination process is ended, and YES in step S34 shown in FIG. 5 is determined as step S35 in FIG.
Proceed to and judge the artificial structure.

On the other hand, in step S67, if Asp representing the aspect ratio ASPECT of the circumscribed quadrangle of the binarized object is less than the threshold value TH20 or greater than TH21 (NO in step S67), or step S68.
In the case where the distance Dis_c is equal to or greater than the threshold value TH15 (NO in step S68), it is determined that the target object captured in the area AREA0 is not a pedestrian (step S68).
70) Step S shown in FIG.
If NO in 34, the process proceeds to step S37 in FIG. 5, and it is determined that the object is not a warning target.

Further, in step S64 of FIG.
If the width ΔWb of the binarized object is less than or equal to the threshold value TH9 (YES in step S64), it is assumed that the object is at least the upper body of the pedestrian or the whole body is a pedestrian extracted by the binarization process. 12 proceeds to step S71,
It is determined whether or not Asp, which represents the aspect ratio ASPECT of the circumscribed quadrangle of the binarized object, is equal to or more than the threshold value TH13 and less than or equal to TH21 (a value appropriate for the upper body and the whole body of a pedestrian) (step S71).

FIG. 12 shows a processing procedure for identifying a pedestrian whose upper body or whole body is extracted by the binarization processing. In step S71, the aspect ratio ASPECT of the circumscribed quadrangle of the binarized object is shown. Is the threshold TH
When it is 13 or more and TH21 or less (YES in step S71), the distance Dis_c in the real space between the circumscribed quadrangle centroid 102 and the centroid G100 of the binarized object is the threshold T.
It is determined whether it is less than H15 (step S72).

In step S72, the distance Dis_c
Is less than the threshold TH15 (Y in step S72)
ES), the target includes a target other than a pedestrian, for example, the front part of the vehicle, and thus has a high correlation with the head pattern registered in advance in the upper mask area AREA1 of the binarized target. It is determined whether or not the part exists (step S73). In step S73, when there is a portion having a high degree of correlation with the head pattern registered in advance in the upper mask area AREA1 of the binarized object (Y in step S73).
ES), the object captured in the area AREA0 is determined to be a pedestrian (step S74), the pedestrian determination process is terminated, and YES is determined in step S34 shown in FIG. 5, the process proceeds to step S35 in FIG. Determine the structure.

On the other hand, in step S71, if Asp representing the aspect ratio ASPECT of the circumscribed quadrangle of the binarized object is less than the threshold value TH13 or greater than TH21 (NO in step S71), or step S71.
In 72, if the distance Dis_c is greater than or equal to the threshold value TH15 (NO in step S72), further step S
73, the upper mask area AREA of the binarized object
If there is no part having a high degree of correlation with the head pattern registered in advance in 1 (NO in step S73), it is determined that the object captured in the area AREA0 is not a pedestrian. (Step S75) The pedestrian determination process is ended, and if NO in step S34 shown in FIG.
In step S37, it is determined that the target object is not the alarm target.

Further, in step S65 of FIG.
If the width ΔWb of the binarized object is less than or equal to the threshold value TH2 (YES in step S65), the object is in the circumscribed rectangle of the object because the pedestrian is walking in parallel. It is determined that a large amount is included, and the process proceeds to step S76 in FIG. 13, and it is determined whether or not Rate, which is the area ratio of the circumscribed quadrangle within the specified time and the area of the binarized object, is less than the threshold TH22 (step S76 ).

FIG. 13 shows a processing procedure in the case where a plurality of pedestrians are walking in parallel as an object.
When Rate, which is the area ratio of the binarization target, is less than the threshold value TH22 (YES in step S76), Asp indicating the aspect ratio ASPECT of the circumscribed quadrangle of the binarization target.
Is greater than or equal to the threshold value TH23 and less than or equal to TH14 (a value appropriate for determining the pedestrian's parallel walking) (step S77).

In step S77, Asp representing the aspect ratio ASPECT of the circumscribed quadrangle of the binarized object is the threshold T.
When it is H23 or more and TH14 or less (step S77
YES), it is determined whether the distance Dis_c in the real space between the circumscribed quadrangle centroid 102 and the centroid G100 of the binarized object is less than the threshold TH15 (step S78). In step S78, the distance Dis_c is the threshold TH15.
If it is less than (YES in step S78), the area A
It is determined that the object captured by REA0 is a pedestrian (step S79), the pedestrian determination process is ended, and YES in step S34 shown in FIG. 5 is determined as step S35 in FIG.
Proceed to and judge the artificial structure.

On the other hand, in step S76, when the rate, which is the area ratio of the circumscribed quadrangle and the area of the binarized object within the specified time, is greater than or equal to the threshold value TH22 (step S76).
No of 76), or As representing the aspect ratio ASPECT of the circumscribed quadrangle of the binarized object in step S77.
If p is less than the threshold TH23 or greater than TH14 (NO in step S77), further step S7.
If the distance Dis_c is equal to or greater than the threshold value TH15 in step 8 (NO in step S78), it is determined that the object captured in the area AREA0 is not a pedestrian (step S80). The person determination process is terminated, the process proceeds to step S37 of FIG. 5 with NO in step S34 shown in FIG. 5, and it is determined that the target object is not a warning target.

In the present embodiment, the image processing unit 1 includes the binarized object extraction means, the grayscale object extraction means, and the pedestrian discrimination means. More specifically, S1 to S13 in FIG. 3 correspond to the binarized object extracting means, and S41 to S42 in FIG. 8 correspond to the grayscale object extracting means. In addition, S43 to S50 of FIG.
S51 to S59 of FIG. 9, S60 to S66 of FIG. 10, and FIG.
1 S67 to S70, FIG. 12 S71 to S75, FIG.
Steps S76 to S80 correspond to pedestrian discrimination means.

Further, in the present embodiment, the image processing unit 1 includes an object extracting means, a heat storage material extracting means, and a pedestrian recognizing means. More specifically, S1 in FIG.
~ S13 corresponds to the object extraction means, S45, S of FIG.
46, S47, S52, S53, S53-1 in FIG. 9 and S60, S61, S62 in FIG. 10 correspond to the heat storage medium extraction means. Further, S48 to S50 in FIG. 8 and S54 to S in FIG.
59, S63 to S66 in FIG. 10, and S67 to S7 in FIG.
0, S71 to S75 in FIG. 12, S76 to S80 in FIG.
Corresponds to the pedestrian recognition means.

As described above, the vehicle periphery monitoring apparatus according to the present embodiment extracts a target object such as a pedestrian by the binarization process from the grayscale image of the image captured by the infrared camera, and then the grayscale By changing the brightness of the image, a grayscale object in the range including the binarized object is extracted from the grayscale image, and a plurality of search areas are set in the area of the grayscale object. A pedestrian in the search area is recognized based on the brightness distribution of the search area. Thereby, for example, when the width of the image of the object is unnatural as a pedestrian or when the height of the image of the object is unnatural as a pedestrian, these objects are removed from the image of the object and The characteristics of a pedestrian satisfying the above condition are whether there is a portion corresponding to the head with high luminance dispersion, or a portion corresponding to the torso with high luminance dispersion, or even a low luminance dispersion such as a wall. It is possible to improve the pedestrian detection accuracy by removing the image of the object whose luminance dispersion is different from the image of the pedestrian photographed from the image of the target object.

[0077]

As described above, according to the vehicle periphery monitoring device of the first aspect, after the position of the binarized object is recognized on the grayscale image by the binarized object extracting means, the grayscale is detected. By setting the grayscale target object in the range including the binarized target object by the target object extraction means, and calculating the luminance variance for each search area set on the grayscale target object by the pedestrian discrimination means, Whether or not the binarized object is a pedestrian is determined based on the characteristics of the luminance dispersion. Therefore,
It is possible to improve the detection accuracy of a pedestrian by removing the image of an object whose luminance dispersion is different from the image of a pedestrian captured from the image of the object.

According to the vehicle periphery monitoring device of the second aspect, the horizontal width of the search area on the grayscale image for which the luminance dispersion is obtained is set to the horizontal width of the binarized object suitable for recognizing a pedestrian. Then, it can be determined whether the binarized object is a pedestrian. Therefore, it is possible to improve the detection accuracy of a pedestrian by removing from the image of the object an object having the same brightness dispersion characteristic as the image of the pedestrian, such as a wall.

According to the vehicle periphery monitoring device of the third aspect, the search area on the grayscale image for obtaining the luminance dispersion is set to a size suitable for recognizing the head of a pedestrian and binarized. It is possible to determine whether the object is a pedestrian.
Therefore, it is possible to improve the pedestrian detection accuracy by removing from the image of the object an object having a size different from the size of the human head, which is characteristic of the brightness distribution of the image of the pedestrian. can get.

According to the vehicle periphery monitoring device of the fourth aspect, the search area of a size suitable for recognizing the head of the pedestrian and the trunk of the pedestrian are displayed on the grayscale image for which the luminance dispersion is obtained. It is possible to determine whether or not the binarized object is a pedestrian by setting a search area having an appropriate size for recognizing a part. Therefore, it is possible to improve the pedestrian detection accuracy by removing from the image of the object an object having the same luminance dispersion characteristic as the image of the pedestrian, such as a curved mirror. .

According to the vehicle periphery monitoring device of the fifth aspect, the heat storage body extraction means causes the object to be erroneously recognized as a pedestrian due to the influence of the external environment although the nature of the object is different from that of the pedestrian. By removing the heat storage that may be recognized, it is possible to determine whether or not it is a pedestrian from the objects that are similar in nature to the pedestrian as much as possible. Therefore, for example, an object that emits heat (infrared rays) equivalent to that of a pedestrian, such as a wall that receives sunlight from the sun, is removed from the target object based on the characteristics of its brightness dispersion, and The effect that the detection accuracy can be improved is obtained.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a vehicle periphery monitoring device according to an embodiment of the present invention.

FIG. 2 is a diagram showing mounting positions of an infrared camera, a sensor, a display and the like in a vehicle.

FIG. 3 is a flowchart showing an object detection / warning operation of the vehicle periphery monitoring device of the same embodiment.

FIG. 4 is a diagram showing a grayscale image obtained by an infrared camera and a binarized image thereof.

FIG. 5 is a flowchart showing an alarm determination processing operation of the same embodiment.

FIG. 6 is a diagram showing a case where a collision is likely to occur.

FIG. 7 is a diagram showing an area division in front of the vehicle.

FIG. 8 is a flowchart showing a pedestrian determination processing operation of the same embodiment.

FIG. 9 is a flowchart showing a pedestrian determination processing operation of the same embodiment.

FIG. 10 is a flowchart showing a pedestrian determination processing operation of the same embodiment.

FIG. 11 is a flowchart showing a pedestrian determination processing operation of the same embodiment.

FIG. 12 is a flowchart showing a pedestrian determination processing operation of the same embodiment.

FIG. 13 is a flowchart showing a pedestrian determination processing operation of the same embodiment.

FIG. 14 is a diagram showing a binarized object shape feature amount according to the same embodiment.

FIG. 15 is a diagram showing mask area setting according to the embodiment.

FIG. 16 is a diagram showing the luminance dispersion of the mask area AREA3 when the object is a part or the whole of a pedestrian or a wall.

[Explanation of symbols]

1 Image processing unit 2R, 2L Infrared camera 3 Yaw rate sensor 4 Vehicle speed sensor 5 Brake sensor 6 Speaker 7 Image display device 10 Own vehicle S1 to S13 Binary object extracting means S41 to S42 Gray scale object extracting means S43 to S80 Walking Person discriminating means S1 to S13 Object extracting means S45, S46, S47, S52, S53, S53-
1, S60, S61, S62 Heat storage body extraction means S48-S50, S54-S59, S63-S80
Pedestrian recognition means

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) B60R 21/00 B60R 21/00 626C 626E G06T 1/00 315 G06T 1/00 315 330 330B 340 340B 7/20 200 7/20 200A (72) Masato Watanabe Inventor Masato Watanabe 1-4-1 Chuo, Wako-shi, Saitama Stock Research Institute Honda Technical Research Institute (72) Inventor Hiroshi Hattori 1-4-1 Chuo, Wako-shi, Saitama Stock Association F term in Honda R & D Co., Ltd. (reference) 5B057 AA16 AA19 BA08 BA13 CA08 CA13 CA16 CE12 CG04 DA08 DA11 DC14 DC22 5C054 CA05 FC12 HA30 5L096 AA06 BA02 BA04 CA04 CA05 EA43 FA02 FA19 FA33 GA34 GA36 HA03 JA18

Claims (5)

[Claims] 1. A vehicle periphery monitoring device for recognizing a pedestrian using images captured by two infrared cameras, wherein the grayscale image of the image is binarized to obtain the grayscale image. Binary object extracting means for extracting a binarized object from the image, and a grayscale object in a range including the binarized object from the grayscale image by changing the brightness of the grayscale image. And a grayscale object extracting means to set a search area in the area of the grayscale object,
A vehicle surroundings monitoring device comprising: a pedestrian discriminating means for recognizing a pedestrian in the gray scale image based on a luminance distribution of the search area. 2. The pedestrian discrimination means sets the horizontal size of the search area as the width of the binarized object image, and the vertical size of the search area as the height of the grayscale object image. The vehicle surroundings monitoring device according to claim 1, wherein: 3. The pedestrian discrimination means sets, as the search region, a head region having a size corresponding to the head of the pedestrian with reference to the upper end of the grayscale object. The vehicle periphery monitoring device according to claim 1. 4. The pedestrian discrimination means, as the search region, a head region having a size corresponding to a pedestrian's head with respect to the upper end of the grayscale object, and a region below the head region. The vehicle periphery monitoring device according to claim 1, wherein a torso region that is larger than the head region corresponding to the torso of the pedestrian is set. 5. A vehicle periphery monitoring device for detecting an object existing in the periphery of a vehicle from an infrared image taken by an infrared camera, and an object extracting means for extracting an object emitting infrared rays from the infrared image. From the object extracted by the object extracting means, a heat storage body extracting means for extracting a heat storage body that stores only the heat given from the outside without generating heat by itself, and the heat storage body extracting means A vehicle surroundings monitoring device comprising: a pedestrian recognition unit that recognizes a pedestrian from the objects excluding the heat storage body.