JP3939626B2 - Vehicle periphery monitoring device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a vehicle periphery monitoring device that extracts an object by binarization processing of an image photographed by an infrared camera.
[0002]
[Prior art]
A conventional vehicle periphery monitoring device extracts an object such as a pedestrian that may collide with the host vehicle from an image around the host vehicle captured by an imaging means such as an infrared camera, and automatically acquires the information. Provide to the vehicle driver. In this device, a portion having a high temperature in an image around the host vehicle captured by a pair of left and right stereo cameras is used as a target, and the distance to the target is calculated by obtaining the parallax of the target. From the moving direction of the object and the position of the object, an object that is likely to affect the traveling of the host vehicle is detected and an alarm is output (for example, see Patent Document 1).
[0003]
[Patent Document 1]
JP 2001-6096 A
[0004]
[Problems to be solved by the invention]
By the way, when a pedestrian is extracted only by shape determination as in a conventional device, the pedestrian on an infrared image has two values depending on the influence of a cap and clothing and the presence environment of the pedestrian. The shape is irregular, and in general, when driving a vehicle, there is a change in the shape of the front road surface and the effect of the pitching of the vehicle, so that pedestrians are detected with different heights from children to adults. For this reason, the center-of-gravity coordinates on the screen of the object cannot be fixed with respect to the distance, and only the pedestrians may not be stably extracted.
[0005]
For this reason, conventionally, a method for calculating the size of an object in real space from a gray scale image and extracting only a pedestrian-like object based on the positional relationship in which the binarized object exists, or a binarized object A method has been proposed in which road structures and vehicles are extracted from objects and excluded from alarm objects as objects other than pedestrians. However, there was a problem that only the pedestrians could not be extracted stably because the amount of infrared radiation was changed by the influence of rainfall.
[0006]
In other words, in a general environment, a heat storage body that stores only heat given from outside without generating heat, such as a signboard, a wall, or a power pole, does not appear on the infrared image because the temperature decreases due to rain (infrared camera). However, although self-heating heating elements such as vending machines are detected by an infrared camera, the infrared radiation part is reduced (it is infinite) and it is difficult to accurately determine the shape. There is. In addition, a part (head or the like) where a human is exposed is detected, but a part covered with clothes is not detected by the infrared camera due to the clothes getting wet. Thus, in rainy weather and other times, the situation around the vehicle changes even at the same point, and the shape of all objects on the grayscale image detected by the infrared camera changes, There is a possibility that only the pedestrians cannot be stably extracted by the conventional method.
Furthermore, during rainy weather, since rain adheres to the surface of the camera lens, the shape of the object on the image tends to be unclear, and there is a problem that it is difficult to determine the shape of the object.
[0007]
The present invention has been made in view of the above problems, and is a vehicle periphery monitoring device that accurately determines an irregular binarized object extracted from an image photographed by a camera and extracts a stable pedestrian. The purpose is to provide.
[0008]
[Means for Solving the Problems]
In order to solve the above-described problem, a vehicle periphery monitoring device according to the invention of claim 1 includes: Two A vehicle periphery monitoring device that detects an object existing around a vehicle from an infrared image captured by an infrared camera, and includes weather detection means for detecting the weather around the vehicle (for example, step S41-1 in the embodiment) ), Object extraction means for extracting an object that emits infrared light from the infrared image (for example, Steps S1 to S13 in the embodiment), and the object extracted by the object extraction means itself generates heat. A heat storage body extracting means for extracting a heat storage body that stores only the heat given from the outside without any heat (for example, steps S45, S46, S47, S52, S53, S53-1, S60, S61, S62 of the embodiment); If the weather around the vehicle is determined to be rainy by the weather detection means, a pedestrian is recognized from the object, otherwise The pedestrian recognition means for recognizing the pedestrian from the object excluding the heat storage body extracted by the heat storage body extraction means (for example, step S48 to step S50, step S54 to step S59, step S63 of the embodiment). To step S80).
[0009]
In the vehicle periphery monitoring device having the above configuration, when the weather detection unit determines that the weather around the vehicle is rainy, the amount of infrared rays emitted from the object extracted by the object extraction unit decreases. Therefore, the pedestrian recognition means recognizes the pedestrian directly from the extracted object. On the other hand, if it is determined that the weather around the vehicle is other than that, only the heat given from the outside is generated by the amount of infrared rays radiated from the object extracted by the object extracting means. Since the difference between the heat storage body that extracts the heat storage body that stores heat and the heat generation body that generates heat like a pedestrian appears in the luminance dispersion of the object image, the heat storage body extraction means extracts the heat storage body from the object, The pedestrian recognition means can perform appropriate pedestrian recognition according to the situation around the vehicle by recognizing the pedestrian from the object excluding the heat storage body.
[0010]
A vehicle periphery monitoring device according to a second aspect of the present invention is the vehicle periphery monitoring device according to the first aspect, wherein the pedestrian recognition means determines the shape of the object (for example, S59 in the embodiment). , S73), and when the weather detecting means determines that the weather around the vehicle is rainy, the shape determining means is stopped.
In the vehicle periphery monitoring device having the above-described configuration, when the weather detection unit determines that the weather around the vehicle is rainy, the shape determination process is stopped, and rain adheres to the surface of the camera lens. Therefore, it is possible to prevent the object shape on the image that has become obscured from being erroneously determined by the pedestrian recognition means.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of a vehicle periphery monitoring device according to an embodiment of the present invention.
In FIG. 1, reference numeral 1 denotes an image processing unit including a CPU (central processing unit) that controls the vehicle periphery monitoring device according to the present embodiment, and includes two infrared cameras 2R and 2L capable of detecting far infrared rays. A yaw rate sensor 3 for detecting the yaw rate of the vehicle, a vehicle speed sensor 4 for detecting a traveling speed (vehicle speed) of the vehicle, and a brake sensor 5 for detecting a brake operation are connected. As a result, the image processing unit 1 detects pedestrians and animals in front of the vehicle from the infrared image around the vehicle and a signal indicating the running state of the vehicle, and issues an alarm when it is determined that the possibility of a collision is high. To emit.
[0012]
In addition, the image processing unit 1 displays a speaker 6 for issuing a warning by voice and images taken by the infrared cameras 2R and 2L, and makes the vehicle driver recognize an object having a high risk of collision. For example, a meter-integrated display integrated with a meter that expresses the running state of the host vehicle, a NAVID display installed on the console of the host vehicle, and information displayed at a position on the front window that does not obstruct the driver's front view An image display device 7 including a HUD (Head Up Display) 7a is connected.
[0013]
The image processing unit 1 includes an A / D conversion circuit that converts an input analog signal into a digital signal, an image memory that stores a digitized image signal, a CPU (central processing unit) that performs various arithmetic processes, RAM (Random Access Memory) used to store data, ROM (Read Only Memory) that stores programs and tables executed by the CPU, maps, etc., speaker 6 drive signals, display signals such as HUD7a, etc. The output signals of the infrared cameras 2R and 2L, the yaw rate sensor 3, the vehicle speed sensor 4 and the brake sensor 5 are converted into digital signals and input to the CPU.
[0014]
As shown in FIG. 2, the infrared cameras 2R and 2L are disposed at the front portion of the host vehicle 10 at substantially target positions with respect to the center of the host vehicle 10 in the vehicle width direction. The optical axes of 2R and 2L are parallel to each other and are fixed so that the heights from both road surfaces are equal. The infrared cameras 2R and 2L have a characteristic that the output signal level increases (the luminance increases) as the temperature of the object increases.
Further, the HUD 7a is provided so that the display screen is displayed at a position that does not obstruct the driver's front view of the front window of the host vehicle 10.
[0015]
Next, the operation of the present embodiment will be described with reference to the drawings.
FIG. 3 is a flowchart showing an object detection / alarm operation of a pedestrian or the like in the image processing unit 1 of the vehicle periphery monitoring device of the present embodiment.
First, the image processing unit 1 acquires infrared images, which are output signals of the infrared cameras 2R and 2L (step S1), performs A / D conversion (step S2), and stores a grayscale image in an image memory (step S1). S3). Here, the right image is obtained by the infrared camera 2R, and the left image is obtained by the infrared camera 2L. In addition, since the horizontal position of the same object on the display screen is shifted in the right image and the left image, the distance to the object can be calculated from this shift (parallax).
[0016]
If a grayscale image is obtained in step S3, then the right image obtained by the infrared camera 2R is used as a reference image, and binarization processing of the image signal, that is, an area brighter than the luminance threshold value ITH is “1” ( White) and the dark area is set to “0” (black) (step S4).
FIG. 4A shows a grayscale image obtained by the infrared camera 2R, and binarization processing is performed on the grayscale image to obtain an image as shown in FIG. 4B. In FIG. 4B, for example, an object surrounded by a frame from P1 to P4 is an object displayed as white on the display screen (hereinafter referred to as “high luminance region”).
When the binarized image data is acquired from the infrared image, the binarized image data is converted into run-length data (step S5). The line represented by the run-length data is an area that is whitened by binarization at the pixel level, and has a width of one pixel in the y direction, and each has a width in the x direction. It has the length of the pixels constituting the run-length data.
[0017]
Next, the target object is labeled from the image data converted into run-length data (step S6), thereby performing a process of extracting the target object (step S7). That is, of the lines converted into run-length data, a line having a portion overlapping in the y direction is regarded as one object, so that, for example, the high luminance regions P1 to P4 shown in FIG. To be grasped as a value object).
When the extraction of the object is completed, next, the center of gravity G, the area S, and the aspect ratio ASPECT of the circumscribed rectangle of the extracted object are calculated (step S8).
[0018]
Here, the area S is (x [i], y [i], run [i], A) (i = 0, 1, 2,... N−1) ), The length of the run length data (run [i] −1) is calculated for the same object (N pieces of run length data). Also, the coordinates (xc, yc) of the center of gravity G of the object A are the length (run [i] -1) of each run length data and the coordinates x [i] or y [i] of each run length data. Are multiplied by the area S and calculated by multiplying them by the area S. Further, the aspect ratio ASPECT is calculated as a ratio Dy / Dx between the length Dy in the vertical direction and the length Dx in the horizontal direction of the circumscribed rectangle of the object.
Since the run-length data is indicated by the number of pixels (number of coordinates) (= run [i]), the actual length needs to be “−1” (subtract 1) (= run [i]. ] -1). Further, the position of the center of gravity G may be substituted by the position of the center of gravity of the circumscribed rectangle.
[0019]
Once the center of gravity, area, and circumscribing aspect ratio of the object can be calculated, next, the object is tracked between times, that is, the same object is recognized for each sampling period (step S9). For tracking between times, the time obtained by discretizing the time t as an analog quantity with the sampling period is set as k. For example, when the objects A and B are extracted at the time k, the objects C and D extracted at the time (k + 1) and the target The identity determination with the objects A and B is performed. When it is determined that the objects A and B and the objects C and D are the same, the objects C and D are changed to labels of the objects A and B, respectively, so that tracking is performed for a time.
Further, the position coordinates (center of gravity) of each object recognized in this way are stored in the memory as time-series position data and used for the subsequent calculation processing.
[0020]
Note that the processes in steps S4 to S9 described above are performed on a binarized reference image (in this embodiment, a right image).
Next, the vehicle speed VCAR detected by the vehicle speed sensor 4 and the yaw rate YR detected by the yaw rate sensor 3 are read, and the turning angle θr of the host vehicle 10 is calculated by integrating the yaw rate YR over time (step S10).
[0021]
On the other hand, in parallel with the processing of step S9 and step S10, in steps S11 to S13, processing for calculating the distance z between the object and the host vehicle 10 is performed. Since this calculation requires a longer time than steps S9 and S10, it is executed in a cycle longer than steps S9 and S11 (for example, a cycle that is about three times the execution cycle of steps S1 to S10).
First, by selecting one of the objects tracked by the binarized image of the reference image (right image), the search image R1 (here, the entire region surrounded by the circumscribed rectangle is searched for from the right image). Are extracted (step S11).
[0022]
Next, a search area for searching for an image corresponding to the search image R1 (hereinafter referred to as “corresponding image”) from the left image is set, and a correlation operation is executed to extract the corresponding image (step S12). Specifically, a search area R2 is set in the left image according to each vertex coordinate of the search image R1, and a luminance difference total value C (a indicating the level of correlation with the search image R1 within the search area R2 , B), and a region where the total value C (a, b) is minimum is extracted as a corresponding image. This correlation calculation is performed using a grayscale image instead of a binarized image.
When there is past position data for the same object, an area R2a narrower than the search area R2 is set as a search area based on the position data.
[0023]
The search image R1 is extracted from the reference image (right image) and the corresponding image R4 corresponding to the target object is extracted from the left image by the processing in step S12. Next, the search image R1 corresponds to the center of gravity position of the search image R1. The position of the center of gravity of the image R4 and the amount of parallax Δd (number of pixels) are obtained, and the distance z between the host vehicle 10 and the object is calculated therefrom (step S13).
Next, when the calculation of the turning angle θr in step S10 and the distance calculation with the object in step S13 are completed, the coordinates (x, y) and the distance z in the image are changed to real space coordinates (X, Y, Z). Conversion is performed (step S14).
Here, the real space coordinates (X, Y, Z) are, as shown in FIG. 2, with the origin O as the midpoint position (position fixed to the host vehicle 10) of the attachment position of the infrared cameras 2R, 2L. The coordinates in the image are determined as shown in the figure, with the center of the image as the origin and the horizontal direction as x and the vertical direction as y.
[0024]
When the real space coordinates are obtained, the turning angle correction for correcting the positional deviation on the image due to the turning of the host vehicle 10 is performed (step S15). In the turning angle correction, when the host vehicle 10 turns, for example, to the left by the turning angle θr during the period from time k to (k + 1), the range of the image is shifted in the x direction by Δx on the image obtained by the camera. This is a process for correcting this.
In the following description, the coordinates after the turning angle correction are displayed as (X, Y, Z).
[0025]
When the turning angle correction with respect to the real space coordinates is completed, N (for example, about N = 10) real space position data after the turning angle correction obtained during the monitoring period of ΔT for the same object, that is, From the time series data, an approximate straight line LMV corresponding to the relative movement vector between the object and the host vehicle 10 is obtained.
Next, the latest position coordinates P (0) = (X (0), Y (0), Z (0)) and (N-1) position coordinates P (N−1) before the sample (before time ΔT). = (X (N-1), Y (N-1), Z (N-1)) is corrected to a position on the approximate straight line LMV, and the corrected position coordinates Pv (0) = (Xv (0), Yv (0), Zv (0)) and Pv (N-1) = (Xv (N-1), Yv (N-1), Zv (N-1)) are obtained.
[0026]
Thereby, a relative movement vector is obtained as a vector from the position coordinates Pv (N−1) to Pv (0) (step S16).
In this way, by calculating an approximate straight line that approximates the relative movement locus of the object with respect to the host vehicle 10 from a plurality (N) of data within the monitoring period ΔT, the influence of the position detection error is reduced. Thus, the possibility of collision with the object can be predicted more accurately.
[0027]
If the relative movement vector is obtained in step S16, an alarm determination process for determining the possibility of collision with the detected object is performed (step S17). Details of the alarm determination process will be described later.
If it is determined in step S17 that there is no possibility of collision between the host vehicle 10 and the detected object (NO in step S17), the process returns to step S1 and the above-described processing is repeated.
If it is determined in step S17 that there is a possibility of collision between the host vehicle 10 and the detected object (YES in step S17), the process proceeds to an alarm output determination process in step S18.
[0028]
In step S18, it is determined whether or not the driver of the host vehicle 10 is performing a brake operation from the output BR of the brake sensor 5, thereby determining whether or not to perform alarm output determination processing, that is, whether or not alarm output is performed (step S18). Step S18).
If the driver of the host vehicle 10 is performing a brake operation, the acceleration Gs generated (deceleration direction is positive) is calculated, and if the acceleration Gs is greater than a predetermined threshold GTH, the brake is It is determined that the collision is avoided by the operation, the alarm output determination process is terminated (NO in step S18), the process returns to step S1, and the above process is repeated.
As a result, when an appropriate brake operation is performed, an alarm is not issued, and the driver is not bothered excessively.
[0029]
Further, when the acceleration Gs is equal to or less than the predetermined threshold GTH, or when the driver of the host vehicle 10 is not performing a brake operation, the process immediately proceeds to the process of step S19 (YES in step S18), and may contact the object. Therefore, an alarm is issued by voice through the speaker 6 (step S19), and an image obtained by, for example, the infrared camera 2R is output to the image display device 7 so that the approaching object 10 Is displayed as an emphasized image for the driver (step S20).
The predetermined threshold GTH is a value corresponding to a condition in which the host vehicle 10 stops at a travel distance equal to or less than the distance Zv (0) between the object and the host vehicle 10 when the acceleration Gs during the brake operation is maintained as it is. It is.
[0030]
The above is the object detection / alarm operation in the image processing unit 1 of the vehicle periphery monitoring device of the present embodiment. Next, referring to the flowchart shown in FIG. 5, step S17 of the flowchart shown in FIG. The alarm determination process in will be described in more detail.
FIG. 5 is a flowchart showing the alarm determination processing operation of the present embodiment.
The warning determination process includes the object detected as the vehicle 10 by the following collision determination process, determination process whether or not the vehicle is in the approach determination area, approach collision determination process, pedestrian determination process, and artificial structure determination process. This is a process for determining the possibility of collision. Hereinafter, as shown in FIG. 6, a case where there is an object 20 traveling at a speed Vp from a direction of approximately 90 ° with respect to the traveling direction of the host vehicle 10 will be described as an example.
[0031]
In FIG. 5, first, the image processing unit 1 performs a collision determination process (step S31). In the collision determination process, in FIG. 6, when the object 20 approaches the distance Zv (0) from the distance Zv (N−1) during the time ΔT, the relative speed Vs in the Z direction with the host vehicle 10 is obtained. This is a process for determining whether or not they collide within the margin time T on the assumption that both move within the height H while maintaining the relative speed Vs. Here, the margin time T is intended to determine the possibility of a collision by a time T before the predicted collision time. Accordingly, the margin time T is set to about 2 to 5 seconds, for example. H is a predetermined height that defines a range in the height direction, and is set to about twice the vehicle height of the host vehicle 10, for example.
[0032]
Next, in step S31, when there is a possibility that the host vehicle 10 and the target object collide within the margin time T (YES in step S31), the image processing unit 1 sets the target in order to further improve the reliability of the determination. A determination process is performed as to whether or not an object exists in the approach determination area (step S32). As shown in FIG. 7, if the area that can be monitored by the infrared cameras 2R and 2L is an outer triangular area AR0 indicated by a thick solid line, as shown in FIG. A region AR1 that is closer to the host vehicle 10 than Vs × T and corresponds to a range in which the object has a margin β (for example, about 50 to 100 cm) on both sides of the vehicle width α of the host vehicle 10, that is, the target This is a process for determining whether or not an object exists in the approach determination area AR1 where the possibility of a collision with the host vehicle 10 is extremely high if the object continues to exist. The approach determination area AR1 also has a predetermined height H.
[0033]
Furthermore, when the target object does not exist in the approach determination area in step S32 (NO in step S32), the image processing unit 1 may enter the approach determination area and collide with the host vehicle 10. An approach collision determination process is performed to determine whether or not (step S33). In the approach collision determination process, the areas AR2 and AR3 in which the absolute value of the X coordinate is larger than the above-described approach determination area AR1 (outside in the lateral direction of the approach determination area) are called the entry determination areas, and the objects in this area are This is a process for determining whether or not the vehicle enters the approach determination area AR1 and collides with the host vehicle 10 by moving. The entry determination areas AR2 and AR3 also have a predetermined height H.
[0034]
On the other hand, when the target object is present in the approach determination area in step S32 (YES in step S32), the image processing unit 1 determines whether the target object is a pedestrian or not. Processing is performed (step S34). The details of the pedestrian determination process will be described later.
If it is determined in step S34 that the object is likely to be a pedestrian (YES in step S34), it is determined whether or not the object is an artificial structure in order to further increase the reliability of the determination. An artificial structure determination process is performed (step S35). The artificial structure determination process is a process in which, for example, when a feature that is impossible for a pedestrian as shown below is detected in the target object image, the target object is determined as an artificial structure and excluded from the target of the alarm It is.
(1) A case where a portion showing a straight edge is included in an image of an object.
(2) When the corner of the image of the object is a right angle.
(3) When an object image includes a plurality of objects having the same shape.
(4) When the image of the object matches the shape of the artificial structure registered in advance.
[0035]
Therefore, in the above-described step S33, when there is a possibility that the target object enters the approach determination area and collides with the own vehicle 10 (YES in step S33), and in step S35, there is a possibility of a pedestrian. When the determined object is not an artificial structure (NO in step S35), the image processing unit 1 has a possibility of collision between the host vehicle 10 and the detected object (being an alarm target). A determination is made (step S36), the process proceeds to step S18 as YES in step S17 shown in FIG. 3, and an alarm output determination process (step S18) is performed.
[0036]
On the other hand, in step S31 described above, when there is no possibility that the host vehicle 10 and the target object collide within the margin time T (NO in step S31), or in step S33, the target object enters the approach determination area. If there is no possibility of colliding with the host vehicle 10 (NO in step S33), or if it is determined in step S34 that the object is not likely to be a pedestrian (NO in step S34), further step S35 is performed. When the object determined to be a pedestrian is an artificial structure (YES in step S35), the image processing unit 1 detects the host vehicle 10. It is determined that there is no possibility of collision with the object (not an alarm target) (step S37), and the process proceeds to step S1 as NO in step S17 shown in FIG. Ri, repeat the object detection and alarm operation such as a pedestrian.
[0037]
Next, the pedestrian determination process in step S34 of the flowchart shown in FIG. 5 will be described in more detail with reference to the flowcharts shown in FIGS. 8 to 13 are flowcharts showing the pedestrian determination processing operation of the present embodiment.
In FIG. 8, first, the image processing unit 1 determines that the binarized object centroid G (xc, yc) calculated in step S8 of the flowchart shown in FIG. 3 (the binarized object centroid shown in FIG. 14). G100), the area S (the binarized object area S101 shown in FIG. 14), the aspect ratio ASPECT of the circumscribed rectangle of the object, and the distance z between the vehicle 10 and the object calculated in step S13. , Using the values of the height hb and width wb of the circumscribed rectangle of the binarized object shown in FIG. 14 and the coordinates of the circumscribed rectangle centroid (xb, yb) (circumscribed rectangle centroid 102 shown in FIG. 14), The binarized object shape feature amount indicating the shape characteristic of the binarized object at is calculated (step S41). Note that the binarized object shape feature value to be obtained is a parallax calculated by correlation calculation of the base line length D [m], camera focal length f [m], pixel pitch p [m / pixel], and left and right images. It calculates using quantity (DELTA) d [pixel].
[0038]
Specifically, the ratio Rate between the circumscribed rectangle and the object area is:
Rate = S / (hb × wb) (1)
Asp representing the aspect ratio ASPECT of the circumscribed square is
Asp = hb / wb (2)
The distance z between the host vehicle 10 and the object is:
z = (f × D) / (Δd × p) (3)
It is expressed as
The width ΔWb and height ΔHb of the binarized object in real space are
ΔWb = wb × z × p / f
ΔHb = hb × z × p / f (4)
[0039]
The barycentric coordinates (Xc, Yc, Zc) of the binarized object are
Xc = xc * z * p / f
Yc = yc × z × p / f
Zc = z (5)
The object circumscribed square barycentric coordinates (Xb, Yb, Zb) are
Xb = xb * z * p / f
Yb = yb × z × p / f
Zb = z (6)
The upper end position coordinates (Xt, Yt, Zt) of the binarized object are
Xt = xb * z * p / f
Yt = yb × z × p / f−ΔHb / 2
Zt = z (7)
Can be calculated.
[0040]
When the binarized object shape feature amount is calculated, a weather detection process for detecting the weather around the host vehicle 10 is performed (step S41-1).
The weather detection process is a process for obtaining, for example, a luminance histogram of a grayscale image obtained by the infrared camera 2R, and determining the weather around the host vehicle 10 based on the luminance histogram. This is a process for determining whether or not the weather is rainy. Specifically, for example, in a grayscale image in fine weather as shown in FIG. 17A and a grayscale image in rainy weather as shown in FIG. Since there is a difference in the amount of infrared rays to be obtained, when the respective luminance histograms are obtained, the result as shown in FIG. 18 is obtained.
[0041]
FIG. 18 is a graph comparing the luminance histogram of a grayscale image in rainy weather with the luminance histogram of a grayscale image in fine weather. As shown in FIG. As compared with the case of, the standard deviation σ of the luminance histogram of the entire image tends to be small (the interval of 2σ is narrowed). Therefore, when the standard deviation σ of the luminance histogram of the entire image is less than the threshold value TH24, the weather around the host vehicle 10 is determined to be rainy.
[0042]
Whether the weather around the host vehicle 10 is rainy or not is determined from a raindrop sensor for detecting rainfall mounted on the host vehicle 10 regardless of the luminance histogram of the gray scale image as described above. And a signal for controlling the operation of the wiper for wiping raindrops on the window glass (for example, an ON / OFF signal for the wiper).
[0043]
If the weather around the host vehicle 10 can be determined in step S41-1, it is determined whether or not the weather around the host vehicle 10 is rainy (step S41-2). If the weather around the host vehicle 10 is not rainy in step S41-2 (NO in step S41-2), the grayscale image acquired in step S3 of the flowchart shown in FIG. In step S42, the height of the object on the gray scale image including the binarized object extracted in step S7 is obtained.
[0044]
The method for obtaining the height of an object on a gray scale image is to set a mask area of a predetermined size on the gray scale image by arranging a plurality of mask areas on the circumscribed rectangle of the binarized object on the gray scale image. The change is large (the object and the background image are included), and the correlation degree of the mask area between the left and right images is high (two or more objects do not exist in the mask area), and further binary A region including a mask region having the same distance (same parallax) as the conversion target object is extracted as a gray scale target region.
Then, the height Height (pixel) of the region of the gray scale object on the image is calculated, and the height ΔHg of the gray scale object is calculated by the equation (8).
ΔHg = z × Height × p / f (8)
[0045]
Also, as shown in FIG. 15, the area of the gray scale object on the image is AREA0, and mask areas AREA1, AREA2, and AREA3 are set therein, and the luminance average value of each mask and the luminance change (dispersion) Is calculated (step S43). Here, the luminance average value of AREA1 is Ave_A1, the luminance variance of AREA2 is Var_A2, and the luminance variance of AREA3 is Var_A3. In the following processing, AREA1 is used for determining the presence of the head of the object, AREA2 is used for determining the presence of the trunk of the object, and AREA3 is used for determining the presence of a shape change from the head to the lower body. AREA3 is a heat storage body that stores only heat given from the outside without generating heat, such as a wall, and a part of an object showing a monotonous change in luminance is extracted by binarization processing. If so, it is also used to distinguish it from a pedestrian. FIG. 15 schematically shows a pedestrian captured by a camera. A hatched area is a part of an object captured by binarization, and an area surrounded by a dotted line is binarized. Although it is not captured, it represents a part of an object in which the presence of an object can be confirmed with respect to the background in a grayscale image. Moreover, the dimension of each part shown in FIG. 15 is an example of the dimension of each part in real space.
[0046]
Then, when the setting of the mask areas AREA1, AREA2, and AREA3 is completed, the pedestrian determination based on the shape of the binarized object and the pedestrian determination using the luminance dispersion of each mask area of the gray scale image are executed as follows.
First, the image processing unit 1 determines whether or not the height, width, existence height, luminance average value, and luminance variance of the binarized object are values within an appropriate range for a pedestrian.
Specifically, since the target is a pedestrian, it is determined whether or not the width ΔWb of the binarized object is not less than a threshold value TH1 and not more than TH2 (appropriate value as the width of the pedestrian) (step S44).
[0047]
In step S41-2, if the weather around the host vehicle 10 is rainy (YES in step S41-2), the height calculation of the object on the grayscale image in step S42 or step S43 or the mask area is performed. Since AREA1, AREA2, and AREA3 are not set, the process proceeds to step S44 to determine whether or not the width ΔWb of the binarized object is greater than or equal to a threshold value TH1 and less than TH2 (step S44).
[0048]
In step S44, when the width ΔWb of the binarized object is not less than the threshold value TH1 and not more than TH2 (YES in step S44), it is determined whether or not the weather around the host vehicle 10 is rainy (step S44-). 1).
In step S44-1, when the weather around the host vehicle 10 is not rainy (NO in step S44-1), the binarized object height ΔHb is set to the threshold TH3 (appropriate as the pedestrian height). It is determined whether or not the height ΔHg of the gray scale object is less than a threshold TH4 (a value appropriate for the height of the pedestrian) (step S45).
[0049]
On the other hand, in step S44-1, when the weather around the host vehicle 10 is rainy (YES in step S44-1), the binarized object height ΔHb is set to the threshold TH3 (appropriate as the height of the pedestrian). It is determined whether or not the value is less than (S45-1).
In step S45, when the height ΔHb of the binarized object is less than the threshold value TH3 and the height ΔHg of the grayscale object is less than the threshold value TH4 (YES in step S45), or step S45-1 When the height ΔHb of the binarized object is less than the threshold value TH3 (YES in step S45-1), the upper end height position Yt of the object from the road surface is the threshold value TH5 (appropriate as the pedestrian height) It is determined whether or not it is less than (a value) (step S46).
[0050]
In step S46, when the upper end height position Yt of the object from the road surface is less than the threshold value TH5 (YES in step S46), it is determined whether or not the weather around the host vehicle 10 is rainy (step S46). -1).
In step S46-1, if the weather around the host vehicle 10 is not rainy (NO in step S46-1), it is determined whether or not the luminance variance Var_A3 of the mask area AREA3 is larger than the threshold value TH6 (step S46-1). S47). This processing will be described with reference to a diagram showing the luminance dispersion of the mask area AREA3 when the object of FIG. 16 is a part or the whole of a pedestrian or a wall.
[0051]
Specifically, when the area width of the mask area AREA3 is set to the binarized object width, as shown in FIG. 16A, when only the head of the pedestrian is extracted by the binarization process, A brightness difference from the lower body part occurs. Also, as shown in FIG. 16B, when at least the upper body or whole body of the pedestrian is extracted by the binarization process, a luminance difference from the background area (image) is generated. On the other hand, as shown in FIG. 16C, in the case of an object such as a wall where the temperature difference of the entire object is small, the luminance difference between the binarized extraction part and the part that is not so is small, and the object is AREA3. As shown in FIG. For this reason, the luminance variance Var_A3 of AREA3 shows a high value for a pedestrian and a low value for an object such as a wall.
Accordingly, in step S47, it is determined whether or not the object is a pedestrian by determining whether or not the luminance variance Var_A3 of the mask area AREA3 is greater than the threshold value TH6.
[0052]
Furthermore, in step S47, when the luminance variance Var_A3 of the mask area AREA3 is larger than the threshold value TH6 (YES in step S47), pedestrian determination is performed based on temporal changes in the object shape.
More specifically, since the binarized object of the pedestrian is targeted, it is considered that the binarized object shape does not change significantly with time. For this reason, it is determined whether or not the difference between the maximum value Max_Rate and the minimum value Min_Rate of the rate that is the area ratio of the circumscribed rectangle within the specified time and the area ratio of the binarized object is less than the threshold value TH7 (step S48).
[0053]
In step S46-1, if the weather around the host vehicle 10 is rainy (YES in step S46-1), the process proceeds to step S48 without determining the luminance variance Var_A3 of the mask area AREA3 in step S47. Then, it is determined whether or not the difference between the maximum value Max_Rate and the minimum value Min_Rate of the rate that is the area ratio of the circumscribed rectangle within the specified time and the area ratio of the binarized object is less than the threshold value TH7 (step S48).
[0054]
On the other hand, in step S44, when the width ΔWb of the binarized object is less than the threshold value TH1 or larger than TH2 (NO in step S44), or in step S45, the height ΔHb of the binarized object is equal to the threshold value TH3. Or when the gray scale object height ΔHg is greater than or equal to the threshold TH4 (NO in step S45), or in step S45-1, the binarized object height ΔHb is greater than or equal to the threshold TH3. In the case (NO in step S45-1) or in step S46, if the upper end height position Yt of the object from the road surface is greater than or equal to the threshold value TH5 (NO in step S46), detection is performed. The determined object is determined not to be a pedestrian (step S49), the pedestrian determination process is terminated, and step S34 shown in FIG. O as the process proceeds to step S37 in FIG. 5, the object is determined not to be a warning target.
[0055]
Similarly, in step S47, when the luminance variance of the mask area AREA3 is equal to or less than the threshold TH6 (NO in step S47), in step S48, the area of the circumscribed rectangle and the area of the binarized object within the specified time. If the difference (Max_Rate−Min_Rate) between the maximum value Max_Rate and the minimum value Min_Rate that is the ratio is greater than or equal to the threshold TH7 (NO in step S48), the detected object is a pedestrian. (Step S49), the pedestrian determination process is terminated, the process proceeds to step S37 in FIG. 5 as NO in step S34 shown in FIG. 5, and the object is determined not to be an alarm target.
[0056]
In step S48, when the difference between the maximum value Max_Rate and the minimum value Min_Rate of the rate that is the area ratio of the circumscribed rectangle within the specified time and the area of the binarized object is less than the threshold value TH7 (YES in step S48). Next, the image processing unit 1 performs pedestrian determination for each shape of the object extracted in more detail.
Specifically, first, it is determined whether or not the upper end height position Yt of the object from the road surface is greater than a threshold TH8 (a value that is suitable as a height that can distinguish the pedestrian's upper body and lower body) (step S50). .
In step S50, when the upper end height position Yt of the object from the road surface is equal to or less than the threshold TH8 (NO in step S50), the process proceeds to step S51 in FIG. As a result, it is determined whether or not the width ΔWb of the binarized object is equal to or smaller than a threshold value TH9 (an appropriate value as the trunk width of the pedestrian) (step S51).
[0057]
FIG. 9 shows a processing procedure for identifying a pedestrian whose lower body has been extracted by the binarization process, or in which the width ΔWb of the binarization object is equal to or less than the threshold value TH9 in step S51. If there is (YES in step S51), it is determined whether or not the weather around the host vehicle 10 is rainy (step S51-1).
And in (step S51-1), when the weather around the own vehicle 10 is not rainy (NO in step S51-1), in order to determine whether the object is a sitting pedestrian, It is determined whether or not the height ΔHg of the gray scale object is less than a threshold value TH10 (a suitable value for the pedestrian height) (step S52).
[0058]
In step S52, when the height ΔHg of the gray scale object is equal to or greater than the threshold TH10 (NO in step S52), it is assumed that the object corresponds to a pedestrian's torso or lower body, and the head is present at the top. In order to determine whether or not to do so, it is determined whether or not the luminance average value Ave_A1 of the upper mask area AREA1 shown in FIG. 15 is larger than the threshold value TH11 (step S53).
In step S53, if the average luminance value Ave_A1 of the mask area AREA1 is larger than the threshold value TH11 (YES in step S53), the body part may be difficult to dissipate heat due to the influence of clothing, so the luminance on the grayscale image As an object having a pattern, it is determined whether or not the luminance variance Var_A2 of the mask area AREA2 is larger than the threshold value TH18 (step S53-1).
[0059]
In step S53-1, when the luminance variance Var_A2 of the mask area AREA2 is larger than the threshold value TH18 (YES in step S53-1), it is determined that the detected object is a pedestrian (step S54). The person determination process is terminated, the process proceeds to step S35 in FIG. 5 as YES in step S34 shown in FIG. 5, and the artificial structure determination is performed.
Further, in (Step S51-1), when the weather around the host vehicle 10 is rainy (YES in Step S51-1), the determination of the mask area from Step S52 to Step S53-1 is not performed, and Step S54 is performed. And the detected object is determined to be a pedestrian (step S54), the pedestrian determination process is terminated, the process proceeds to step S35 in FIG. 5 as YES in step S34 shown in FIG. I do.
[0060]
On the other hand, when the width ΔWb of the binarized object is larger than the threshold value TH9 in step S51 (NO in step S51), or in step S53, the luminance average value Ave_A1 of the mask area AREA1 is equal to or smaller than the threshold value TH11 ( If the luminance variance Var_A2 of the mask area AREA2 is equal to or less than the threshold TH18 (NO in step S53-1) in step S53-1 and further in step S53-1, the detected object is detected. Is determined not to be a pedestrian (step S55), the pedestrian determination process is terminated, the process proceeds to step S37 in FIG. 5 as NO in step S34 shown in FIG. 5, and the target is determined not to be an alarm target.
[0061]
In step S52, when the height ΔHg of the gray scale object is less than the threshold value TH10 (YES in step S52), the object is regarded as a sitting pedestrian, and the binarized object from the road surface. It is determined whether or not the upper end height position Yt of the target object is larger than a threshold value TH12 (a value that is suitable as a height that can distinguish a pedestrian sitting from a standing pedestrian) (step S56).
In step S56, when the upper end height position Yt of the object from the road surface of the binarized object is larger than the threshold value TH12 (YES in step S56), the aspect ratio ASPECT of the circumscribed rectangle of the binarized object is expressed. It is determined whether Asp is a threshold value TH13 or more and TH14 or less (a suitable value for a pedestrian) (step S57).
[0062]
In step S57, when the Asp representing the aspect ratio ASPECT of the circumscribed square of the binarized object is not less than the threshold TH13 and not more than TH14 (YES in step S57), the circumscribed square centroids 102 and 2 represented by the equation (9) are used. It is determined whether or not the distance Dis_c in real space with the center of gravity G100 of the object to be digitized is less than a threshold value TH15 (a value appropriate for a pedestrian) (step S58).
Dis_c = SQRT ((Xb−Xc) ² + (Yb-Yc) ² (9)
In step S58, when the distance Dis_c is less than the threshold value TH15 (YES in step S58), for example, an object other than a pedestrian is included in an object having ΔWb of 1.0 m or less and ΔHg of less than 1.0 m. Therefore, it is determined whether or not there is a portion having a high degree of correlation with the head pattern registered in advance in the upper mask area AREA1 of the binarized object (step S59).
[0063]
In step S59, when there is a part having a high degree of correlation with the head pattern registered in advance in the upper mask area AREA1 of the binarized object (YES in step S59), it is determined that the detected object is a pedestrian. Then, the pedestrian determination process is terminated (YES in step S34 shown in FIG. 5), the process proceeds to step S35 in FIG. 5, and the artificial structure determination is performed.
[0064]
On the other hand, when the upper end height position Yt of the object from the road surface of the binarized object is equal to or less than the threshold value TH12 in step S56 (NO in step S56), or the circumscribing of the binarized object in step S57. When Asp representing the rectangular aspect ratio ASPECT is less than threshold TH13 or greater than TH14 (NO in step S57), or in step S58, when distance Dis_c is greater than or equal to threshold TH15 (NO in step S58), In step S59, if there is no part having a high degree of correlation with the head pattern registered in advance in the upper mask area AREA1 of the binarized object (NO in step S59), the detected object It is determined that the object is not a pedestrian (step S55), and the pedestrian determination process is terminated. Proceeds to step S37 in FIG. 5 as NO in step S34 shown in, the object is determined not to be a warning target.
[0065]
In addition, in step S50 of FIG. 9, when the upper end height position Yt of the object from the road surface is larger than the threshold value TH8 (a suitable value as a height that can distinguish the pedestrian's upper body and lower body) (YES in step S50). The process proceeds to step S50-1 in FIG. 10 to determine whether or not the weather around the host vehicle 10 is rainy (step S50-1).
And in (step S50-1), when the weather around the own vehicle 10 is not rainy (NO in step S50-1), an object in which the object is floating in the air (for example, an object such as a curve mirror) In order to determine whether or not the object is an object), it is determined whether or not the height ΔHg of the gray scale object is greater than a threshold value TH16 (the same value as the above-described threshold value TH8) (step S60).
[0066]
FIG. 10 shows a processing procedure for identifying a pedestrian whose head and upper body have been extracted by the binarization process. In step S60, the gray scale object height ΔHg is greater than the threshold value TH16. In this case (YES in step S60), since the object is not an object floating in the air, it is next determined whether the head part is present at the upper end part of the object area (AREA0) or the body part is present. To do. Specifically, first, since the head is exposed, it is determined whether or not the average luminance value Ave_A1 of the mask area AREA1 is larger than the threshold value TH17 (step S61).
[0067]
In step S61, if the average luminance value Ave_A1 of the mask area AREA1 is larger than the threshold value TH17 (YES in step S61), the body part may not easily dissipate heat due to the influence of clothing, so the luminance pattern on the grayscale image. As a certain object, it is determined whether or not the luminance variance Var_A2 of the mask area AREA2 is larger than the threshold value TH18 (step S62).
In step S62, when the luminance variance Var_A2 of the mask area AREA2 is larger than the threshold value TH18 (YES in step S62), first, in order to determine a pedestrian whose head or upper body is extracted by the binarization process, It is determined whether or not the width ΔWb of the binarized object is equal to or less than a threshold value TH19 (a suitable value as a width that can distinguish the pedestrian's head or upper body) (step S63).
[0068]
In (Step S50-1), when the weather around the host vehicle 10 is rainy (YES in Step S50-1), the mask area is not determined in Step S60 to Step S62, and the process proceeds to Step S63. In order to determine a pedestrian whose head or upper body is extracted by the binarization process, the width ΔWb of the binarized object is an appropriate threshold TH19 (a width that can distinguish the pedestrian's head or upper body) Value) or less (step S63).
[0069]
Next, when the width ΔWb of the binarized object is larger than the threshold value TH19 in step S63 (NO in step S63), at least the upper body of the pedestrian or the pedestrian whose whole body is extracted by the binarization process is determined. Therefore, it is determined whether or not the width ΔWb of the binarized object is equal to or less than a threshold value TH9 (an appropriate value for the pedestrian's trunk width) (step S64).
Furthermore, in step S64, when the width ΔWb of the binarized object is larger than the threshold value TH9 (NO in step S64), binarization is performed in order to determine whether a plurality of pedestrians are performing parallel walking. It is determined whether or not the width ΔWb of the object is equal to or less than a threshold value TH2 (a value appropriate for the pedestrian's trunk width) (step S65).
[0070]
In the above determination, when the gray scale object height ΔHg is equal to or smaller than the threshold TH16 in step S60 (NO in step S60), or in step S61, the luminance average value Ave_A1 of the mask area AREA1 is the threshold TH17. If it is below (NO in step S61), or if the luminance variance Var_A2 of the mask area AREA2 is less than or equal to the threshold TH18 (NO in step S62) in step S62, and further in step S65, the binarized object If the width ΔWb is greater than the threshold value TH2 (NO in step S65), the detected object is determined not to be a pedestrian (step S66), and the pedestrian determination process is terminated. Step S37 in FIG. 5 is determined as NO in Step S34 shown in FIG. Proceed to, and determine that the object is not an alarm target.
[0071]
On the other hand, when the width ΔWb of the binarized object is equal to or smaller than the threshold TH19 in step S63 (YES in step S63), the object is a pedestrian whose head or upper body is extracted by the binarization process. Then, the process proceeds to step S67 in FIG. 11 to determine whether Asp representing the aspect ratio ASPECT of the circumscribed square of the binarized object is a threshold value TH20 or more and TH21 or less (appropriate value for a pedestrian's head or upper body). (Step S67).
[0072]
FIG. 11 shows a processing procedure for identifying a pedestrian whose head and upper body have been extracted by binarization processing. In step S67, the aspect ratio ASPECT of the circumscribed rectangle of the binarized object is shown. When Asp is greater than or equal to the threshold TH20 and less than TH21 (YES in step S67), it is determined whether or not the distance Dis_c in the real space between the circumscribed square centroid 102 and the centroid G100 of the binarized object is less than the threshold TH15. (Step S68).
If the distance Dis_c is less than the threshold TH15 in step S68 (YES in step S68), it is determined that the detected object is a pedestrian (step S69), and the pedestrian determination process is terminated, and FIG. As YES in step S34 shown, the process proceeds to step S35 in FIG. 5 to perform artificial structure determination.
[0073]
On the other hand, if Asp representing the aspect ratio ASPECT of the circumscribed rectangle of the binarized object is less than the threshold TH20 or greater than TH21 (NO in step S67) or in step S68, the distance Dis_c is the threshold TH15. When it is above (NO of step S68), it determines with the detected target object not being a pedestrian (step S70), a pedestrian determination process is complete | finished, and NO of step S34 shown in FIG. Proceeding to step S37, it is determined that the object is not an alarm target.
[0074]
If the binarized object width ΔWb is equal to or smaller than the threshold value TH9 in step S64 of FIG. 10 (YES in step S64), at least the upper body of the pedestrian or the whole body is extracted by the binarization process. Assuming that the user is a pedestrian, the process proceeds to step S71 in FIG. Is determined (step S71).
[0075]
FIG. 12 shows a processing procedure for identifying a pedestrian whose upper body or whole body has been extracted by the binarization process. In step S71, Asp representing the aspect ratio ASPECT of the circumscribed rectangle of the binarized object. Is not less than the threshold TH13 and not more than TH21 (YES in Step S71), it is determined whether or not the distance Dis_c in the real space between the circumscribed square centroid 102 and the centroid G100 of the binarized object is less than the threshold TH15. (Step S72).
[0076]
If the distance Dis_c is less than the threshold TH15 in step S72 (YES in step S72), it is determined whether or not the weather around the host vehicle 10 is rainy (step S72-1).
And in (step S72-1), when the weather around the own vehicle 10 is not rainy (NO in step S72-1), the object is an object other than a pedestrian, for example, the front part of the vehicle. In the upper mask area AREA1 of the binarized object, it is determined whether or not there is a part having a high degree of correlation with the head pattern registered in advance (step S73).
[0077]
In step S73, when a part having a high degree of correlation with the head pattern registered in advance in the upper mask area AREA1 of the binarized object exists (YES in step S73), the detected object is determined to be a pedestrian. Then, the pedestrian determination process is terminated (YES in step S34 shown in FIG. 5) and the process proceeds to step S35 in FIG. 5 to perform artificial structure determination.
In (Step S72-1), if the weather around the host vehicle 10 is rainy (YES in Step S72-1), the determination of the mask area in Step S73 is not performed, but the process proceeds to Step S74 and is detected. The target object is determined to be a pedestrian (step S74), the pedestrian determination process is terminated, the process proceeds to step S35 in FIG. 5 as YES in step S34 shown in FIG. 5, and artificial structure determination is performed.
[0078]
On the other hand, if Asp representing the aspect ratio ASPECT of the circumscribed square of the binarized object is less than the threshold TH13 or greater than TH21 (NO in step S71), or the distance Dis_c is the threshold in step S72. If TH15 or more (NO in step S72), and further in step S73, there is no portion having a high degree of correlation with the head pattern registered in advance in the upper mask area AREA1 of the binarized object (in step S73). NO), it is determined that the detected object is not a pedestrian (step S75), the pedestrian determination process is terminated, and the step of FIG. 5 is performed as NO of step S34 shown in FIG. Proceeding to S37, it is determined that the object is not an alarm target.
[0079]
In addition, when the width ΔWb of the binarized object is equal to or less than the threshold value TH2 in step S65 in FIG. 10 (YES in step S65), the object is the object because a plurality of pedestrians are walking in parallel. It is determined that a large number of background regions are included in the circumscribed rectangle, and the process proceeds to step S76 in FIG. 13, where Rate, which is the ratio of the area of the circumscribed rectangle and the area of the binarized object within the specified time, is less than the threshold value TH22. Is determined (step S76).
[0080]
FIG. 13 shows a processing procedure when the object is a plurality of pedestrians walking in parallel. In step S76, the area of the circumscribed rectangle within the specified time and the area ratio of the binarized object are shown. When a certain rate is less than the threshold value TH22 (YES in step S76), Asp representing the aspect ratio ASPECT of the circumscribed rectangle of the binarized object is greater than or equal to the threshold value TH23 and not more than TH14 (in order to determine pedestrian parallel walking It is determined whether it is an appropriate value (step S77).
[0081]
In step S77, when Asp representing the aspect ratio ASPECT of the circumscribed rectangle of the binarized object is not less than the threshold value TH23 and not more than TH14 (YES in step S77), the circumscribed rectangle centroid 102 and the centroid of the binarized object are described above. It is determined whether or not the distance Dis_c in the real space with G100 is less than the threshold value TH15 (step S78).
If the distance Dis_c is less than the threshold TH15 in step S78 (YES in step S78), it is determined that the detected object is a pedestrian (step S79), and the pedestrian determination process is terminated, and FIG. As YES in step S34 shown, the process proceeds to step S35 in FIG. 5 to perform artificial structure determination.
[0082]
On the other hand, in step S76, when the rate that is the ratio of the area of the circumscribed rectangle within the specified time and the area ratio of the binarized object is equal to or greater than the threshold TH22 (NO in step S76), or in binarization target in step S77. When Asp representing the aspect ratio ASPECT of the circumscribed rectangle of the object is less than the threshold value TH23 or greater than TH14 (NO in step S77), and in step S78, the distance Dis_c is greater than or equal to the threshold value TH15 (in step S78). NO), it is determined that the detected object is not a pedestrian (step S80), the pedestrian determination process is terminated, and step S34 shown in FIG. Proceeding to S37, it is determined that the object is not an alarm target.
[0083]
In the present embodiment, the image processing unit 1 includes weather detection means, object extraction means, heat storage body extraction means, pedestrian recognition means, and shape determination means. More specifically, S41-1 in FIG. 8 corresponds to the weather detection means, S1 to S13 in FIG. 3 correspond to the object extraction means, S45, S46, S47 in FIG. 8, S52, S53 in FIG. , S53-1, and S60, S61, and S62 in FIG. 10 correspond to the heat storage element extraction means.
[0084]
Further, S48 and S49 in FIG. 8, S50 and S54 to S59 in FIG. 9, S63 to S66 in FIG. 10, S67 to S70 in FIG. 11, S71 to S75 in FIG. 12, and S76 to S80 in FIG. It corresponds to. In particular, S59 in FIG. 9 and S73 in FIG. 12 correspond to the shape determining means.
[0085]
As described above, the vehicle periphery monitoring device according to the present embodiment extracts an object such as a pedestrian from a grayscale image captured by an infrared camera by binarization processing, and then surrounds the vehicle 10 When the weather is not rainy, the grayscale object in a range including the binarized object is extracted from the grayscale image due to the luminance change of the grayscale image, and a plurality of grayscale objects are further included in the region of the grayscale object. A search area is set, and a pedestrian in the search area is recognized based on the shape of the search area and the luminance dispersion of the search area.
In addition, when the weather around the host vehicle 10 is rainy, only the existence condition of the binarized object is determined, and the binarized image is determined based on the height and size of the binarized object in the image. Recognize pedestrians.
[0086]
Thereby, when the weather around the host vehicle 10 is not rainy, for example, when the width of the image of the object is unnatural as a pedestrian, or when the height of the image of the object is unnatural as a pedestrian In addition to removing these objects from the image of the object, the pedestrian that satisfies these features has a portion with high luminance dispersion corresponding to the head, or a portion with high luminance dispersion corresponding to the torso. Furthermore, it is determined whether the luminance dispersion of the wall or the like is low, and an object image having a luminance dispersion different from the image obtained by photographing the pedestrian is removed from the image of the object, thereby improving the detection accuracy of the pedestrian. The effect that it can be improved is obtained.
[0087]
In addition, when the weather around the host vehicle 10 is rainy, the amount of infrared rays emitted from the object decreases, so determination by luminance dispersion is not performed. For example, the image width of the object is unnatural as a pedestrian. In cases where the height of the image of the object is unnatural as a pedestrian, it is only necessary to remove these objects from the image of the object and prevent erroneous detection of pedestrians by determination using luminance dispersion And the effect that the detection accuracy of a pedestrian can be maintained is acquired.
[0088]
【The invention's effect】
As described above, according to the vehicle periphery monitoring device of the first aspect, when the weather detection unit determines that the weather around the vehicle is rainy, the pedestrian recognition unit directly selects the extracted object. Recognize pedestrians among objects. On the other hand, if it is determined that the weather around the vehicle is other than that, the heat storage body extraction means extracts the heat storage body from the object, and the pedestrian recognition means walks from the object excluding the heat storage body. By recognizing the person, appropriate pedestrian recognition can be performed according to the situation around the vehicle.
Therefore, when the weather around the vehicle is not rainy, for example, a wall that emits heat (infrared rays) equivalent to that of a pedestrian, such as a wall that receives sunlight from the sun. Based on the above, it is possible to remove the object from the object and improve the detection accuracy of the pedestrian. In addition, when the weather around the vehicle is rainy, judgment based on luminance dispersion is not performed, and conversely, erroneous detection of pedestrians by judgment using luminance dispersion is prevented, and pedestrian detection accuracy is improved. The effect that it can maintain is acquired.
[0089]
According to the vehicle periphery monitoring apparatus according to claim 2, when the weather around the vehicle is determined to be rainy by the weather detection unit, the shape determination process is stopped, thereby the object shape on the image Can be prevented from being erroneously determined by the pedestrian recognition means.
Therefore, when the weather around the vehicle is rainy, it is possible to prevent erroneous detection of the pedestrian by determining the shape of the object and maintain the detection accuracy of the pedestrian.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a vehicle periphery monitoring device according to an embodiment of the present invention.
FIG. 2 is a diagram showing attachment positions of an infrared camera, a sensor, a display, and the like in a vehicle.
FIG. 3 is a flowchart showing an object detection / alarm operation of the vehicle periphery monitoring device according to the embodiment;
FIG. 4 is a diagram illustrating a grayscale image obtained by an infrared camera and a binarized image thereof.
FIG. 5 is a flowchart showing an alarm determination processing operation according to the embodiment;
FIG. 6 is a diagram illustrating a case where a collision is likely to occur.
FIG. 7 is a diagram showing a region division in front of the vehicle.
FIG. 8 is a flowchart showing a pedestrian determination processing operation according to the embodiment.
FIG. 9 is a flowchart showing a pedestrian determination processing operation according to the embodiment;
FIG. 10 is a flowchart showing a pedestrian determination processing operation according to the embodiment.
FIG. 11 is a flowchart showing a pedestrian determination processing operation according to the embodiment;
FIG. 12 is a flowchart showing a pedestrian determination processing operation according to the embodiment.
FIG. 13 is a flowchart showing a pedestrian determination processing operation according to the embodiment;
FIG. 14 is a diagram showing a binarized object shape feature amount according to the embodiment;
FIG. 15 is a diagram showing mask area setting according to the embodiment;
FIG. 16 is a diagram showing luminance dispersion of the mask area AREA3 when the object is a part or the whole of a pedestrian or a wall.
FIG. 17 is a diagram illustrating a grayscale image in fine weather and a grayscale image in rainy weather in the vehicle periphery monitoring device of the embodiment.
FIG. 18 is a graph comparing a luminance histogram of a grayscale image in rainy weather and a luminance histogram of a grayscale image in fine weather in the vehicle periphery monitoring device of the embodiment.
[Explanation of symbols]
1 Image processing unit
2R, 2L infrared camera
3 Yaw rate sensor
4 Vehicle speed sensor
5 Brake sensor
6 Speaker
7 Image display device
10 Own vehicle
S1 to S13 Object extraction means
S41-1 Weather detection means
S45, S46, S47, S52, S53, S53-1, S60, S61, S62 Heat storage element extraction means
S48-S50, S54-S59, S63-S80 Pedestrian recognition means
S59, S73 Shape determination means

Claims (2)

A vehicle periphery monitoring device for detecting an object existing around a vehicle from infrared images taken by two infrared cameras,
Weather detection means for detecting weather around the vehicle;
An object extraction means for extracting an object emitting infrared rays from the infrared image;
From the object extracted by the object extraction means, the heat storage body extraction means for extracting the heat storage body that stores only the heat given from the outside without itself generating heat,
If the weather around the vehicle is determined to be rainy by the weather detection means, a pedestrian is recognized from the object, otherwise it is extracted by the heat storage body extraction means. A vehicle periphery monitoring device comprising: pedestrian recognition means for recognizing a pedestrian among the objects excluding the heat storage body. The pedestrian recognition means
Including shape determining means for determining the shape of the object,
The vehicle periphery monitoring device according to claim 1, wherein when the weather around the vehicle is determined to be rainy by the weather detection means, the shape determination means is stopped.

Images