JP2007328630A - Object candidate region detector, object candidate region detection method, pedestrian recognition system, and vehicle control device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect a region where an object may exist with high accuracy from an image of an on-vehicle camera. <P>SOLUTION: A reference pattern database 15 stores a plurality of reference patterns (for sky, road, other than road, or the like) depending on the background of the image. A background region dividing section 12 divides the background of the input image using a vanishing point or lane recognition. A reference pattern selection section 14 selects a reference pattern used by a pedestrian candidate region detecting section 19 depending on the background region from which a determination region is cut. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an object candidate area detection device, an object candidate area detection method, and an object candidate area detection for detecting an area where a specific object may exist as an object candidate area from an input image taken by an in-vehicle camera. The present invention relates to a pedestrian recognition device and a vehicle control device used.

  Conventionally, a technique for detecting a pedestrian or the like by image recognition on an image captured by a camera has been devised. For example, Patent Document 1 discloses a technique for recognizing a pedestrian or a vehicle from an image taken by a fixed point camera fixed at a specific location.

  In addition, as a specific method of such image recognition, for example, a saliency calculation method (saliency algorithm) in which a background pattern is learned in advance and a difference between the input image and the background pattern is calculated as saliency can be applied. It is.

Japanese Patent Laid-Open No. 2002-190012 JP 2001-236508 A JP 2004-178243 A

  By the way, in recent years, it has become an important issue to implement a technology that supports a driving by mounting a camera or the like on a vehicle and recognizing a pedestrian or the like by an image from the in-vehicle camera and preventing an accident.

  However, in an image taken from a moving body such as an in-vehicle camera, the background image always changes due to the movement. Therefore, even if the above-described saliency calculation method is used, there is a problem that an event that needs to be learned for setting a background pattern becomes large and detection accuracy is lowered.

  In addition, there is a technique for dividing an input image into a plurality of regions and comparing them as in Patent Document 3, but in the case of in-vehicle use, the background changes variously in each region even if it is simply divided. Therefore, it cannot be detected efficiently.

  The present invention has been made to solve the above-described problems in the prior art and to solve the problems, and is an object candidate area for detecting an area where an object may exist with high accuracy from an image of a vehicle-mounted camera. It is an object of the present invention to provide a detection device, an object candidate region detection method, a pedestrian recognition device that recognizes a pedestrian with high accuracy, and a vehicle control device that uses pedestrian recognition.

  In order to solve the above-described problems and achieve the object, the object candidate area detection device according to the invention of claim 1 uses, as an object candidate area, an area in which a specific object may exist from an input image captured by a camera. An object candidate area detection device for detecting, wherein a reference pattern storage means for storing a plurality of reference patterns different for each of a road area and an area outside the road with respect to a background of the input image, and an input image currently taken by a camera A background area dividing unit that divides the background into a road area and an area outside the road, a determination area extraction unit that extracts a determination area from an input image currently captured by the camera, and the determination area is a road area A reference pattern for selecting a reference pattern corresponding to the background area from among the plurality of reference patterns depending on which background area of the area outside the road is cut out A down selection unit, by comparing the said selected reference pattern and the determination target region, out of the determination target region, characterized by comprising a detecting means for detecting the object candidate region.

  According to the first aspect of the present invention, the object candidate area detection device stores a plurality of reference patterns different for each of the road area and the outside road area for the background of the input image, and the road area for the input image for the background. A reference pattern is selected based on which background area the divided area is cut out for each area outside the road, and an object candidate area is detected by comparing the determined area with the selected reference pattern. Inside the road and outside the road, there is a characteristic especially in how the background changes (if you are inside the road, pedestrians, cars, crosswalks, etc. often appear). Therefore, it is possible to detect efficiently by dividing by these and narrowing down the reference pattern.

  According to a second aspect of the present invention, there is provided the object candidate area detecting device according to the first aspect of the invention, wherein the background area dividing means divides the upper part from the vanishing point of the input image as an area outside the road. And

  According to the second aspect of the present invention, the object candidate area detecting device divides the upper side from the vanishing point of the input image as the outside road area, and when the determination target area is cut out from the outside road area, the reference pattern for road outside is used. The object candidate area is detected by comparison with.

  According to a third aspect of the present invention, there is provided the object candidate region detection apparatus according to the first or second aspect, wherein the background region dividing means uses a lane recognition result for the input image and uses a road lane recognition result and a road outside region. It is characterized by dividing the area.

  According to the third aspect of the present invention, the object candidate area detecting device divides the road area and the road area using the lane recognition result, and uses the reference area depending on whether the determined area is cut out from inside or outside the road. Select a pattern.

  An object candidate area detection device according to a fourth aspect of the present invention is an object candidate area detection device that detects an area where a specific object may exist as an object candidate area from an input image captured by a camera, With respect to the background of the input image, a reference pattern storage means for storing a plurality of different reference patterns for each road area and outside the road, and a determination area extraction means for cutting out the determination area from the input image captured by the current camera And a detection means for detecting the object candidate area in the determination area by comparing the determination area and each of the plurality of reference patterns and combining the plurality of comparison results. It is characterized by that.

  According to the fourth aspect of the present invention, the object candidate area detecting device stores a plurality of reference patterns different for each of the road area and the area outside the road with respect to the background of the input image, An object candidate region is detected by combining a plurality of comparison results obtained by comparing each of the plurality of reference patterns.

  According to a fifth aspect of the present invention, there is provided the object candidate area detecting device according to the fourth aspect, wherein the detecting means digitizes a difference between the determination target area and the plurality of reference patterns. When the average value of a plurality of numerical values is equal to or greater than a threshold value, the determination target region is output as the object candidate region.

  According to the fifth aspect of the present invention, the object candidate region detection device digitizes the difference between the determination target region and the plurality of reference patterns, and determines the determination target when the average value of the plurality of obtained numerical values is equal to or greater than a threshold value. The area is output as an object candidate area.

  The object candidate area detection method according to the invention of claim 6 is an object candidate area detection method for detecting an area where a specific object may exist as an object candidate area from an input image captured by a camera, A reference pattern storing step for storing a plurality of reference patterns different for each of a road area and an area outside the road for the background of the input image, and a comparison between the determination target area cut out from the input image and the plurality of reference patterns And a detection step of detecting the object candidate region.

  According to the sixth aspect of the present invention, the object candidate region detection method stores a plurality of reference patterns different for each of a road region and a region outside the road with respect to the background of the input image, An object candidate region is detected by comparison with a plurality of reference patterns.

  The pedestrian recognition device according to the invention of claim 7 is a pedestrian recognition device for recognizing a pedestrian from an input image taken by an in-vehicle camera, and the background of the input image A pedestrian is an area where a pedestrian may exist by comparing a reference pattern storage unit that stores a plurality of different reference patterns for each area, a determination target area cut out from the input image, and the plurality of reference patterns. It is characterized by comprising detection means for detecting as a candidate area, and pedestrian discrimination means for discriminating whether or not the pedestrian candidate area corresponds to a pedestrian.

  According to the seventh aspect of the present invention, the pedestrian recognition device detects a pedestrian candidate area where a pedestrian may exist by comparing the determination target area and a plurality of reference patterns that differ for each background, and the pedestrian It is determined whether or not the pedestrian candidate area is a pedestrian by pattern matching or neuro-determination for the candidate area.

  In addition, the vehicle control device according to the invention of claim 8 includes: a reference pattern storage unit that learns and stores in advance a plurality of reference patterns that are different for each of a road area and an area outside the road with respect to a background of an image; Detection means for detecting a region where a pedestrian may exist as a pedestrian candidate region by comparing the determined region cut out from the captured input image and the plurality of reference patterns, and the pedestrian candidate region walking A pedestrian discriminating unit that discriminates whether or not the vehicle corresponds to a pedestrian, a notification control that informs about the presence of a pedestrian when the pedestrian is recognized by the pedestrian discriminating unit, and / or a control of the running state of the vehicle And a control means for executing the running control.

  According to the invention of claim 8, the vehicle control device detects a pedestrian candidate region where a pedestrian may exist by comparing the determination target region and a plurality of reference patterns different for each background, and the pedestrian When the candidate area is a pedestrian, notification control for notifying the driver of the presence of the candidate area and vehicle operation control are executed.

  According to the first aspect of the present invention, the object candidate area detection device stores a plurality of reference patterns different for each of a road area and an area outside the road with respect to the background of the input image, The reference pattern is selected according to which background area the segmented area is cut out from each area outside the road, and the object candidate area is detected by comparing the segmented area with the selected reference pattern. It is possible to obtain an object candidate area detection device that detects an area where an object may exist with high accuracy from the image of.

  According to the second aspect of the present invention, the object candidate area detecting device divides the upper part from the vanishing point of the input image as an outside road area, and when the determined area is cut out from the outside road area, the object outside area reference Since the object candidate area is detected by comparison with the pattern, an object candidate area detection device that easily divides the area corresponding to the outside of the road and detects the area where the object may exist with high accuracy from the camera image is provided. There is an effect that it can be obtained.

  According to the invention of claim 3, the object candidate area detection device divides the road area and the road area using the lane recognition result, and uses the determined area depending on whether the determined area is cut out from inside or outside the road. Since the reference pattern is selected, it is possible to obtain an object candidate area detection device that detects an area where an object may exist with high accuracy from a camera image by recognition inside and outside the road.

  According to a fourth aspect of the present invention, the object candidate region detection device stores a plurality of reference patterns that differ for each of a road region and a region outside the road with respect to the background of the input image, and is a determined region cut out from the input image The object candidate area is detected by combining the multiple comparison results obtained by comparing each of the reference pattern and each of the plurality of reference patterns, so the area where the object may exist is detected with high accuracy from the camera image. There is an effect that an object candidate region detection device can be obtained.

  According to the invention of claim 5, the object candidate area detecting device digitizes the difference between the determination target area and the plurality of reference patterns, and the target area is detected when an average value of the obtained plurality of numerical values is equal to or greater than a threshold value. Since the determination area is output as an object candidate area, there is an effect that an object candidate area detection device that detects an area where an object may exist with high accuracy can be obtained from a camera image.

  According to a sixth aspect of the present invention, the object candidate region detection method stores a plurality of reference patterns that differ for each of a road region and a region outside the road with respect to the background of the input image, and is a determined region cut out from the input image Since the object candidate area is detected by comparing with a plurality of reference patterns, it is possible to obtain an object candidate area detection method for detecting an area where an object may exist with high accuracy from a camera image. .

  According to the invention of claim 7, the pedestrian recognition device detects a pedestrian candidate area where a pedestrian may exist by comparing the determination target area and a plurality of reference patterns different for each background, and walks. Since it is determined whether or not the pedestrian candidate area is a pedestrian by pattern matching or neuro determination for the pedestrian candidate area, it is possible to obtain a pedestrian recognition device that recognizes a pedestrian with high accuracy from an on-vehicle camera image. There is an effect.

  According to the invention of claim 8, the vehicle control device detects a pedestrian candidate region where a pedestrian may exist by comparing the determination target region and a plurality of reference patterns different for each background, and the walking When the candidate area is a pedestrian, the vehicle control device that recognizes the pedestrian with high accuracy and performs safety control is performed because the notification control for notifying the driver of the existence and the operation control of the vehicle are executed. There is an effect that it can be obtained.

  Exemplary embodiments of an object candidate region detection device, an object candidate region detection method, a pedestrian recognition device, and a vehicle control device according to the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is a schematic configuration diagram showing a schematic configuration of an in-vehicle image recognition apparatus 10 that is Embodiment 1 of the present invention. The vehicle has the following configuration. As shown in the figure, the image recognition device 10 is connected to a navigation device 30, a camera 31, and a pre-crash ECU 40.

  The navigation device 30 is an in-vehicle device that sets and guides a travel route using the position of the host vehicle specified by communicating with a GPS (Global Positioning System) artificial satellite and map data 30a stored in advance. In addition, the navigation device 30 provides the image recognition device 10 with position information of the own vehicle, surrounding map information, information about roads, and the like.

  The camera 31 captures the vicinity of the host vehicle and inputs the captured result to the image recognition apparatus 10. When the image recognition unit 10 predicts a collision of the host vehicle, the pre-crash ECU 40 is an electronic device that performs notification of occupants by the display 43 and the speaker 44, and the operation control of the vehicle by the brake 41 and the engine control device (EFI) 42. It is a control device and functions as a vehicle control device.

  The image recognition device 10 includes a preprocessing unit 11, a background region dividing unit 12, a reference pattern selection unit 14, a reference pattern database 15, a vehicle recognition unit 16, a white line recognition unit 17, a pedestrian recognition unit 18, and a pedestrian candidate. An area detection unit 19 and a collision determination unit 20 are included. Here, the background region division unit 12, the reference pattern selection unit 14, the reference pattern database 15, the vehicle recognition unit 16, the white line recognition unit 17, the pedestrian recognition unit 18, the pedestrian candidate region detection unit 19, and the collision determination unit 20 For example, it is preferably realized by a single microcomputer 10a (an arithmetic processing unit composed of a combination of CPU, ROM, and RAM).

  The preprocessing unit 11 performs processing such as filtering, edge detection, and contour extraction on the image captured by the camera 31, and then the background region dividing unit 12, the vehicle recognition unit 16, the white line recognition unit 17, and the pedestrian candidate region. Output to the detection unit 19.

  The vehicle recognition unit 16 performs pattern matching on the image output from the preprocessing unit 11 to recognize the vehicle, and outputs the recognition result to the collision determination unit 20. Similarly, the white line recognition unit 18 performs pattern matching on the image output from the preprocessing unit 11 to recognize a white line and outputs the recognition result to the collision determination unit 20.

  On the other hand, the pedestrian candidate area detection unit 19 detects an area where a pedestrian may exist from the image output by the preprocessing unit 11 using a saliency algorithm, and the detection result is sent to the pedestrian recognition unit 18. Output. Thereafter, the pedestrian recognition unit 18 performs pattern matching or neuro-determination processing using a neural network on the pedestrian candidate region to recognize the pedestrian and outputs the pedestrian to the collision determination unit 20. And when a pedestrian is recognized, the distance to a pedestrian is calculated by image processing etc., and the distance is also output to the collision determination part 20 as pedestrian information.

  The collision determination unit 20 predicts a collision between a pedestrian or another vehicle and the host vehicle using the recognition results obtained by the vehicle recognition unit 16, the white line recognition unit 17, the pedestrian recognition unit 18, and the position information output by the navigation device 30. Then, the prediction result is output to the pre-crash ECU 40. On the other hand, the pre-crash ECU 40 controls the driver using the display 43, the speaker 44, etc. if there is a possibility of a collision based on the prediction result (eg, a pedestrian is present ahead and the distance is close to several meters). In addition, vehicle travel control (brake control, accelerator control, etc.) is performed depending on the risk of collision.

  Here, when recognizing a vehicle or a white line with little variation in shape, pattern matching is directly performed on the image output by the pre-processing unit 11. In the case of recognition, the detection accuracy is improved and the processing load is reduced by detecting candidate regions in advance by a saliency algorithm.

  Detection of candidate areas by the saliency algorithm will be described with reference to FIG. As shown in the figure, the saliency algorithm first creates a reference pattern by a learning process, and the pedestrian candidate area detection process calculates saliency by comparing the input image with the reference pattern.

  In the learning process, a plurality of learning images are input in advance before using the apparatus, and a learning area is cut out from the learning image. This learning area is cut out in a size of 20 × 20 pixels, for example, in accordance with the size of the pedestrian in the image. The learning image is a background image of a road or the like that does not include a target such as a pedestrian.

  This cut-out learning area is treated as an N-dimensional (N is the number of pixels, and if it is 20 × 20 pixels, 400 dimensions) vector, N-dimensional principal component analysis is performed on all learning areas, and a feature vector (N The principal component of the dimension vector) is calculated. The calculation of the feature vector is executed for all learning images, and the common vector component is used as a reference pattern.

  In the pedestrian candidate area detection process, a determination area having the same size as the learning area is cut out from the input image captured by the camera. Then, the cut-out determination area is set as an N-dimensional vector and developed with a reference pattern (takes an inner product).

  By this expansion, a vector component similar to the reference pattern is obtained in the judgment area, so that the residual between the vector obtained by the development and the vector of the judgment area is significant (a value indicating the degree of deviation from the reference pattern). And outputs a result image in which only a highly significant area is left as a pedestrian candidate area.

  However, depending on the learning image to be input, an appropriate reference pattern cannot be obtained, and the detection accuracy of the candidate area may decrease. For example, when various images are used as learning images, each learning image has a different main component, so that the reference pattern obtained therefrom has many directions, and a residual with a pedestrian is obtained. Becomes difficult. That is, the pedestrian's features are buried in the background, and it is determined that there is no saliency.

  Further, if the threshold is lowered to a point where the saliency is low and if any saliency is detected, it is detected as a pedestrian candidate other than the pedestrian, and conversely, the detection accuracy is lowered. For example, as shown in FIG. 3, when a tree and a building are input as learning images in order to detect objects other than trees and buildings from the input image, a reference pattern in which the characteristics of the trees and the characteristics of the buildings are mixed is created. .

  As a result, it may be determined that the tree pattern or the building image is different from the reference pattern, that is, “conspicuous”.

  In this way, when the events to be input in the learning process are large (the commonality between the images for learning is low and wide-ranging), an appropriate reference pattern cannot be obtained, the saliency detection accuracy is low, and the pedestrian Or a large number of areas are output as “significant pedestrian candidate areas”. As a result, a processing load increase and a recognition accuracy decrease in the pedestrian recognition unit 18 are caused.

  Therefore, the image recognition apparatus 10 improves the detection accuracy of the pedestrian candidate region by using the reference pattern for each background in the input image.

  Specifically, as shown in FIG. 4, first, in the learning process, the learning image is classified into “sky background image”, “background image in the road”, and “background image outside the road” for each background. And, from “sky background image” to “empty reference pattern”, “background image in road” to “reference pattern for road”, “background image outside road” to “reference pattern for road outside”, Create a reference pattern corresponding to the background.

  Next, in the pedestrian candidate region detection process, the background region dividing unit 12 divides the image processed by the preprocessing unit 11 into three background regions, for example, the sky, the road, and the road outside, for each background. Then, the reference pattern selection unit 14 selects a reference pattern to be used by the pedestrian candidate region detection unit 19 depending on from which background region the determination region is cut out.

  In this way, by using different reference patterns for each background region, it is possible to reduce the events input in the learning process for each reference pattern and improve the detection accuracy of candidate regions. That is, in determining the saliency of a determination target region cut out from a predetermined background region (for example, a road), by using a reference pattern (for example, for a road) corresponding to the background region, the characteristics of the pedestrian image are used as the background. It can be prevented from being buried and being judged as “no saliency”. Therefore, it is not necessary to lower the detection threshold, and the pedestrian candidate area can be detected with high accuracy.

  Next, a specific example of background area division by the background area dividing unit 12 will be described. The background area dividing unit 12 includes a vanishing point recognizing unit 12a and a lane recognizing unit 12b.

  The vanishing point recognizing unit 12a performs processing for recognizing the vanishing point in the screen using an optical flow or the like. Then, the background area dividing unit 12 divides the area above the vanishing point as empty as shown in FIG.

  The lane recognition unit 12b performs processing for recognizing a lane in the image using the recognition result of the white line recognition unit 17. Then, the background area dividing unit 12 divides the area inside the lane as the inside of the road as shown in FIG. 5, and sets the outside of the lane as the outside of the road.

  Next, a processing operation for pedestrian recognition by the microcomputer 10a will be described with reference to a flowchart of FIG. This processing flow starts when power is supplied to the microcomputer 10a when the IG is turned on, and is repeatedly executed for each image frame processing (for example, every several milliseconds) when the image processing apparatus 10 is executing image recognition. Process. The microcomputer 10a also performs white line recognition and vehicle recognition, but a description thereof is omitted in this processing flow.

  First, the to-be-determined area extracting unit 19a inside the pedestrian candidate area detecting unit 19 acquires an image that has been processed by the preprocessing unit 11, and extracts a determination area from the acquired image (step S101). Further, the vanishing point recognizing unit 12a recognizes the vanishing point by using the optical flow for the image subjected to the processing by the preprocessing unit 11 (step S102).

  If the determination area is cut out from above the vanishing point (step S103, Yes), the reference pattern selection unit 14 selects an empty reference pattern (step S107).

  On the other hand, if the determination area is cut out from the vanishing point or less (step S103, No), the lane recognition unit 12b recognizes the lane using the recognition result of the white line recognition unit 17 (step S104).

  If the determination area is cut out from the lane, that is, from the road (Yes in step S105), the reference pattern selection unit 14 selects the reference pattern for road (step S108), and the determination area is the lane. If it is cut out from the outside, that is, outside the road (No at Step S105), the reference pattern selection unit 14 selects the reference pattern for outside the road (Step S106).

  After the selection of the reference pattern (steps S106, S107, S108), the pedestrian candidate area selection unit 19 compares the selected reference pattern with the clipped area to be determined (step S109), and is there saliency? It is determined whether or not (step S110).

  As a result, if there is no saliency (No in step S110), the process is terminated as it is, and if there is saliency (step S110, Yes), the pedestrian recognition unit 18 is a pedestrian by pattern matching or a neuro determination process. If it is a pedestrian (step S111, Yes), the pedestrian information and the distance to the pedestrian are output to the collision determination unit 20 (step S112), and the process is terminated. .

  Next, the processing operation of the pre-crash ECU 40 will be described with reference to the flowchart of FIG. This process flow is a process that starts when power is supplied when IG is turned on and the like, and is repeatedly executed.

  First, the pre-crash ECU 40 receives data indicating the result of image recognition from the image recognition device 10 (step S201). If the risk of collision with a pedestrian or the like is high (step S202, Yes), the driver 44 is notified by a warning sound output such as a buzzer by the speaker 44 or a display output by the display 43 (step S203). Or vehicle control by EFI 42 is executed (step S204), and the process is terminated.

  As described above, the image recognition apparatus 10 according to the first embodiment creates a different reference pattern for each background, divides the input image into background regions, and determines from which background region the determined region is cut out. By selecting the reference pattern, the detection accuracy of the pedestrian candidate area by the saliency algorithm is improved.

  In the above-described first embodiment, the configuration in which the reference pattern is selected corresponding to the background area from which the determination target area is cut out has been described. However, in this second embodiment, a plurality of different backgrounds for each determination target area. A configuration for comparing with the reference pattern will be described.

  FIG. 8 is a schematic configuration diagram showing a schematic configuration of an in-vehicle image recognition apparatus 50 that is Embodiment 2 of the present invention. As shown in the figure, an image recognition device 50 is connected to a navigation device 30, a camera 31, and a pre-crash ECU 40, and includes a preprocessing unit 11, a reference pattern database 15, a vehicle recognition unit 16, a white line recognition unit 17, A pedestrian recognition unit 18, a pedestrian candidate area detection unit 52, and a collision determination unit 20 are included.

  The pedestrian candidate region detection unit 52 includes a plurality of saliency calculation units corresponding to the number of reference patterns, an average processing unit 52a, and a determination region extraction unit 52b. Then, the to-be-determined area cutout unit 52b cuts out the to-be-determined area from the image output from the preprocessing unit 11, performs saliency calculation for all the reference patterns on the extracted to-be-determined area, and averages each calculation result. And output.

  That is, when the determination target region cutout unit 52b cuts out one determination target region, it is notable for all reference patterns such as empty, in-road use, and out-of-road use regardless of where the determination target region is cut out. The sex is calculated and the value of the calculation result is averaged.

  For this reason, the judged area that has a large saliency value for all of the reference patterns for air, road, and road is prominent for any of the reference patterns for sky, road, and road. The value of the calculation result is larger than that of the determination target area where the sex value is small. The determination target area having a large value is output to the pedestrian recognition unit as a pedestrian candidate area.

  Therefore, it is possible to make a comprehensive judgment while reducing the events input in the learning process for each reference pattern.

  Since other configurations and operations are the same as those of the image recognition apparatus 10 shown in the first embodiment, the same components are denoted by the same reference numerals and description thereof is omitted.

  As described above, the image recognition apparatus 10 according to the second embodiment uses a different reference pattern for each background and takes the average of the calculation results of each reference pattern, thereby using a saliency algorithm with a simple configuration. The detection accuracy of the pedestrian candidate area is improved.

  In the second embodiment, the case where the calculation results of a plurality of reference patterns are averaged has been described. However, for example, the calculation method for a plurality of calculation results such as weighting or employing a maximum value is appropriately changed. Can be implemented.

  Further, the present invention is not limited to the configurations shown in the first embodiment and the second embodiment, and can be implemented by an arbitrary configuration. For example, the configuration shown in the first embodiment and the configuration shown in the second embodiment are combined, and for a determination target region cut out across a plurality of background regions, calculation using a reference pattern corresponding to each background is performed. You may implement this invention by the structure which takes the average of a result.

  In the first embodiment and the second embodiment, the case where a specific object, that is, a pedestrian is recognized from an image taken by the in-vehicle camera is described as an example. For example, a traffic monitoring camera that can move (rotate). It is also applicable to images taken by In this embodiment, the saliency calculation is applied. However, the present invention is not limited to this, and a background difference method of storing a background image as a reference pattern and extracting a candidate image based on a difference from the current image may be applied.

  As described above, the object candidate region detection device, the object candidate region detection method, the pedestrian recognition device, and the vehicle control device according to the present invention are useful for image recognition in a vehicle, and particularly for improving detection accuracy of a pedestrian candidate region. Is suitable.

1 is a schematic configuration diagram showing a schematic configuration of an image recognition apparatus according to Embodiment 1 of the present invention. It is explanatory drawing explaining the detection of the candidate area | region by a saliency algorithm. It is explanatory drawing explaining the detection accuracy fall of a candidate area | region. It is explanatory drawing explaining the pedestrian candidate area | region detection based on the background of an image. It is explanatory drawing explaining the division | segmentation of the background area | region in an input image. It is a flowchart explaining the processing operation of pedestrian recognition. It is a flowchart explaining the processing operation of the pre-crash ECU shown in FIG. It is a schematic block diagram which shows schematic structure of the image recognition apparatus concerning Example 2 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image recognition apparatus 10a Microcomputer 11 Pre-processing part 12 Background area division part 12a Vanishing point recognition part 12b Lane recognition part 14 Reference pattern selection part 15 Reference pattern database 16 Vehicle recognition part 17 White line recognition part 18 Pedestrian recognition part 19,52 Walking Person candidate area detection unit 19a, 52a To-be-determined area extraction unit 20 Collision determination unit 30 Navigation device 30a Map data 31 Camera

Claims (8)

An object candidate area detection device that detects an area where a specific object may exist from an input image captured by a camera as an object candidate area,
For the background of the input image, reference pattern storage means for storing a plurality of different reference patterns for each of the road area and the outside road area;
A background area dividing means for dividing an input image currently captured by the camera into a road area and an area outside the road for the background;
To-be-determined area cutting out means for cutting out the to-be-determined area from the input image currently captured by the camera;
A reference pattern selecting means for selecting a reference pattern corresponding to the background area from the plurality of reference patterns depending on whether the determined area is cut out from a background area of a road area or an area outside the road;
Detecting means for detecting the object candidate area in the determined area by comparing the selected reference pattern and the determined area;
An object candidate area detecting apparatus comprising:   The object candidate region detection apparatus according to claim 1, wherein the background region dividing unit divides the upper side from the vanishing point of the input image as a region outside the road.   The object candidate region detection apparatus according to claim 1, wherein the background region dividing unit divides an in-road region and an out-of-road region using a lane recognition result for the input image. An object candidate area detection device that detects an area where a specific object may exist from an input image captured by a camera as an object candidate area,
For the background of the input image, reference pattern storage means for storing a plurality of different reference patterns for each of the road area and the outside road area;
To-be-determined area cutting out means for cutting out the to-be-determined area from the input image currently captured by the camera;
Detecting means for detecting the object candidate region of the determination target region by comparing the determination target region and each of the plurality of reference patterns and combining the plurality of comparison results;
An object candidate area detecting apparatus comprising:   The detecting means digitizes the difference between the determination target area and the plurality of reference patterns, and outputs the determination target area as the object candidate area when an average value of the obtained plurality of numerical values is equal to or greater than a threshold value. The object candidate region detection apparatus according to claim 4, wherein An object candidate area detection method for detecting an area where a specific object may exist from an input image captured by a camera as an object candidate area,
For the background of the input image, a reference pattern storage step for storing a plurality of reference patterns different for each of the road area and the area outside the road;
A detection step of detecting the object candidate region by comparing the determination target region cut out from the input image and the plurality of reference patterns;
The object candidate area | region detection method characterized by including. A pedestrian recognition device that recognizes a pedestrian from an input image taken by an in-vehicle camera,
For the background of the input image, reference pattern storage means for storing a plurality of different reference patterns for each of the road area and the outside road area;
Detecting means for detecting a region where a pedestrian may exist as a pedestrian candidate region by comparing the to-be-determined region cut out from the input image and the plurality of reference patterns;
Pedestrian determination means for determining whether the pedestrian candidate area corresponds to a pedestrian,
A pedestrian recognition device comprising: Reference pattern storage means for learning in advance and storing a plurality of reference patterns that differ for each of the road area and the area outside the road for the background of the image;
A detection unit that detects a region where a pedestrian may exist as a pedestrian candidate region by comparing the determination target region cut out from the input image captured by the in-vehicle camera and the plurality of reference patterns;
Pedestrian determination means for determining whether the pedestrian candidate area corresponds to a pedestrian,
Control means for executing notification control for notifying the presence of a pedestrian and / or running control for controlling the running state of the vehicle when a pedestrian is recognized by the pedestrian discrimination means;
A vehicle control device comprising:

Images