JP2021033510A - Driving assistance device - Google Patents

Abstract

To provide a driving assistance device capable of detecting objects needing attention, and notifying thereof, in a more accurate manner.SOLUTION: Provided is a driving assistance device 100 for detecting objects present on a peripheral road of a vehicle 1, comprising: an image obtaining section 10 for obtaining images from an on-vehicle camera 200 which is loaded on the vehicle 1 and photographs the peripheral road; an image analysis section 20 for detecting an object candidate from the images using a trained identifier D1, and calculating reliability pertaining to the object candidate; and a notifying section 30 for determining whether or not the reliability of the object candidate detected at the image analysis section 20 is equal to or greater than a threshold, and performing notification on the object candidate as an object needing attention only if the reliability is equal to or greater than the threshold, wherein the notifying section 30 sets the threshold to be applied to the determination of the object candidate, based on a position or a size of the object candidate within the images.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a driving support device.

Conventionally, an image processing device that detects an object existing in an image by performing image analysis using a classifier model (hereinafter, abbreviated as "classifier") such as a convolutional neural network is known. (See, for example, Patent Document 1).

In this type of image processing device, object detection is generally performed using a classifier that has been trained using the image of the object to be detected as training data, and is similar to these objects to be detected in the image. When there is an image area to be used, the image area is detected as a candidate area in which an object exists (hereinafter, referred to as “object candidate”).

This type of classifier is usually configured to output, for each of the detected object candidates, a reliability indicating the probability that the object candidate corresponds to the type of object to be detected. Then, in the image processing device, in order to suppress erroneous detection, it is determined whether or not the reliability of the object candidate detected by the classifier is equal to or higher than the threshold value, and when the reliability of the object candidate is equal to or higher than the threshold value. Only, the object candidate is targeted for notification.

JP-A-2018-195293

By the way, in recent years, this kind of image processing technology detects pedestrians, other vehicles, falling objects, etc. (hereinafter referred to as "objects requiring attention") existing on the road in front of the vehicle to support the driving of the vehicle. It is being considered to be applied to driving support devices.

When applying this type of image processing technology to a driving assistance device, for objects requiring attention on the road ahead of the vehicle, the user of the vehicle or the vehicle can be used at an early stage even if the distance from the vehicle is long. It is desirable to notify the ECU (Electronic Control Unit).

However, the classifier generally tends to output the reliability of the object candidate as a small value when the size of the object candidate detected in the image is relatively small. Further, the classifier generally tends to output the reliability as a large value when the size of the object candidate detected in the image is relatively large.

As a result, if this kind of image processing technology is diverted to the driving support device as it is, there is a possibility that an object requiring attention distant from the vehicle will not be detected. Further, when an object other than the object requiring attention (for example, a building or a signboard) exists in the vicinity of the vehicle, there is a possibility that they are erroneously notified as the object requiring attention.

The present disclosure has been made in view of the above problems, and an object of the present disclosure is to provide a driving support device capable of more accurately detecting and notifying an object requiring attention.

The main disclosure that solves the above-mentioned problems is
It is a driving support device that detects objects existing on the road around the vehicle.
An image acquisition unit that acquires an image from an in-vehicle camera that photographs the surrounding roads mounted on the vehicle, and an image acquisition unit.
An image analysis unit that detects object candidates from the image using a trained classifier, and
A notification unit that determines whether or not the reliability of the object candidate detected by the image analysis unit is equal to or higher than the threshold value, and notifies the object candidate as an object requiring attention only when the reliability is equal to or higher than the threshold value. ,
With
The notification unit sets the threshold value applied to the determination of the object candidate based on the position or size of the object candidate in the image.
It is a driving support device.

According to the driving support device according to the present disclosure, it is possible to more accurately detect and notify an object requiring attention.

The figure which shows the structure of the vehicle which mounts the driving support device which concerns on 1st Embodiment The figure which shows the hardware configuration of the driving support device which concerns on 1st Embodiment The figure which shows an example of the display screen which the driving support device which concerns on 1st Embodiment displays on an in-vehicle monitor. Diagram showing an example of the configuration of a general convolutional neutral network The figure explaining the method of setting the threshold value of the notification part which concerns on 1st Embodiment. A flowchart showing an example of the operation of the driving support device according to the first embodiment. The figure explaining the method of setting the threshold value of the notification part which concerns on 2nd Embodiment. The figure explaining the calculation method of the position in the depth direction of the object candidate by the notification part which concerns on 3rd Embodiment. The figure explaining the calculation method of the position in the depth direction of the object candidate by the notification part which concerns on 4th Embodiment.

Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same function are designated by the same reference numerals, so that duplicate description will be omitted.

[Driving support device configuration]
Hereinafter, the configuration of the driving support device according to the first embodiment will be described with reference to FIG.

The driving support device according to the present embodiment is mounted on a vehicle such as an automobile, detects an object requiring attention on the road in front of the vehicle, and alerts the user to assist the user in driving or the vehicle. It is used for vehicle motion control of automatic driving by notifying the ECU of. Hereinafter, when the term "vehicle" is used, it means a vehicle equipped with the driving support device according to the present embodiment, and when referring to a vehicle different from the vehicle equipped with the driving support device according to the present embodiment, "vehicle" is used. Called "another vehicle".

FIG. 1 is a diagram showing a configuration of a vehicle 1 equipped with a driving support device 100 according to the present embodiment. FIG. 2 is a diagram showing a hardware configuration of the driving support device 100 according to the present embodiment. FIG. 3 is a diagram showing an example of a display screen displayed on the in-vehicle monitor 300 by the driving support device 100 according to the present embodiment.

The vehicle 1 includes, for example, a driving support device 100, an in-vehicle camera 200, and an in-vehicle monitor 300. These are mounted in the vehicle 1, for example.

The driving support device 100 detects an object requiring attention existing on the road ahead of the vehicle 1 based on the image of the road ahead of the vehicle 1 generated by the vehicle-mounted camera 200. Then, for example, the driving support device 100 superimposes an image that calls attention to the detected object requiring attention on the image generated by the vehicle-mounted camera 200 and displays it on the vehicle-mounted monitor 300.

The operation support device 100 has, as main components, a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an external storage device (for example, a flash memory) 104, a communication interface 105, and the like. It is a computer equipped with. Each function described later in the driving support device 100 is realized, for example, by the CPU 101 referring to a control program or various data stored in the ROM 102, the RAM 103, or the external storage device 104. The external storage device 104 stores, for example, a control program for realizing various functions described later, model data of a discriminator referred to by the image analysis unit 20, and the like.

However, a part or all of each function may be realized by the processing by the DSP (Digital Signal Processor) instead of or in combination with the processing by the CPU. Similarly, a part or all of each function may be realized by processing by a dedicated hardware circuit instead of or together with processing by software.

The in-vehicle camera 200 is, for example, a general camera or a wide-angle camera, and AD-converts an image signal generated by its own image sensor to generate image data (hereinafter, abbreviated as “image”). The in-vehicle camera 200 is fixed in the vicinity of the windshield in the vehicle 1, for example, and photographs the road ahead of the vehicle 1 through the windshield. The in-vehicle camera 200 continuously generates, for example, frame-by-frame images to generate a moving image of the road ahead of the vehicle 1.

The in-vehicle monitor 300 is, for example, a general display and displays an image acquired from the driving support device 100. The in-vehicle monitor 300 functions as a notification means for notifying the user of the vehicle 1 of the object detected by the driving support device 100.

For example, as shown in FIG. 3, the vehicle-mounted monitor 300 displays a monitor screen with markers M1 and M2 attached to the detected object so as to be superimposed on the image generated by the vehicle-mounted camera 200. In FIG. 3, R represents the entire image area, P1 represents a pedestrian, P2 represents another vehicle, P3 represents a signboard, P4 represents a building, and L represents a road area. Here, only the pedestrian P1 and the other vehicle P2 are detected as objects requiring attention as the notification target, and the signboard P3 and the building P4 are not detected as the notification target objects.

The driving support device 100 includes, for example, an image acquisition unit 10, an image analysis unit 20, and a notification unit 30.

The image acquisition unit 10 acquires an image generated by the in-vehicle camera 200.

The image analysis unit 20 uses the trained classifier D1 to perform image analysis processing on the image acquired from the in-vehicle camera 200, detects an object candidate from the image, and has a reliability related to the object candidate. Is calculated. Here, a convolutional neutral network is used as the classifier D1. As described above, the "reliability" means whether or not the object candidate is an object that actually exists when the classifier D1 detects the object candidate from the image, or the object candidate is It is a value to be output as an index indicating whether or not it corresponds to the type of the object to be detected.

FIG. 4 is a diagram showing an example of the configuration of a general convolutional neutral network.

The convolutional neutral network has a feature extraction unit Da and an identification unit Db, the feature extraction unit Da performs a process of extracting an image feature from an input image, and the identification unit Db is an object of an object reflected in the image. Performs processing to identify the type.

The feature extraction unit Da is configured by hierarchically connecting a plurality of feature extraction layers Da1, Da2, .... Each feature amount extraction layer Da1, Da2 ... Includes a convolution layer, an activation layer, and a pooling layer, respectively.

The feature amount extraction layer Da1 of the first layer scans the input image for each predetermined size by raster scanning. Then, the feature amount extraction layer Da1 extracts the feature amount included in the input image by performing the feature amount extraction process on the scanned data by the convolutional layer, the activation layer and the pooling layer. The feature amount extraction layer Da1 of the first layer extracts a relatively simple single feature amount such as a linear feature amount extending in the horizontal direction and a linear feature amount extending in the diagonal direction, for example.

The feature amount extraction layer Da2 of the second layer scans an image (also referred to as a feature map) input from the feature amount extraction layer Da1 of the previous layer for each predetermined size by, for example, raster scanning. Then, the feature amount extraction layer Da2 extracts the feature amount included in the input image by similarly performing the feature amount extraction process by the convolutional layer, the activation layer and the pooling layer on the scanned data. It should be noted that the feature amount extraction layer Da2 of the second layer is integrated while considering the positional relationship of a plurality of feature amounts extracted by the feature amount extraction layer Da1 of the first layer, so that it is a higher-dimensional complex. Extract features.

The feature amount extraction layer (not shown) of the second and subsequent layers executes the same processing as the feature amount extraction layer Da2 of the second layer. Then, the output of the feature amount extraction layer of the final layer (each value in the map of the plurality of feature maps) is input to the identification unit Db.

The identification unit Db is composed of, for example, a multi-layer perceptron in which a plurality of fully connected layers (Fully Connected) are hierarchically connected.

The fully connected layer on the input side of the identification unit Db is fully connected to each value in the map of the plurality of feature maps acquired from the feature extraction unit Da, and the product-sum operation is performed while making the weighting coefficient different for each value. Go and output. The fully connected layer of the next layer of the identification unit Db is fully coupled to the values output by each element of the fully connected layer of the previous layer, and the product-sum operation is performed while making the weighting coefficients different for each value.

An output element is provided for each category of the object to be detected (that is, an object requiring attention) in the fully connected layer at the final stage of the identification unit Db, for example. As a result, when the identification unit Db detects an object candidate corresponding to the detection target object in the image, the identification unit Db is configured to output the probability that the object candidate corresponds to the detection target object as the reliability.

Further, in the fully connected layer at the last stage of the identification unit Db, for example, for each category of the object to be detected, a rectangular shape surrounding the position where the object reflected in the image exists (for example, the area where the object reflected in the image exists) is formed. An output element is provided to output (XY coordinates of the four corners of the above). As a result, the identification unit Db can output the position of the detected object candidate in the image.

The convolutional neutral network is subjected to learning processing (for example, learning processing by the error backpropagation method) using an image showing an object to be detected with a correct answer category (and the correct position where the object exists). Network parameters (eg, convolution layer weighting factors and biases, fully connected layer weighting factors and biases) are adjusted to function as described above.

The structure of the convolutional neutral network shown in FIG. 4 is an example, and the structure of the convolutional neutral network for object detection can be variously modified. For example, a detector for detecting an object region (bounding box) may be provided in front of the feature extraction unit Da.

The classifier D1 (here, the convolutional neutral network) according to the present embodiment detects an object that needs to be recognized by the user or the ECU of the vehicle 1, for example, a vehicle, a pedestrian, a falling object, or the like from the image. Is subjected to learning processing, and is configured so that these objects can be detected in the image.

The trained model data of the classifier D1 (network structure data, trained parameter data, etc.) is stored in the external storage device 104 together with the control program, for example.

The image analysis unit 20 detects an object candidate from the image and calculates the reliability by performing forward propagation processing of the learned classifier D1 (convolution neutral network), for example.

The notification unit 30 determines whether or not the reliability of the object candidate detected by the image analysis unit 20 is equal to or higher than the threshold value. Then, when the reliability of the object candidate is equal to or higher than the threshold value, the notification unit 30 notifies the object candidate as an object requiring attention. On the other hand, when the reliability of the object candidate is less than the threshold value, the notification unit 30 does not target the object candidate for notification.

However, the notification unit 30 according to the present embodiment sets the threshold value based on the position or size of the object candidate in the image. That is, when a plurality of object candidates are detected from the image, the notification unit 30 sets a separate threshold value for each of the plurality of object candidates. The notification unit 30 typically receives a distance between the object candidate and the vehicle 1 when the distance between the object candidate and the vehicle 1 estimated from the position or size in the image of the object candidate is equal to or greater than a predetermined distance. The threshold value applied to the object candidate is made smaller than when the distance is less than a predetermined distance.

As described above, in a general image processing apparatus, it is determined whether or not the object candidate is to be notified based on a certain threshold value regardless of the position of the detected object candidate. However, in such an embodiment, it becomes difficult to detect an object existing in a distant place, which may hinder the safe driving of the user or the ECU of the vehicle 1 (for example, the vehicle ECU that executes automatic driving). Further, in such an embodiment, for an object existing at a short distance, an object that is not a detection target may be erroneously detected as a detection target object, which may give a complicated impression to the user or induce an erroneous operation of the user or the vehicle ECU. There is also.

Therefore, the notification unit 30 according to the present embodiment sets the threshold value applied to the object candidate to be smaller as the distance between the object candidate and the vehicle 1 becomes longer, and the distance between the object candidate and the vehicle 1 becomes shorter. Indeed, the threshold value applied to the object candidate is set large. As a result, the object candidate existing far from the vehicle 1 is likely to be the notification target.

FIG. 5 is a diagram illustrating a method of setting a threshold value by the notification unit 30 according to the present embodiment. R in FIG. 5 represents the entire image region of the image generated by the vehicle-mounted camera 200, Ra represents the region in which the object candidate exists, and the f value represents the threshold value before normalization. In FIG. 5, x _min , x _max , y _min , and y _max represent the x and y coordinates of the four corners of the region Ra where the object candidate exists when the upper left edge of the image is the origin. Further, _{Y _Hori} in Figure 5 represents the horizon estimation position in the _{image, Y _near} the vicinity of the vehicle 1 in the image (e.g., distance from the vehicle position about 3m) represents.

For example, the notification unit 30 reduces the threshold value applied to the object candidate as the distance between the object candidate and the vehicle 1 increases so that an object existing in a distant place can be easily notified. At this time, it is desirable that the notification unit 30 estimates the distance between the object candidate and the vehicle 1 based on, for example, the horizon estimation position _{Y_hori in the image.} For example, the notification unit 30 estimates that the closer the position in the image of the object candidate is from the lower end of the image to the estimated horizon position _{Y_hori} of the road ahead, the longer the distance between the object candidate and the vehicle 1.

When the installation position of the in-vehicle camera 200 is fixed, the horizon is usually reflected at a predetermined position in the image (here, a flat road is assumed), so that the notification unit 30 is set to the predetermined position. _{May be used} as the estimated horizon position Y_hori. On the other hand, the notification unit 30 may arbitrarily set the _{horizon estimated position Y_hori} as long as it is a position below the _{horizon estimated position Y_hori on the image.}

The notification unit 30 calculates the threshold value applied to the object candidate by using, for example, the following equations (1-1) to (1-3) and equation (2). For example, the notification unit 30 normalizes the f value calculated by the equations (1-1) to (1-3) to the scale of the reliability output from the classifier D1 by using the equation (2). _{The threshold value Cth} applied to the object candidate is calculated accordingly.

Here, when the position of the object candidate is Y _{_ori} <y _max <Y _{_near} , it means that the detected object candidate exists in the road area and is located farther from the vehicle than the nearest neighbor area. Since the object candidate is likely to be an object requiring attention, it is desirable to make it easy to be notified. From this point of view, the notification unit 30 determines that when Y _{_hori} <y _max <Y _{_near} , the detected object candidate does not exist in the road area or exists within the nearest neighbor area from the vehicle (y _max <Y _{_hori} , Alternatively, the threshold value is set smaller than _{Y _nea} <y _max). In addition, in order to detect a distant object more accurately, the notification unit 30 sets the threshold value smaller as _{y max} approaches from Y _{_near to Y _ hori.}

On the other hand, when the position of the object candidate is y _max < _{Y_hori} , it means that the detected object candidate exists in the sky above the road area. Since the object candidate is likely to be an object other than an object requiring attention, such as a pedestrian bridge or a signboard, it is desirable to exclude it from the notification target. From this point of view, the notification unit 30 sets the threshold value near the maximum value when _{y max} <Y _{_ori.}

On the other hand, when the position of the object candidate is Y _{_nea} <y _max , it means that the detected object candidate exists in the vicinity of the vehicle 1. The object candidate includes a case where it is an object requiring attention, but also includes a case where it is an object other than that requiring attention. When the detected object candidate exists in the vicinity of the vehicle 1, the reliability of the object candidate tends to be calculated to a relatively large value. From this point of view, the notification unit 30 sets the threshold value near the maximum value when _{y max} <Y _{_ori} , in order to suppress erroneous detection of an object other than the one requiring attention.

In the above, the mode of calculating the threshold value applied to the object candidate is shown by referring only to the Y coordinate of the object candidate. However, when calculating the threshold value applied to the object candidate, the X coordinate of the object candidate may also be referred to. Thereby, for example, it is possible to more accurately identify whether or not the object candidate exists in the road area.

In this way, the notification unit 30 sets a threshold value applied to the object candidate for each of the plurality of object candidates detected by the image analysis unit 20. Then, the notification unit 30 notifies only the object candidates whose reliability is equal to or higher than the threshold value among the object candidates detected by the image analysis unit 20 as actually existing objects. At this time, for example, the notification unit 30 generates a display image with marks M1 and M2 in the area where the object to be notified exists with respect to the image generated by the vehicle-mounted camera 200, and outputs the display image to the vehicle-mounted monitor 300. (See FIG. 3). As a result, the user can visually recognize the in-vehicle monitor 300 and recognize the object existing on the road ahead.

The mode of notification by the notification unit 30 is arbitrary. For example, the notification unit 30 may notify by voice using a speaker. Further, the notification unit 30 may output a notification signal to an ECU (not shown) that controls the vehicle 1 so that the ECU can automatically control the moving state of the vehicle 1. ..

[Operation flow]
FIG. 6 is a flowchart showing an example of the operation of the driving support device 100 according to the present embodiment.

The driving support device 100 first acquires an image from the in-vehicle camera 200 (step S1).

Next, the driving support device 100 uses the classifier D1 to perform image analysis of the image acquired from the in-vehicle camera 200, and detects an object candidate from the image. At this time, the driving support device 100 also calculates the position and reliability of the object candidate in the image (step S2).

Next, the driving support device 100 sets a threshold value applied to determine whether or not the object candidate is to be notified based on the position of the object candidate in the image (step S3).

Next, the driving support device 100 determines whether or not the reliability of the object candidate is equal to or higher than the threshold value (step S4). Then, when the reliability of the object candidate is equal to or higher than the threshold value (step S4: YES), the process proceeds to step S5, and the notification process for the object candidate is executed (step S5). On the other hand, when the reliability of the object candidate is less than the threshold value (step S4: NO), a series of processes are completed without executing the notification process for the object candidate.

When a plurality of object candidates are detected in the image, the driving support device 100 sets a threshold value for each of the plurality of object candidates in step S3. Then, in step S4, the driving support device 100 determines whether or not the reliability of the object candidates is equal to or higher than the threshold value for each of the plurality of object candidates.

[effect]
As described above, the driving support device 100 according to the present embodiment uses the image acquisition unit 10 that acquires an image from the in-vehicle camera 200 that captures the front road mounted on the vehicle 1 and the trained classifier D1. , The image analysis unit 20 that detects an object candidate from an image determines whether or not the reliability of the object candidate is equal to or higher than the threshold value, and pays attention to the object candidate only when the reliability of the object candidate is equal to or higher than the threshold value. A notification unit 30 for notifying as an object is provided, and the notification unit 30 sets a threshold value applied to the object candidate based on the position or size of the object candidate in the image.

Therefore, according to the driving support device 100 according to the present embodiment, it is possible to suppress undetection of an object requiring attention (for example, another vehicle or a pedestrian) existing far from the vehicle 1. Further, according to the driving support device 100 according to the present embodiment, an object existing in a region close to the vehicle 1 that does not require special attention (for example, a building) is erroneously detected as an object requiring attention. It can be suppressed.

(Second embodiment)
Next, the driving support device 100 according to the second embodiment will be described with reference to FIG. 7.

The driving support device 100 according to the present embodiment is different from the driving support device 100 according to the first embodiment in that a threshold value applied to the object candidate is set based on the size of the object candidate. The configuration common to the first embodiment will be omitted from the description (hereinafter, the same applies to the other embodiments).

FIG. 7 is a diagram illustrating a method of setting a threshold value of the notification unit 30 according to the present embodiment. In FIG. 7, size is the size of the object candidate (area Ra where the object candidate exists), size _max is the size of the object that needs attention when applied with the maximum threshold value, and size _min is the size of the object that needs attention when setting the minimum threshold value. Represents the size of an object. Note that size _max and size _min are set for each category of objects requiring attention.

The notification unit 30 according to the present embodiment estimates the distance between the object candidate and the vehicle 1 based on the size of the object candidate detected by the classifier D1 in the image, and sets a threshold value applied to the object candidate. Set. The size of the object candidate is, for example, from the x-coordinates and y-coordinates (that is, x _min , x _max , y _min , y _max ) of the four corners of the area Ra where the object candidate is output from the classifier D1. Calculated as area. The size of the object candidate may be a value obtained by taking a route with respect to the area of the rectangular region.

The objects requiring attention to be detected by the driving support device 100 are vehicles and pedestrians, and the reference size is fixed. Therefore, the driving support device 100 stores, for example, a reference size for each object category (for example, a vehicle or a pedestrian) in advance in ROM 102 or the like, and the size of the object candidate detected in the image and the object candidate. By comparing with the reference size of the object candidate of the same category, the distance between the object candidate and the vehicle 1 can be estimated.

The notification unit 30 calculates the threshold value using, for example, the following equations (3-1) to (3-3) and equation (4). The notification unit 30 normalizes the f-number calculated by the equations (3-1) to (3-3) to the scale of the reliability output from the classifier D1 by using the equation (4). Calculate the threshold C _th.

The above equation represents that the threshold value is set to a smaller value as the size of the object candidate to be detected becomes smaller. This is because it is estimated that the detected object candidate exists farther as the size of the object candidate to be detected becomes smaller.

As a result, even in a situation where the position of the object candidate cannot be accurately recognized (for example, a situation where the estimated horizon position cannot be recognized in the image), the notification unit 30 determines whether or not the object candidate is to be notified. It is possible to set the threshold appropriately.

The notification unit 30 uses any of the following equations (5-1) to (5-3) and equation (6) to set a threshold value based on both the position and size of the object in the image. It may be set.

This makes it possible to deal with both a situation in which the position of the object candidate cannot be accurately recognized and a situation in which the size of the object candidate cannot be accurately recognized.

As described above, according to the driving support device 100 according to the present embodiment, it is possible to more appropriately set the threshold value applied to the object candidate.

(Third Embodiment)
Next, the driving support device 100 according to the third embodiment will be described with reference to FIG. The driving support device 100 according to the present embodiment is different from the driving support device 100 according to the first embodiment in that a threshold value applied to the object candidate is set based on the position of the object candidate in the depth direction.

FIG. 8 is a diagram illustrating a method of calculating the position of the object candidate in the depth direction by the notification unit 30. In FIG. 8, Po represents an object candidate, 201 represents an imaging surface of the vehicle-mounted camera 200, and 202 represents a lens 202 of the vehicle-mounted camera 200.

The notification unit 30 according to the present embodiment uses the following equations (7) and (8) based on the y-coordinate of the object candidate detected in the image, for example, to position the object candidate in the depth direction (that is,). , Distance between the vehicle-mounted camera 200 and the object candidate) Z is calculated. Then, the notification unit 30 calculates the f-number from the distance Z between the vehicle-mounted camera 200 and the object candidate, for example, using the equation (9). _{Then, the threshold value Cth} is calculated by normalizing the calculated f value to the scale of the reliability output from the classifier D1 using the equation (2). In the equations (7) and (8), the distance between the object candidate and the vehicle 1 is estimated on the assumption that the lower end of the object requiring attention is always on the road surface.

As described above, according to the driving support device 100 according to the present embodiment, the position of the object candidate in the depth direction (that is, the distance between the in-vehicle camera 200 and the object candidate) can be calculated more accurately. , The threshold value applied by the notification unit 30 can be set more appropriately.

(Fourth Embodiment)
Next, the driving support device 100 according to the fourth embodiment will be described with reference to FIG. The driving support device 100 according to the present embodiment is different from the driving support device 100 according to the first embodiment in that a threshold value applied to the object candidate is set based on the position of the object candidate in the depth direction.

This embodiment is applied when a stereo camera is used as the vehicle-mounted camera 200. The stereo camera can directly estimate the position of the object candidate in the depth direction at the time of shooting by the principle of triangulation.

FIG. 9 is a diagram illustrating a method of calculating the position of the object candidate in the depth direction by the notification unit 30. In FIG. 9, Po is an object candidate, 201 is an imaging surface (here, for convenience of explanation, the left imaging unit and the right imaging unit are commonly shown), 202a is a lens of the left imaging unit, and 202b is a lens of the left imaging unit. Represents the lens of the right imaging unit. Further, x _L represents the distance from the lens central axis of the lens 202a of the left imaging unit, and x _R represents the distance from the lens central axis of the lens 202a of the right imaging unit.

The notification unit 30 according to the present embodiment uses, for example, the following equation (10) based on the image of the left side image pickup unit and the image of the right side image pickup unit acquired from the vehicle-mounted camera 200 (here, the stereo camera). , The position (that is, the distance between the vehicle-mounted camera 200 and the object candidate) Z in the depth direction of the object candidate is calculated. Then, the notification unit 30 calculates an f-number from the distance Z between the vehicle-mounted camera 200 and the object candidate, for example, using the equation (9) in the same manner as the method described in the third embodiment, and the equation (9) is used. Using 2), the threshold value _Cth is calculated from the f value.

As described above, according to the driving support device 100 according to the present embodiment, the position of the object candidate in the depth direction (that is, the distance between the in-vehicle camera 200 and the object candidate) can be calculated more accurately. , The threshold value applied by the notification unit 30 can be set more appropriately.

(Other embodiments)
The present invention is not limited to the above embodiment, and various modifications can be considered.

For example, in each of the above embodiments, a convolutional neutral network is shown as an example of the classifier D1, but the classifier D1 expresses a discriminating function by machine learning such as a Bayesian classifier or an SVM (Support Vector Machine) classifier. It may be composed of other classifiers. Also, when a convolutional neutral network is used as the classifier D1, the model structure can be changed in various ways.

Further, in each of the above embodiments, as an example of the classifier D1, a configuration is shown in which the x-coordinates and y-coordinates of the four corners of the region Ra in which the object candidate exists are output. However, the information related to the position of the object candidate output by the classifier D1 may be only one point of the center of gravity of the object candidate.

Further, in the third and fourth embodiments, the notification unit 30 shows an aspect of calculating the position of the object candidate in the depth direction (that is, the distance between the vehicle-mounted camera 200 and the object candidate). However, these arithmetic processes may be executed in the in-vehicle camera 200.

Although specific examples of the present invention have been described in detail above, these are merely examples and do not limit the scope of claims. The techniques described in the claims include various modifications and modifications of the specific examples illustrated above.

According to the driving support device according to the present disclosure, it is possible to more accurately detect and notify an object requiring attention.

1 Vehicle 10 Image acquisition unit 20 Image analysis unit 30 Notification unit 100 Driving support device
101 CPU
102 ROM
103 RAM
104 External storage device 105 Communication interface 200 On-board camera 201 Imaging surface 202, 202a, 202b Lens 300 On-board monitor D1 classifier

Claims (9)

It is a driving support device that detects objects existing on the road around the vehicle.
An image acquisition unit that acquires an image from an in-vehicle camera that photographs the surrounding roads mounted on the vehicle, and an image acquisition unit.
An image analysis unit that detects object candidates from the image using a trained classifier, and
A notification unit that determines whether or not the reliability of the object candidate detected by the image analysis unit is equal to or higher than the threshold value, and notifies the object candidate as an object requiring attention only when the reliability is equal to or higher than the threshold value. ,
With
The notification unit sets the threshold value applied to the determination of the object candidate based on the position or size of the object candidate in the image.
Driving support device. When the distance between the object candidate and the vehicle estimated from the position or size of the object candidate in the image is greater than or equal to a predetermined distance, the notification unit is more likely than when the distance is less than the predetermined distance. Decrease the threshold,
The driving support device according to claim 1. The notification unit reduces the threshold value as the distance increases.
The driving support device according to claim 2. The notification unit estimates that the closer the position of the object candidate in the image is from the lower end of the image to the estimated position of the horizon in the image, the longer the distance is.
The driving support device according to claim 2 or 3. The notification unit estimates that the distance becomes longer as the size of the object candidate in the image becomes smaller than the reference size of the type related to the object candidate.
The driving support device according to any one of claims 2 to 4. The notification unit sets the threshold value for determining whether or not the object candidate is to be notified based on both the position and size of the object candidate in the image.
The driving support device according to any one of claims 1 to 5. When the position of the object candidate in the image is outside the road area of the surrounding road, the notification unit is more than when the position of the object candidate in the image is inside the road area of the surrounding road. Increase the threshold,
The driving support device according to any one of claims 1 to 6. The in-vehicle camera is a stereo camera.
The driving support device according to any one of claims 1 to 7. The classifier is a convolutional neural network.
The driving support device according to any one of claims 1 to 8.

Images