JP7051326B2 - Information processing equipment, information processing methods and programs - Google Patents

Description

The present disclosure relates to information processing devices, information processing methods and programs.

In places where the driver of the vehicle has poor visibility, such as at an intersection, accidents such as contact with or colliding with other vehicles or pedestrians are likely to occur. Therefore, it is expected to develop a technology that can predict the danger that may lead to the occurrence of an accident at an intersection or the like and prevent the occurrence of an accident.

For example, Patent Document 1 discloses a technique for driving a vehicle more safely by utilizing data recorded by a drive recorder. In Patent Document 1, this drive recorder records data showing a running situation when a peculiar thing occurs in a vehicle. Then, when it is predicted that the driving situation grasped from the data recorded by the drive recorder will be reproduced, the driver is alerted. In this way, the safety can be enhanced, so that the vehicle can be driven more safely.

Japanese Unexamined Patent Publication No. 2007-193577

However, in the method of directly utilizing the data observed by the drive recorder to predict the danger, the time section in which the peculiar thing leading to the hiyari hat occurs is short, so the driving situation when the peculiar thing occurs can be checked. It may be difficult to adequately record the data shown. Therefore, it is difficult to predict the reproduction of the driving situation and, by extension, the danger that may lead to the occurrence of an accident. In other words, with the above-mentioned conventional technology, it is difficult to predict a danger that may lead to an accident and to provide driving support such as controlling a vehicle or calling attention to a driver. In particular, in places with poor visibility, observation conditions such as narrowing of the observation range deteriorate, so the data observed by the drive recorder is likely to be insufficient.

The present disclosure has been made in view of the above circumstances, and an object of the present disclosure is to provide an information processing device and the like that can support safe driving of a vehicle in a place with poor visibility such as an intersection.

In order to achieve the above object, the information processing apparatus according to one aspect of the present disclosure includes an acquisition unit that acquires map information or image information indicating an image obtained by imaging of an image pickup device mounted on a vehicle, and the map information. Alternatively, a detection unit that detects a curve mirror around the vehicle based on the image information, and a feature determination unit that determines the detected characteristics of the installation location of the curve mirror based on the map information or the image information. It includes a generation unit that generates driving support information based on the generated characteristics, and an output unit that outputs the generated driving support information.

It should be noted that these general or specific embodiments may be realized in a recording medium such as a system, method, integrated circuit, computer program or computer-readable CD-ROM, and the system, method, integrated circuit, computer. It may be realized by any combination of a program and a recording medium.

According to the present disclosure, it is possible to realize an information processing device or the like that can support safe driving of a vehicle in a place with poor visibility such as an intersection.

FIG. 1 is a block diagram showing an example of the configuration of the system according to the first embodiment. FIG. 2 is a diagram showing an example of the functional configuration of the image acquisition unit according to the first embodiment. FIG. 3 is a diagram showing an example of a state when the image acquisition unit according to the first embodiment is mounted on a vehicle. FIG. 4 is an explanatory diagram of an example of an image acquired by the image acquisition unit in the first embodiment. FIG. 5 is an explanatory diagram of detection of a curved mirror based on detection of a simple circle, rectangle, or the like. FIG. 6 is an explanatory diagram of detection of a curved mirror based on statistical image recognition. FIG. 7 is an explanatory diagram of a method of recognizing an animal body reflected on a curved mirror using deep learning. FIG. 8 is a diagram showing an example of the functional configuration of the calculation unit according to the first embodiment. FIG. 9 is an explanatory diagram showing an example of the calculation process performed by the calculation unit in the first embodiment. FIG. 10 is an explanatory diagram showing an example of a blind spot determination process performed by the blind spot determination unit in the first embodiment. FIG. 11 is an explanatory diagram of a blind spot of a curved mirror. FIG. 12 is a diagram showing an example of the functional configuration of the output processing unit according to the first embodiment. FIG. 13 is a conceptual diagram showing the amount of change in the size of the object in FIG. 9B. FIG. 14 is a diagram showing an example of a curved curved road in the first embodiment. FIG. 15 is an explanatory diagram showing an example of output processing performed by the output processing unit according to the first embodiment. FIG. 16 is a diagram showing an example of a T-shaped intersection in the first embodiment. FIG. 17 is an explanatory diagram showing another example of the output processing performed by the output processing unit according to the first embodiment. FIG. 18 is a flowchart showing an outline of the information processing method of the system according to the first embodiment. FIG. 19 is a flowchart showing details of the information processing method of the system according to the first embodiment. FIG. 20 is an explanatory diagram showing an example of risk prediction processing performed by the output processing unit in the first modification of the first embodiment. FIG. 21 is an explanatory diagram showing an example of risk prediction processing performed by the output processing unit in the first modification of the first embodiment. FIG. 22 is an explanatory diagram showing an example of risk prediction processing performed by the output processing unit in the second modification of the first embodiment. FIG. 23 is an explanatory diagram showing an example of risk prediction processing performed by the output processing unit in the second modification of the first embodiment. FIG. 24 is an explanatory diagram showing an example of risk prediction processing performed by the output processing unit in the modification 3 of the first embodiment. FIG. 25 is an explanatory diagram showing an example of risk prediction processing performed by the output processing unit in the modification 3 of the first embodiment. FIG. 26 is a block diagram showing an example of the configuration of the system according to the second embodiment. FIG. 27A is a diagram showing an example of an object reflected in a curved mirror visually recognized by the driver of a vehicle temporarily stopped at an intersection. FIG. 27B is an explanatory diagram of the characteristics of the curved mirror. FIG. 27C is an explanatory diagram of the characteristics of the curved mirror. FIG. 28 is a diagram showing an example of the functional configuration of the calculation unit according to the second embodiment. FIG. 29A is a diagram showing an example of the position of the object calculated by the object position calculation unit shown in FIG. 28. FIG. 29B is a diagram showing an example of a road surface area calculated by the road surface area calculation unit shown in FIG. 28. FIG. 30 is an explanatory diagram of a method of calculating the central axis calculated by the central axis calculation unit shown in FIG. 28. FIG. 31 is a diagram showing an example of the functional configuration of the output processing unit according to the second embodiment. FIG. 32 is a diagram showing an example of the position of an object in the curved mirror in the second embodiment. FIG. 33 is an explanatory diagram of the degree of danger when an object is located behind the curved mirror in the second embodiment. FIG. 34 is an explanatory diagram of the degree of danger when an object is located behind the curved mirror in the second embodiment. FIG. 35 is an explanatory diagram of the degree of danger when an object is located behind the curved mirror in the second embodiment. FIG. 36 is an explanatory diagram of the degree of danger when an object is located on the front side of the curved mirror in the second embodiment. FIG. 37 is an explanatory diagram of the degree of danger when the object is located on the front side of the curved mirror in the second embodiment. FIG. 38 is an explanatory diagram of the degree of danger when an object is located on the front side of the curved mirror in the second embodiment. FIG. 39 is a diagram showing an example of a T-shaped intersection in the second embodiment. FIG. 40 is an explanatory diagram showing another example of the output processing performed by the output processing unit according to the second embodiment. FIG. 41 is a flowchart showing an outline of the information processing method of the system according to the second embodiment. FIG. 42 is a flowchart showing details of the information processing method of the system according to the second embodiment. FIG. 43 is an explanatory diagram showing an example of the risk prediction process performed by the output processing unit in the first modification of the second embodiment. FIG. 44 is an explanatory diagram showing an example of risk prediction processing performed by the output processing unit in the second modification of the second embodiment. FIG. 45 is an explanatory diagram showing an example of risk prediction processing performed by the output processing unit in the modification 3 of the second embodiment. FIG. 46 is a block diagram showing an example of the configuration of the system according to the third embodiment. FIG. 47 is a diagram showing an example of the functional configuration of the feature determination unit in the third embodiment. FIG. 48 is an explanatory diagram showing an example of a dynamic map. FIG. 49A is an example of the surrounding environment of the curved mirror reflected in the image in the third embodiment. FIG. 49B is an example of the surrounding environment of the curved mirror reflected in the image in the third embodiment. FIG. 49C is an example of the surrounding environment of the curved mirror reflected in the image in the third embodiment. FIG. 49D is an example of the surrounding environment of the curved mirror reflected in the image in the third embodiment. FIG. 49E is an example of the surrounding environment of the curved mirror reflected in the image in the third embodiment. FIG. 50 is a diagram showing an example of a map used by the installation location acquisition unit in the third embodiment. FIG. 51 is a diagram showing an example of a local dynamic map used by the first risk determination unit in the third embodiment for determination. FIG. 52 is a diagram showing an example of a local dynamic map used by the first risk determination unit in the third embodiment for determination. FIG. 53 is a diagram showing an example of a local dynamic map used by the first risk determination unit in the third embodiment for determination. FIG. 54 is a diagram showing an example of a local dynamic map used by the first risk determination unit in the third embodiment for determination. FIG. 55 is a diagram showing an example of the functional configuration of the output processing unit according to the third embodiment. FIG. 56 is an explanatory diagram showing an example of output processing performed by the output processing unit according to the third embodiment. FIG. 57 is an explanatory diagram showing an example of output processing performed by the output processing unit according to the third embodiment. FIG. 58 is a flowchart showing an outline of the information processing method of the system according to the third embodiment. FIG. 59 is a flowchart showing details of the information processing method of the system according to the third embodiment. FIG. 60 is an explanatory diagram showing an example of a second risk level determination process performed by the second risk level determination unit in the first modification of the third embodiment. FIG. 61 is an explanatory diagram showing an example of a second risk level determination process performed by the second risk level determination unit in the second modification of the third embodiment. FIG. 62 is an explanatory diagram showing an example of the second risk level determination process performed by the second risk level determination unit in the third modification of the third embodiment.

The information processing apparatus according to one embodiment of the present disclosure includes an acquisition unit that acquires map information or image information indicating an image obtained by imaging of an image pickup device mounted on a vehicle, and the map information or the image information based on the image information. A detection unit that detects a curve mirror around the vehicle, a feature determination unit that determines the detected feature of the installation location of the curve mirror based on the map information or the image information, and driving based on the determined feature. It includes a generation unit that generates support information and an output unit that outputs the generated operation support information.

As a result, the driving support information of the vehicle can be output by using the curved mirror in a place where the visibility is poor such as an intersection, so that the safe driving of the vehicle can be supported.

Here, for example, the detection unit detects the curve mirror based on the position of the vehicle and the installation location of the curve mirror indicated by the map information, and the feature determination unit detects the detected curve mirror. The feature may be determined based on the map information in the vicinity.

Here, for example, the map information includes additional map information indicating at least one of a traffic accident, a traffic jam, a construction, a road surface condition, and a weather on the map, and the feature is the aspect of the road at the place where the curved mirror is installed. Alternatively, including the mode of passage of the object at the installation location, the feature determination unit determines the mode of the road or the mode of passage of the object from the additional map information, and the generation unit determines the mode of passage of the determined road. The driving support information may be generated according to the mode or the mode of passage of the object.

Further, for example, the detection unit detects the curve mirror based on the image information by using the identification information of the curve mirror or the discriminator of the curve mirror, recognizes the surrounding environment of the curve mirror, and causes the said. The feature determination unit may determine the feature based on the surrounding environment of the curve mirror.

Further, for example, the feature includes a mode of a road at a place where the curved mirror is installed or a mode of passage of an object at the place of installation, and the feature determination unit is a mode of the road or the mode of the road from the surrounding environment of the curved mirror. The mode of passage of the object may be determined, and the generation unit may generate the driving support information according to the determined mode of the road or the mode of passage of the object.

Further, for example, when the determined mode of the road or the mode of passage of the object is a mode that hinders the safe running of the vehicle, the generation unit may perform the operation for decelerating, stopping, or detouring the vehicle. Support information may be generated.

Further, for example, the feature determination unit may further determine the feature based on the object reflected in the curve mirror.

Further, for example, the feature determination unit may use the driving support for decelerating, stopping, or detouring the vehicle depending on whether an object is reflected in the curved mirror or the amount of the object reflected in the curved mirror. Information may be generated.

Further, for example, the generation unit may generate the driving support information according to the danger predicted from the characteristics.

Further, for example, the driving support information may include control command information of the behavior of the vehicle.

Further, for example, the driving support information may include presentation information presented to the occupants of the vehicle.

Further, for example, the presented information may include information that presents the danger predicted from the characteristics at the place where the curved mirror is installed.

Further, for example, the information processing apparatus may include a recognition unit that recognizes an object reflected on the curved mirror.

Further, for example, the generation unit may generate the driving support information according to the attributes of the object recognized by the recognition unit.

Further, for example, when the attribute of the object recognized by the recognition unit is a person, the generation unit is generated by changing the driving support information according to the information regarding the age of the object recognized by the recognition unit. You may do so.

Further, for example, when the attribute of the object recognized by the recognition unit is a person, the generation unit may support the operation depending on whether or not the object recognized by the recognition unit is inadvertently acting. The information may be changed and generated.

Further, in the information processing method according to one embodiment of the present disclosure, a processor is used to acquire map information or image information indicating an image obtained by imaging an image pickup device mounted on a vehicle, and the map information or the image information. The curve mirror around the vehicle is detected based on the above, the detected feature of the installation location of the curve mirror is determined based on the map information or the image information, and the driving support information is obtained based on the determined feature. It is generated and the generated driving support information is output.

Further, the program according to one embodiment of the present disclosure is a program that can be read by a computer that performs an information processing method, and acquires map information or image information indicating an image obtained by imaging an image pickup device mounted on a vehicle. , The curve mirror around the vehicle is detected based on the map information or the image information, and the characteristics of the detected installation location of the curve mirror are determined based on the map information or the image information, and the determination is made. The driving support information is generated based on the characteristics, and the generated driving support information is output.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

Each of the embodiments described below is a specific example of the present disclosure. The numerical values, shapes, components, steps, order of steps, etc. shown in the following embodiments are examples, and are not intended to limit the present disclosure. Further, among the components in the following embodiments, the components not described in the independent claim indicating the highest level concept are described as arbitrary components. Moreover, in all the embodiments, each content can be combined.

(Embodiment 1)
[System 1 configuration]
FIG. 1 is a block diagram showing an example of the configuration of the system 1 according to the first embodiment.

The system 1 shown in FIG. 1 is mounted on a vehicle such as an automobile, includes an information processing device 10, an image acquisition unit 11, and a detection processing unit 12a, and outputs driving support information of the vehicle. Details of these configurations and the like will be described below. The image acquisition unit 11 and the detection processing unit 12a are not limited to the case shown in FIG. 1, and may be provided in the information processing device 10.

Here, curved mirrors are often installed in places with poor visibility. For example, a curved mirror is a convex mirror for ancillary safety checks that is installed on the road so that you can see what you cannot see directly at an intersection with poor visibility or a curve that is difficult to see ahead. Therefore, the information processing device 10 supports safer driving of the vehicle by using the curve mirror. The following curved mirror includes a physically existing circular or rectangular mirror, but may include a circular or rectangular electronic mirror that displays a state that cannot be directly seen by an image.

[Image acquisition unit 11]
FIG. 2 is a diagram showing an example of the functional configuration of the image acquisition unit 11 in the first embodiment. FIG. 3 is a diagram showing an example of a state when the image acquisition unit 11 according to the first embodiment is mounted on a vehicle. FIG. 4 is an explanatory diagram of an example of an image acquired by the image acquisition unit 11 in the first embodiment.

The image acquisition unit 11 acquires image information indicating an image obtained by imaging an image pickup device mounted on a vehicle. In the present embodiment, the image acquisition unit 11 includes a sensing unit 111 and an image recording unit 112, as shown in FIG. For example, the image acquisition unit 11 is an in-vehicle camera as shown in FIG. 3, and is mounted on a vehicle.

The sensing unit 111 continuously captures images in front of the moving vehicle in chronological order and records them in the image recording unit 112. Here, the sensing unit 111 is a moving image recording device such as a camera. The sensing unit 111 may be a visible light camera that performs visible light photography or an InGaAs camera that performs infrared light photography. In this case, the sensing unit 111 may take an image with visible light during the day and take an image with infrared light at night.

The image recording unit 112 includes an HDD (Hard Disk Drive), a memory, or the like, and records an image taken by the sensing unit 111.

In the following, for example, as in the image 50 shown in FIG. 4, a case where the curve mirror 51 is reflected in an image taken by the sensing unit 111 in front of a vehicle in progress at an intersection or other place with poor visibility will be described as an example. do.

[Detection processing unit 12a]
The detection processing unit 12a detects the curved mirror reflected in the image and the object reflected in the curved mirror based on the image information indicating the image obtained by the imaging of the image pickup device mounted on the vehicle. Further, the detection processing unit 12a may recognize the attribute of the object reflected on the curved mirror. In the present embodiment, the detection processing unit 12a includes a detection unit 12 and a recognition unit 13.

<Detection unit 12>
The detection unit 12 detects the curved mirror reflected in the image based on the image information indicating the image obtained by the image pickup of the image pickup device mounted on the vehicle. More specifically, the detection unit 12 detects a curved mirror reflected in an image obtained by continuously photographing the front of a moving vehicle in chronological order. The imaging direction may be a direction other than the front of the vehicle. For example, when the vehicle travels backward, the imaging direction may be behind the vehicle.

In the present embodiment, the detection unit 12 detects the curve mirror 51 acquired by the image acquisition unit 11, for example, reflected in the image 50 shown in FIG. Here, as a method for detecting a curved mirror in an image, for example, there is a method based on the detection of a simple circle or a rectangle, a method based on statistical image recognition, and the like. Hereinafter, a method of detecting a curved mirror in an image will be described with reference to the figure.

FIG. 5 is an explanatory diagram of detection of a curved mirror based on detection of a simple circle, rectangle, or the like. The same elements as in FIG. 4 are designated by the same reference numerals, and detailed description thereof will be omitted.

First, the detection unit 12 performs edge detection in, for example, the image 50 shown in FIG. 5 acquired by the image acquisition unit 11. As a result, the detection unit 12 detects the edge 51a corresponding to the shape of the curved mirror, as shown in the image 50a.

Next, the detection unit 12 searches for the edge similarity in the image 50a in which the edge is detected. More specifically, as shown in the image 50a, the detection unit 12 scans the filter (mirror shape filter 52) for detecting a circle or a rectangle in the image 50a to obtain an edge similar to the mirror shape filter 52. Explore. The mirror shape filter 52 is a filter for detecting the contour of the curved mirror, and for example, a sobel filter, a canny filter, or the like can be used.

Then, the detection unit 12 creates, for example, a similarity map 54, and detects a position 53 having a high degree of similarity in the similarity map 54 as a curve mirror 51.

If the position of the subject reflected in the image acquired by the image acquisition unit 11 on the map is known, a filter for detecting a circle or a rectangle based on information such as the shape of the curved mirror that can be estimated from the position is used. It may be selected and used.

FIG. 6 is an explanatory diagram of detection of a curved mirror based on statistical image recognition. The same elements as in FIG. 4 are designated by the same reference numerals, and detailed description thereof will be omitted.

For example, the detection unit 12 scans the matching region 52a including the circles of different sizes in the image 50 acquired by the image acquisition unit 11, for example, shown in FIG. 6A.

As a result, as shown in FIG. 6B, the detection unit 12 creates a similarity map 54a showing the similarity with the circle according to each position in the image 50 and the size at that position.

Then, the detection unit 12 detects the position 53a having a high degree of similarity in the similarity map 54a as the curve mirror 51.

Note that statistical image recognition is not limited to the case described with reference to FIG. The detection unit 12 may include a classifier (Deep Learning, SVM, etc.) that has learned a large number of images of the curved mirror and images other than the curved mirror in advance. In this case, the detection unit 12 may detect the curve mirror 51 by using this discriminator to determine whether or not the image is a curve mirror image at each position in the image 50.

<Recognition unit 13>
The recognition unit 13 recognizes an object reflected on a curved mirror based on image information indicating an image obtained by imaging by an image pickup device mounted on a vehicle. More specifically, the recognition unit 13 recognizes an object in the curved mirror reflected in the image acquired by the image acquisition unit 11. The recognition unit 13 may recognize the attribute of the object reflected on the curved mirror.

In the present embodiment, the recognition unit 13 recognizes the position and size of a moving object (hereinafter, also referred to as an animal body) such as a person, a bicycle, or a car reflected in the curved mirror included in the image detected by the detection unit 12. .. Here, as a method of recognizing an animal body existing in a curved mirror reflected in an image, there is a method of using machine learning such as deep learning. Hereinafter, a method of recognizing an animal body existing in a curved mirror reflected in an image will be described with reference to FIG. 7.

FIG. 7 is an explanatory diagram of a method of recognizing an animal body reflected on a curved mirror using deep learning. In FIGS. 7A and 7B, an example is shown in which the person 60 is reflected in the curve mirror 51, that is, the person 60 is present in the curve mirror 51. The same elements as in FIG. 4 are designated by the same reference numerals, and detailed description thereof will be omitted.

The recognition unit 13 changes the position and size of the region of the curve mirror 51 detected by the detection unit 12 in the image 50 shown in FIG. 7 (a), for example, as shown in FIG. 7 (b). , Performs recognition processing to recognize the person 60.

In addition, in order for the recognition unit 13 to perform the above recognition process, it is sufficient that the recognition unit 13 is previously learned to recognize an animal body, that is, a person, a bicycle, a car, and a non-animal body. As a result, the recognition unit 13 recognizes that the animal body is reflected in the curve mirror if the reliability of the thing recognized as existing in the curve mirror exceeds the threshold value and the maximum size is found. Can be done. Further, when the recognition unit 13 recognizes that the animal body is reflected on the curved mirror, the recognition unit 13 outputs the position and size. Here, the reliability indicates a value indicating a shape or contour-likeness indicating an animal body of a person, a bicycle, or an automobile. On the other hand, the recognition unit 13 recognizes that the animal body is not reflected in the curve mirror unless the reliability of what is recognized to exist in the curve mirror exceeds the threshold value.

[Information processing device 10]
Next, the information processing apparatus 10 according to the first embodiment will be described.

The information processing apparatus 10 in the first embodiment outputs driving support information of the vehicle by using the curve mirror. In the present embodiment, as shown in FIG. 1, the information processing apparatus 10 includes a calculation unit 101, an output processing unit 102, and a blind spot determination unit 103. Hereinafter, details of these configurations and the like will be described.

<Calculation unit 101>
The calculation unit 101 calculates the difference between the size of the curve mirror detected by the detection processing unit 12a and the reference size, and calculates the size of the object using the calculated difference. Here, the difference includes the magnification of the curved mirror with respect to the reference size. The size of the object is calculated by enlarging or reducing the object reflected in the curved mirror according to the difference.

FIG. 8 is a diagram showing an example of the functional configuration of the calculation unit 101 in the first embodiment.

As shown in FIG. 8, the calculation unit 101 in the first embodiment includes a difference calculation unit 1011 and an object size calculation unit 1012.

The difference calculation unit 1011 calculates the difference between the size of the curve mirror detected by the detection unit 12 and the reference size. More specifically, the difference calculation unit 1011 is a curve mirror detected by the detection unit 12, and is a curve mirror reflected in an image obtained by continuously photographing the front of the moving vehicle in chronological order. Calculate the difference between the size and the standard size. In the present embodiment, the difference calculation unit 1011 calculates, for example, the magnification with respect to the reference size of the curve mirror reflected in the image obtained by photographing as the difference. The reference size is a size indicated by a predetermined height and width of the curved mirror.

The reference size may be the size of the largest curved mirror among the curved mirrors reflected in a plurality of images captured in a predetermined time, or the size of the curved mirror reflected in the image one frame before the captured image. Further, the difference calculation unit 1011 uses the amount of change in the size of the object obtained by enlarging or reducing the object in the curve mirror according to the magnification calculated in at least two images continuous in time series as the above difference. It may be calculated.

Further, the difference calculation unit 1011 may calculate the difference from one image. Specifically, the reference size is a value stored in advance, and the difference calculation unit 1011 calculates the above-mentioned magnification based on the size of the curve mirror reflected in one image and the stored reference size. The reference size is associated with the distance to the subject and may be corrected according to the distance between the vehicle and the curved mirror reflected in the image.

The object size calculation unit 1012 calculates the size of the object using the difference calculated by the difference calculation unit 1011. More specifically, the object size calculation unit 1012 calculates the size of the object using the object reflected by the curve mirror recognized by the recognition unit 13 and the difference calculated by the difference calculation unit 1011. In the present embodiment, the object size calculation unit 1012 enlarges or reduces the object reflected in the curve mirror recognized by the recognition unit 13 as the size of the object reflected in the curve mirror according to the difference calculated by the difference calculation unit 1011. The size of the object is calculated by letting it.

Here, an example of the calculation process performed by the calculation unit 101 configured as described above will be described with reference to the drawings.

FIG. 9 is an explanatory diagram showing an example of the calculation process performed by the calculation unit 101 in the first embodiment. The same elements as in FIG. 4 are designated by the same reference numerals, and detailed description thereof will be omitted.

The images 50 _n , the image 50 _{n + 1} , and the image 50 _{n + 2} shown in FIG. 9A are taken in front of the vehicle in chronological order at time t _n , time t _{n + 1} , and time t _{n + 2} . Is. The image 50 _n , the image 50 _{n + 1} , and the image 50 _{n + 2} , respectively, include a curve mirror 51, a curve mirror 51 _{n + 1} , and a curve mirror 51 _{n + 2} , and the curve mirror 51, the curve mirror 51 _{n + 1} , and the curve mirror 51 _{n + 2} include. People 60, people 60 _{n + 1} , and people 60 _{n + 2} are shown. The sizes of the curved mirror 51, the curved mirror 51 _{n + 1} , and the curved mirror 51 _{n + 2} are, for example, (w _1a , h _1a ), (w _1b , h _1b ) and (w1 _c , h _1c ), and the width and height. It is expressed by. The sizes of the curve mirror 51, the curve mirror 51 _{n + 1} , and the curve mirror 51 _{n + 2} are detected by the detection unit 12. FIG. 9B shows a curve mirror 51', a curve mirror 51'n _{+ 1} , and a curve mirror 51'n _{+ 2} enlarged to a reference size.

In this case, when the calculation unit 101, that is, the difference calculation unit 1011 enlarges or reduces the curve mirror 51, the curve mirror 51 _{n + 1} , and the curve mirror 51 _{n + 2} reflected in the image 50 _n , the image 50 _{n + 1} , and the image 50 _{n + 2} , to the reference size. Calculate the magnification. When the reference size is, for example, ( _ws , h _s ), the difference calculation unit 1011 sets the magnification when the curve mirrors 51, 51 _{n + 1} and 51 _{n + 2} are enlarged or reduced to the reference size, w _s / w _1a , w. Calculated as _s / w _1b and w _s / w _1c . It may be calculated as h _s / h _1a , h _s / h _1b , h _s / h _1c .

Then, as shown in FIG. 9B, the calculation unit 101, that is, the object size calculation unit 1012, has the magnifications w _s / w _1a , w _s / w _1b , w _s / w calculated by the difference calculation unit 1011. The curve mirror 51, the curve mirror 51 _{n + 1} and the curve mirror 51 _{n + 2} , and the person 60', the person 60'n _{+ 1 and the person 60'n + 2 when the person 60, the person 60 n + 1 and the} person 60 _{n + 2} are enlarged according to _1b . Just calculate the size.

<Blind spot determination unit 103>
The blind spot determination unit 103 is a detection processing unit for an object detected (recognized) by the detection processing unit 12a in at least one image of a plurality of consecutive images in time series in an image after at least one image in time series. If it is not detected (recognized) by 12a, the image after that is determined to be a blind spot image in which the object can exist in the blind spot of the curved mirror. In other words, the blind spot determination unit 103 determines whether or not the animal body that was visible in the curved mirror can no longer be seen in the images that are continuous in time series, and determines that the invisible image is a blind spot image.

Here, an example of the blind spot determination process performed by the blind spot determination unit 103 configured as described above will be described with reference to the drawings.

FIG. 10 is an explanatory diagram showing an example of a blind spot determination process performed by the blind spot determination unit 103 in the first embodiment. The same elements as in FIG. 4 are designated by the same reference numerals, and detailed description thereof will be omitted.

The images 50 _m , 50 m _{+ 1} and 50 _{m + 2} shown in FIG. 10 are taken in front of the moving vehicle at time tm + 2, time tm + 2 and time tm _{+ 2} , which are continuous in time series. Further, the curved mirror 51 _m , the curved mirror 51 _{m + 1} and the curved mirror 51 _{m + 2} are reflected in the image 50 _m , the image 50 _{m + 1} and the image 50 _{m + 2} , respectively, and the curved mirror 51 _m , the curved mirror 51 m + ₁ and the curved mirror 51 are reflected. In _{m + 2} , a person 60 _m and a person 60 _{m + 1} are further reflected.

In this case, the person 60 _m and the person 60 _{m + 1} recognized by the recognition unit 13 are reflected in the curve mirror 51 _m and the curve mirror 51 _{m + 1} of the images 50 _m and the image 50 _{m + 1} which are continuous in time series. On the other hand, no person is reflected in the curve mirror 51 _{m + 2} of the image 50 _{m + 2} after the image 50 _{m + 1} , and no person is recognized by the recognition unit 13. Therefore, the blind spot determination unit 103 determines that the image 50 _{m + 2} is a blind spot image. The reason why the blind spot determination unit 103 can determine the blind spot image in this way will be described with reference to FIG.

FIG. 11 is an explanatory diagram of a blind spot of a curved mirror.

The curved mirror also has an angle of view, and there is a blind spot, that is, a blind spot area. For example, FIG. 11 shows a case where the animal body 61 _{m + 1} , which is a person riding a bicycle, moves to the position of the animal body 61 _{m + 2} and enters the blind spot area of the curve mirror 72 while the vehicle 2 is temporarily stopped at the intersection. An example is shown. In this case, the driver of the temporarily stopped vehicle 2 can confirm the animal body 61 _{m + 1} reflected on the curve mirror 71, but cannot confirm the animal body 61 _{m + 2} on the curve mirror 72. However, even if the animal body 61 m + 2 is not reflected on the curve mirror 72, the animal body 61 _{m + 2} exists. Therefore, the blind spot determination unit 103 determines that the image 50 _{m + 2} is a blind spot image in which a human animal can exist in the blind spot of the curve mirror 51 _{m + 2} by determining as described above.

When the blind spot determination unit 103 determines the blind spot image, the output processing unit 102 generates and outputs vehicle control information for suspending the vehicle until a person can be seen directly from the vehicle or in the curve mirror. .. Further, if the blind spot determination unit 103 determines the blind spot image and then the person cannot be seen even after a certain period of time, it may be determined that the time-out has occurred and the blind spot determination process may be performed from the beginning. If a person cannot be seen in the curved mirror even after a certain period of time, an animal body such as a person existing in the blind spot area of the curved mirror will enter a house or a store, and wait until a certain time. This is because it is possible that people cannot be seen in the curved mirror.

Further, the blind spot determination unit 103 detects (recognizes) an object having a different size in at least two images consecutive in time series among the plurality of images by the detection processing unit 12a, and later in time series of at least two images. When the object is not detected (recognized) by the detection processing unit 12a in the image of the above, the image after the object may be determined as the blind spot image. For example, to explain using FIG. 10, the size of the person 60 _{m + 1} in the curved mirror reflected in the image 50 _{m + 1} is larger than the size of the person 60 _m in the curved mirror 51 _m reflected in the previous image 50 _m in chronological order. It has become. This means that person 60 is approaching the intersection. A person is not reflected in the curve mirror 51 _{m + 2} of the image 50 _{m + 2} which is later in the time series of the image 50 _{m + 1} . This means that person 60 has entered a blind spot near the intersection. Therefore, the blind spot determination unit 103 compares the size of the person 60 _{m + 1} in the curved mirror reflected in the image 50 _{m + 1} with the size of the person 60 _m in the curved mirror 51 _m reflected in the previous image 50 _m in chronological order. Based on the presence or absence of the person 60 in the image 50 _m + 2, it can be determined that the image 50 _{m + 2} is a blind spot image. As a result, when the person 60 moves away from the intersection and enters the blind spot, the image in which the person 60 is not reflected on the curve mirror is not determined as the blind spot image, so that the vehicle is stopped even though it is safe. Can be suppressed.

<Output processing unit 102>
The output processing unit 102 generates vehicle driving support information based on the size of the object calculated by the calculation unit 101, and outputs the generated driving support information. The driving support information is generated and output according to the change information of the size of each object corresponding to each of at least two images consecutive in time series. Here, the driving assistance information includes at least one of information for decelerating the vehicle and information for moving the vehicle away from the object when the change information indicates an increase in size. In addition, the driving support information includes information for driving the vehicle after the object moves away from the vehicle when the change information indicates an increase in size. Further, the driving support information may be the control command information of the behavior of the vehicle or the presentation information presented to the occupants of the vehicle.

FIG. 12 is a diagram showing an example of the functional configuration of the output processing unit 102 in the first embodiment.

In the present embodiment, as shown in FIG. 12, the output processing unit 102 includes a danger prediction unit 1031, an information generation unit 1032, and an information output unit 1033.

The danger prediction unit 1031 predicts the danger predicted when the vehicle travels as it is, based on the size of the object calculated by the calculation unit 101. The risk may be a degree of risk or a risk content (eg, contact, collision, entrainment, etc.). The danger prediction unit 1031 determines the degree of danger according to the amount of change in the size of the object corresponding to at least two consecutive images in time series. Here, as an example, it will be described that the degree of danger can be predicted according to the amount of change in the size of the object.

FIG. 13 is a conceptual diagram showing the amount of change in the size of the object in FIG. 9B. The same elements as in FIG. 9B are designated by the same reference numerals, and detailed description thereof will be omitted. FIG. 13 conceptually shows the human 60', the human 60'n _{+ 1} , and the human 60'n _{+ 2} , which are the objects reflected in the curved mirror enlarged or reduced to the reference size, which are calculated by the object size calculation unit 1012. There is.

As shown in FIG. 13, the danger prediction unit 1031 has the size of the person 60', the person 60'n _{+ 1} and the person 60'n _{+ 2} increasing in order, so that the person is approaching the curve mirror 51, that is, the intersection. You can see that it is approaching. Then, the danger prediction unit 1031 knows that when the amount of change in the size of the person 60', the person 60'n _{+ 1} and the person 60'n _{+ 2} is large, the speed and the speed change in which the person is approaching the curve mirror 51 are large. , It can be judged that the degree of danger is high. That is, when the amount of change in the sizes of the person 60', the person 60'n _{+ 1} and the person 60'n _{+ 2} is larger than the threshold value, the person is rapidly approaching the intersection where the curve mirror 51 is located, and the vehicle equipped with the information processing device 10 is installed. It can be judged that there is a high possibility of collision or contact with. In this way, the danger prediction unit 1031 can predict the degree of danger according to the amount of change in the size of the object.

The information generation unit 1032 generates vehicle driving support information based on the size of the object calculated by the calculation unit 101. The information generation unit 1032 may generate vehicle driving support information according to the size of the object. For example, the information generation unit 1032 may generate driving support information according to the change information of the size of each object corresponding to each of at least two images continuous in time series. Further, the information generation unit 1032 may generate driving support information of the vehicle based on the state of the vehicle in addition to the size of the object.

Further, the information generation unit 1032 may generate driving support information according to the danger predicted from the change information in the situation where the vehicle travels in the direction toward the curve mirror. The information generation unit 1032 may generate vehicle control information based on the danger predicted by the danger prediction unit 1031.

The information generation unit 1032 may generate information indicating the degree of danger predicted by the danger prediction unit 1031 as driving support information. For example, the information generation unit 1032 generates information indicating the degree of danger determined by the danger prediction unit 1031 according to the amount of change in the size of an object corresponding to at least two consecutive images in time series as driving support information. You may.

Further, the information generation unit 1032 may generate vehicle control information for suspending the vehicle based on the determination result of the blind spot image by the blind spot determination unit 103. Specifically, in the information generation unit 1032, an object detected (recognized) by the detection processing unit 12a in at least one image among a plurality of images consecutive in time series is time-series than at least one image. If it is not detected (recognized) by the detection processing unit 12a in the later image, driving support information may be generated. Further, the information generation unit 1032 detects (recognizes) an object having a different size in at least two images consecutive in time series among a plurality of images by the detection processing unit 12a, and later in time series of at least two images. When the object is not detected (recognized) by the detection processing unit 12a in the image of the above, driving support information may be generated.

The information output unit 1033 outputs the driving support information generated by the information generation unit 1032.

Hereinafter, an example of the output processing performed by the output processing unit 102 configured as described above will be described with reference to the drawings.

FIG. 14 is a diagram showing an example of a curved curved road in the first embodiment. FIG. 15 is an explanatory diagram showing an example of output processing performed by the output processing unit 102 in the first embodiment.

The curved curved road shown in FIG. 14 is an example of a place with poor visibility. FIG. 14 shows a vehicle 2 equipped with the information processing device 10 or the system 1 of the present embodiment traveling on a curved curved road. Further, FIG. 14 shows a vehicle 3 as an example of an object, and shows a case where the vehicle 3 reflected in the curve mirror 73 can be confirmed from the vehicle 2.

In this case, the output processing unit 102, that is, the danger prediction unit 1031 determines the degree of danger according to the size change of the object reflected in the curved mirror that has been enlarged or reduced to the reference size, that is, normalized by the calculation unit 101.

More specifically, as shown in FIG. 15, the danger prediction unit 1031 determines that the degree of danger is low when the size change of the object reflected in the normalized curve mirror becomes small or there is no size change. .. Here, the output processing unit 102, that is, the information generation unit 1032 may generate information indicating that the degree of danger determined by the danger prediction unit 1031 is low. Further, as shown in FIG. 15, the information generation unit 1032 may generate vehicle control information indicating that there is no change in control such as the speed of the vehicle 2 based on the low degree of danger. In the example shown in FIG. 14, the danger prediction unit 1031 is moving or stopping in the direction away from the vehicle 2 when the size change of the vehicle 3 in the normalized curve mirror 73 becomes small or there is no size change. Since it can be determined that the vehicle is inside, it is determined that the risk is low. Then, the information generation unit 1032 may generate information indicating that the determined risk level is low or vehicle control information indicating that there is no change in control such as the speed of the vehicle 2.

Further, as shown in FIG. 15, the danger prediction unit 1031 determines that the degree of danger is medium when the size of the object reflected in the normalized curve mirror is gradually increasing. Here, the information generation unit 1032 may output information indicating that the determined risk level is medium, or as shown in FIG. 15, the vehicle is set based on the fact that the risk level is medium. Vehicle control information for decelerating may be generated. In the example shown in FIG. 14, the danger prediction unit 1031 advances the vehicle 3 in the direction approaching the vehicle 2 when the size change of the vehicle 3 in the normalized curve mirror 73 is larger than the threshold value a and gradually increases. Since it can be determined that it is inside, it is determined that the degree of danger is medium. Then, the information generation unit 1032 may generate information indicating that the determined danger level is medium or vehicle control information for decelerating the vehicle 2.

Further, as shown in FIG. 15, the danger prediction unit 1031 determines that the degree of danger is high when the size of the object reflected in the normalized curve mirror suddenly increases. Here, the information generation unit 1032 may generate information indicating that the determined risk level is high. Further, as shown in FIG. 15, the information generation unit 1032 may generate vehicle control information for decelerating and moving the vehicle in a direction away from the object based on the high degree of danger. In the example shown in FIG. 14, when the size change of the vehicle 3 in the normalized curve mirror 73 is larger than the threshold value b, the vehicle 3 is larger in the direction approaching the vehicle 2. Since it can be determined that the vehicle is in progress at a speed, it is determined that the risk is high. Then, the information generation unit 1032 decelerates the information indicating that the determined danger level is high or the vehicle 2 so that the vehicle 3 is in a position where it does not collide even if it goes out of the lane of the curved curved road. Vehicle control information for moving in a direction away from 3 may be generated.

Note that FIGS. 14 and 15 have described the case where the vehicle 2 is traveling on a curved curved road, that is, traveling, but the present invention is not limited to this. The same can be said when the vehicle 2 travels at an intersection where roads intersect.

FIG. 16 is a diagram showing an example of a T-shaped intersection in the first embodiment. FIG. 17 is an explanatory diagram showing another example of the output processing performed by the output processing unit 102 in the first embodiment.

The T-shaped intersection shown in FIG. 16 is an example of a place with poor visibility. FIG. 16 shows a vehicle 2 equipped with the information processing device 10 or the system 1 of the present embodiment traveling at a T-shaped intersection. Further, FIG. 16 shows a vehicle 3 as an example of an object, and shows a case where the vehicle 3 reflected in the curve mirror 74 can be confirmed from the vehicle 2.

In this case, as shown in FIG. 17, the output processing unit 102, that is, the danger prediction unit 1031 has a low degree of danger when the size change of the object reflected in the normalized curve mirror becomes small or there is no size change. Is determined. Here, the output processing unit 102, that is, the information generation unit 1032 may generate information indicating that the degree of danger determined by the danger prediction unit 1031 is low. Further, as shown in FIG. 17, the information generation unit 1032 may generate vehicle control information for temporarily stopping and then starting the vehicle 2 based on the low degree of danger. In the example shown in FIG. 16, the danger prediction unit 1031 is moving or stopping in the direction away from the vehicle 2 when the size change of the vehicle 3 reflected in the normalized curve mirror 74 becomes small or there is no size change. Since it can be determined that the vehicle is inside, it is determined that the risk is low. Then, the information generation unit 1032 may generate information indicating that the determined risk level is low or vehicle control information for suspending the vehicle 2 and then starting the vehicle 2.

Further, as shown in FIG. 17, the danger prediction unit 1031 determines that the degree of danger is high when the size change of the object reflected in the normalized curve mirror becomes large. Here, the information generation unit 1032 may generate information indicating that the degree of danger determined by the danger prediction unit 1031 is high. Further, as shown in FIG. 17, the information generation unit 1032 temporarily stops the vehicle 2 based on the high degree of danger, and starts after confirming the passage of the target vehicle 2 with a sensor or the like. You may generate vehicle control information for the vehicle. In the example shown in FIG. 16, the danger prediction unit 1031 can determine that the vehicle 3 is moving toward the vehicle 2 when the size change of the vehicle 3 in the normalized curve mirror 74 becomes large. , Judge that the degree of danger is high. Then, the information generation unit 1032 generates information indicating that the determined danger level is high or vehicle control information for suspending the vehicle 2 and confirming the passage of the vehicle 2 with a sensor or the like before starting the vehicle. May be good.

In the above, the example in which the size change information is the size change amount has been described, but the size change information may be information indicating the size change content. For example, the content of the change in size may be that the size has increased (or decreased), the size has become larger than the predetermined size (or smaller than the predetermined size), and the like.

Further, in the above, the example in which the driving support information is the control command information has been described, but the driving support information may be the presented information. For example, the presented information may be information indicating a danger described later or information indicating a recommended operation for the driver.

[Operation of system 1]
Next, the information processing method of the system 1 configured as described above will be described. FIG. 18 is a flowchart showing an outline of the information processing method of the system 1 in the first embodiment. FIG. 19 is a flowchart showing details of the information processing method of the system 1 in the first embodiment. The same elements are designated by the same reference numerals in FIGS. 18 and 19, and detailed description thereof will be omitted.

First, as shown in FIG. 18, the system 1 detects the curved mirror reflected in the image and the object reflected in the curved mirror based on the image information (S10). Next, the system 1 calculates the difference between the size of the curved mirror reflected in the image and the reference size, and calculates the size of the object reflected in the curved mirror using the difference (S11). Next, the system 1 generates vehicle driving support information based on the size of the object calculated in S11 (S12). Then, the system 1 outputs the driving support information generated in S12 (S13).

More specifically, as shown in FIG. 19, first, the system 1 acquires image information indicating an image obtained by imaging of an image pickup device mounted on a vehicle in progress by the image acquisition unit 11. (S101).

Next, the system 1 performs the process of S10 described with reference to FIG. More specifically, the system 1 performs a detection process in S10 to detect the curved mirror reflected in the image obtained in S101 (S102). Next, the system 1 performs a recognition process of recognizing (detecting) an object in the curved mirror reflected in the image obtained in S101 (S103).

Next, the system 1 performs the process of S11 described with reference to FIG. More specifically, in S11, the system 1 performs a difference calculation process for calculating the difference between the size of the curve mirror detected in S102 and the reference size (S104). Next, the system 1 performs an object size calculation process for calculating the size of the object in the curved mirror recognized in S103 using the difference calculated in S104 (S105). Since the details of the processes performed in S104 and S105 are as described above, detailed description thereof will be omitted here.

Next, the system 1 determines that the object recognized by S103 in two images consecutive in time series out of the three images consecutive in time series is not recognized by S103 in the image after the two images. A blind spot determination process for determining the subsequent image as a blind spot image in which the object can exist in the blind spot of the curved mirror is performed (S106).

Here, the system 1 determines whether the object has appeared in S103 after determining the blind spot image in S106, or whether a certain time has elapsed after the determination (S106). When the object appears in S103 after determining the blind spot image in S106, or when the time-out, that is, when a certain period of time elapses after determining the blind spot image in S106, the system 1 cannot recognize the appearance of the object in S103 (Yes in S107). ), Proceed to the next process (S12). In other cases (No in S107), the determination process of S107 is repeated.

Next, the system 1 performs the process of S12 described with reference to FIG. More specifically, in S12, the system 1 performs a risk prediction process for predicting the degree of danger determined when the vehicle travels as it is, based on the size of the object calculated in S105 (S108). Next, the system 1 generates information indicating the degree of danger determined in S108 as driving support information, or generates vehicle control information for controlling the vehicle based on the degree of danger determined in S108. Information generation processing is performed (S109). Since the details of the processes performed in S108 and S109 are as described above, the description thereof will be omitted here.

Then, the system 1 performs the process of S13 described with reference to FIG. More specifically, in S13, the system 1 performs information output processing for outputting the driving support information and the like generated in S109 (S110).

The processing order of S103, S104, and S105 is not limited to the case shown in FIG. S103 may be performed after S104, and S105 may be performed. Further, the processing of S106 and S107 may be performed before the calculation processing of S11.

[Effects of Embodiment 1 and the like]
As described above, according to the information processing apparatus 10 or the system 1 according to the first embodiment, the risk level can be determined by using the curve mirror in a place where the visibility is poor such as an intersection, and the determined risk level can be determined. It is possible to generate and output vehicle driving support information based on the above. This makes it possible to support safe driving of the vehicle equipped with the information processing device 10.

Specifically, if the amount of movement of an object such as a person reflected in a curved mirror is large at an intersection or the like where visibility is poor, there is a high possibility that the object will jump out to the place. In this case, it is necessary to decelerate the vehicle at an early stage to avoid danger.

Therefore, in the present embodiment, since the size of the curved mirror reflected in the image obtained from the moving vehicle changes, normalization is performed to align the size of the curved mirror with a predetermined reference size. As a result, the amount of movement of an object such as a person in the curve mirror can be converted into the size of the object in the curve mirror, that is, the size, so that the degree of danger can be determined according to the amount of change in size. In this way, the information processing apparatus 10 according to the first embodiment can determine the degree of danger by using the curve mirror.

For example, when the vehicle during automatic driving is equipped with the information processing device 10, the vehicle can determine the degree of danger by using the curve mirror as described above, so that the degree of danger can be determined according to the determined degree of danger. Vehicle control can be performed.

Further, as described above, driving support information may be generated according to the change information of the size of each object corresponding to each of at least two images continuous in time series, and the change information may indicate an increase in size. .. In this case, the driving assistance information may include at least one of information for decelerating the vehicle and information for moving the vehicle away from the object, or for driving the vehicle after the object moves away from the object. Information may be included. This is because it is possible to ensure the safety of objects and vehicles approaching the intersection.

Further, the driving support information may be generated based on the state of the vehicle in addition to the size of the object. This is because it is possible to control without waste according to the state of the vehicle (running state, stopped state, etc.).

Further, the driving support information may be generated according to the size of the object. This is because the danger can be predicted without processing using the size change information.

As described above, according to the information processing apparatus 10 or the system 1 according to the first embodiment, it is possible to support the safe driving of the vehicle by using the curved mirror in a place where the visibility is poor such as an intersection.

(Modification 1)
In the first embodiment, the case where the degree of danger is determined according to the amount of change in the size of the object in the normalized curved mirror has been described, but the present invention is not limited to this. The recognition unit 13 may recognize the attribute of the object reflected on the curve mirror, and the calculation unit 101 may determine the degree of danger in consideration of the attribute. In this modification, a case where the degree of danger is determined by further considering the attribute related to the moving speed of the object reflected in the curved mirror will be described.

20 and 21 are explanatory views showing an example of the risk prediction process performed by the output processing unit 102 in the modification 1 of the first embodiment. The same wording is described in the same contents as those in FIGS. 15 and 17, and detailed description thereof will be omitted.

FIG. 20 shows an example of the risk prediction process performed by the output processing unit 102 in the modified example 1 on a curved curved road, and FIG. 21 shows the risk prediction performed by the output processing unit 102 in the modified example 1 at a T-shaped intersection. An example of the process is shown. The vehicle control information based on the low, medium, or high risk level may be as shown in FIGS. 15 and 17, and is not shown in FIGS. 20 and 21.

When the object is a person as shown in FIGS. 20 and 21, the danger prediction unit 1031 determines the degree of danger in the same manner as in FIGS. 15 and 17. On the other hand, when the object is a bicycle, a motorcycle, or an automobile whose moving speed is faster than that of a person, the danger prediction unit 1031 determines a higher degree of danger according to the moving speed than when the object is a person.

As described above, in this modification, the output processing unit 102 generates driving support information according to the attributes of the object recognized by the recognition unit 13, and outputs the generated driving support information.

(Modification 2)
In the first modification, the case where the degree of danger is determined by further considering the attribute related to the moving speed of the object reflected in the curved mirror has been described, but the present invention is not limited to this. When the object reflected in the curve mirror is a person, the danger prediction unit 1031 may determine the degree of danger by further considering the attribute related to the age of the person. This case will be described below as Modification 2.

22 and 23 are explanatory views showing an example of the risk prediction process performed by the output processing unit 102 in the modification 2 of the first embodiment. The same wording is described in the same contents as those in FIGS. 15 and 17, and detailed description thereof will be omitted.

FIG. 22 shows an example of the risk prediction process performed by the output processing unit 102 in the modified example 2 on a curved curved road, and FIG. 23 shows the risk prediction performed by the output processing unit 102 in the modified example 1 at the T-shaped intersection. An example of the process is shown. Since the vehicle control information based on the low, medium, or high risk level may be as shown in FIGS. 15 and 17, the illustration is omitted in FIGS. 22 and 23.

As shown in FIGS. 22 and 23, when the object is a person and the person is a child or an elderly person, the danger prediction unit 1031 may determine the same degree of danger as in FIGS. 15 and 17. .. On the other hand, if the object is a person and the person is something other than a child or the elderly, the risk may be determined to be higher than when the person is a child or the elderly.

As described above, in this modification, when the attribute of the object recognized by the recognition unit 13 is a person, the output processing unit 102 has driving support information according to the information regarding the age of the object recognized by the recognition unit 13. Is changed and output. In the above, the example in which the information regarding the age is the generation of the person has been described, but the information regarding the age may be the age or the age of the person.

(Modification 3)
In the second modification, when the object reflected in the curved mirror is a person, the case where the degree of danger is determined by further considering the attribute related to the age of the person has been described, but the present invention is not limited to this. When the object reflected in the curve mirror is a person, the danger prediction unit 1031 may further consider the attribute of whether or not the person is inadvertently acting to determine the degree of danger. Specifically, inattention behavior is non-forward vision behavior. For example, non-forward vision behavior includes gaze at a mobile terminal such as a smartphone or a book. This case will be described below as Modification 3.

24 and 25 are explanatory views showing an example of the risk prediction process performed by the output processing unit 102 in the modification 3 of the first embodiment. The same wording is described in the same contents as those in FIGS. 15 and 17, and detailed description thereof will be omitted.

FIG. 24 shows an example of the risk prediction process performed by the output processing unit 102 in the modified example 3 on a curved curved road, and FIG. 25 shows the risk prediction performed by the output processing unit 102 in the modified example 1 at the T-shaped intersection. An example of the process is shown. Since the vehicle control information based on the low, medium, or high risk level may be as shown in FIGS. 15 and 17, the illustration is omitted in FIGS. 24 and 25.

As shown in FIGS. 24 and 25, when the object is a person and the person does not have a non-forward-looking behavior of looking at the mobile terminal and performing a moving action, that is, the person is not walking while looking at the mobile terminal. In the meantime, the danger prediction unit 1031 may determine the same degree of danger as in FIGS. 15 and 17. On the other hand, when the object is a person and the person is non-forward-looking, that is, the person is walking while looking at the mobile terminal, the danger prediction unit 1031 indicates that the person is non-forward-looking. It suffices to determine a higher degree of risk than when not acting. In the above, an example in which the careless behavior is a non-forward vision behavior has been described, but the careless behavior is a behavior in front of the person but looking upward or downward, or a baby carriage or a ball in front of the person. It may be an action of gazing at a specific object.

As described above, in this modification, when the attribute of the object recognized by the recognition unit 13 is a person, the output processing unit 102 determines whether or not the object recognized by the recognition unit 13 is inadvertently acting. The driving support information is generated and output accordingly.

(Embodiment 2)
In the first embodiment, an information processing device or the like that can support safe driving of an ongoing vehicle by using a curved mirror in a place where visibility is poor such as an intersection has been described, but the present invention is not limited to this. Even in a temporarily stopped vehicle, the curved mirror can be used to support safe driving. Hereinafter, this case will be described as the second embodiment.

[System 1A configuration]
FIG. 26 is a block diagram showing an example of the configuration of the system 1A according to the second embodiment. The same elements as those in FIG. 1 are designated by the same reference numerals, and detailed description thereof will be omitted.

The system 1A shown in FIG. 26 has a different configuration of the information processing apparatus 10A as compared with the system 1 in the first embodiment. Similar to the system 1, the system 1A is also mounted on a vehicle such as an automobile, and by using a curve mirror, it is possible to output driving support information of the vehicle.

In the present embodiment, in order to support the safe driving of the temporarily stopped vehicle, the information processing apparatus 10A also takes into account the position of the object in the curved mirror. The image acquisition unit 11 and the detection processing unit 12a may be configured in the information processing device 10A as in the first embodiment.

[Information processing device 10A]
The information processing apparatus 10A according to the second embodiment outputs driving support information of the temporarily stopped vehicle by using the curve mirror. In the present embodiment, in order to support the safe driving of the temporarily stopped vehicle, the information processing apparatus 10A also takes into account the position of the object in the curved mirror. The reason for this will be described with reference to FIGS. 27A to 27C. FIG. 27A is a diagram showing an example of an object 63 reflected in the curved mirror 75 visually recognized by the driver of the vehicle 2a temporarily stopped at the intersection. 27B and 27C are explanatory views of the characteristics of the curved mirror.

For example, in FIG. 27A, when the driver of the vehicle 2a temporarily stopped at an intersection visually recognizes the curved mirror 75 installed at the intersection, the person riding the bicycle can be confirmed as the object 63 reflected on the curved mirror 75. An example of is shown. In FIG. 27A, since the object 63, which is a person, is reflected in the back of the curved mirror 75, that is, in the upper part of the curved mirror, the driver of the vehicle 2a sees the object 63 from the vehicle 2a as shown in FIG. 27B. It seems to be located at the back of the road. However, due to the characteristics of the curved mirror in which the left and right sides are reversed, the object 63 actually exists at the position shown in FIG. 27B, that is, at the position on the front side of the road when viewed from the vehicle 2a.

Therefore, the information processing apparatus 10A in the present embodiment outputs the driving support information of the vehicle 2a after taking into consideration the position of the object in the curve mirror.

More specifically, as shown in FIG. 26, the information processing apparatus 10A according to the present embodiment includes a calculation unit 101A, an output processing unit 102A, and a blind spot determination unit 103. The information processing device 10A shown in FIG. 26 has a different configuration between the calculation unit 101A and the output processing unit 102A as compared with the information processing device 10 in the first embodiment. Hereinafter, details such as the configuration of the calculation unit 101A and the output processing unit 102A will be described.

<Calculation unit 101A>
FIG. 28 is a diagram showing an example of the functional configuration of the calculation unit 101A in the second embodiment. FIG. 29A is a diagram showing an example of the position of the object calculated by the object position calculation unit 1013 shown in FIG. 28. FIG. 29B is a diagram showing an example of a road surface area calculated by the road surface area calculation unit 1015 shown in FIG. 28. FIG. 30 is an explanatory diagram of a method of calculating the central axis calculated by the central axis calculation unit 1016 shown in FIG. 28.

The calculation unit 101A in the second embodiment calculates the position of the object on the curve mirror detected by the detection processing unit 12a. More specifically, the calculation unit 101A is an object that is a curve mirror detected by the detection unit 12 and is recognized by the recognition unit 13 in the curve mirror reflected in the image obtained by the image pickup of the image pickup device mounted on the vehicle. Calculate the position of the curve mirror.

For example, as shown in FIG. 28, the calculation unit 101A includes an object position calculation unit 1013, an object size calculation unit 1014, a road surface area calculation unit 1015, and a central axis calculation unit 1016. The calculation unit 101A may have only the object position calculation unit 1013, or may have only the object position calculation unit 1013 and the object size calculation unit 1014.

The object position calculation unit 1013 calculates the position of the object on the curve mirror. More specifically, the object position calculation unit 1013 is a curve mirror detected by the detection unit 12, and the curve reflected in the image obtained by continuously photographing the front of the temporarily stopped vehicle in chronological order. The position of the object recognized by the recognition unit 13 in the mirror on the curved mirror is calculated. For example, as shown in FIG. 29A, the object position calculation unit 1013 calculates the position of the object 63, which is a person riding a bicycle, in the region of the curve mirror 75. When the calculation unit 101A has only the object position calculation unit 1013, the object position calculation unit 1013 calculates whether the position of the object in the curve mirror is to the right or left of the threshold value indicating a predetermined position in the curve mirror. You may. Further, the object position calculation unit 1013 may calculate whether the position of the object in the curve mirror is above or below the threshold value indicating a predetermined position in the curve mirror.

The object size calculation unit 1014 calculates the size of the object with reference to the curved mirror. More specifically, the object size calculation unit 1014 is a curve mirror detected by the detection unit 12, and the curve reflected in the image obtained by continuously photographing the front of the temporarily stopped vehicle in chronological order. The size of the object recognized by the recognition unit 13 in the mirror is calculated.

The road surface area calculation unit 1015 calculates the road surface area in the curve mirror. More specifically, the road surface area calculation unit 1015 calculates a road surface area which is a region indicating a road reflected in the curve mirror detected by the detection unit 12. For example, the road surface area calculation unit 1015 calculates the road surface area 80 indicating the road existing in the area of the curve mirror 75, as shown in FIG. 29B. The road surface area calculation unit 1015 can calculate the road surface area as described above by learning in advance using a large number of images of a curved mirror reflecting a road by deep learning using a convolution layer or the like.

The central axis calculation unit 1016 calculates the central axis of the road surface area calculated by the road surface area calculation unit 1015. More specifically, as shown in FIG. 30, for example, the central axis calculation unit 1016 first, in the road surface area 80 calculated by the road surface area calculation unit 1015, first, first, a straight line corresponding to the road shoulder of the road surface (referred to as a road shoulder straight line). Is calculated using the Hough transform. Next, the central axis calculation unit 1016 passes through the position of the road shoulder straight line having the same distance d from the end point of the calculated road shoulder straight line, and calculates the line L1 and the line L2 orthogonal to the road shoulder straight line. Similarly, the lines L3 and L4 and the lines L5 and L6 orthogonal to the road shoulder straight line are calculated by passing through the positions of the road shoulder straight lines having the same distance from the end points of the calculated road shoulder straight lines. Then, the central axis calculation unit 1016 calculates the straight line connecting these intersections as the central axis 82 of the road surface region.

As described above, the central axis calculation unit 1016 has explained that the central axis 82 is calculated by calculating three sets of lines L1 and L2, lines L3 and L4, and lines L5 and L5. However, it is not limited to this. Since the central axis 82 is obtained from at least two intersections, the central axis calculation unit 1016 can also calculate from the calculated intersections of the lines L1 and L2 and the end points of the shoulder straight line. Further, the central axis calculation unit 1016 calculates four or more sets of lines, and uses the least squares method to calculate a straight line that minimizes the squared error between the intersection of the four or more sets of lines and the straight line as the central axis 82. May be good.

<Output processing unit 102A>
The output processing unit 102A generates driving support information based on the position of the object on the curve mirror calculated by the calculation unit 101A, and outputs the generated driving support information. Here, the driving support information may include control command information of the behavior of the vehicle, or may include presentation information presented to the occupants of the vehicle.

FIG. 31 is a diagram showing an example of the functional configuration of the output processing unit 102A according to the second embodiment. FIG. 32 is a diagram showing an example of the position of an object in the curved mirror in the second embodiment.

In the present embodiment, as shown in FIG. 31, the output processing unit 102A includes a danger prediction unit 1031A, an information generation unit 1032A, and an information output unit 1033A.

<< Danger Prediction Unit 1031A >>
The danger prediction unit 1031A predicts the danger based on the position of the object in the curve mirror calculated by the calculation unit 101A. For example, the danger prediction unit 1031A may predict the danger from the position of the object in the curve mirror in the situation where the vehicle travels in the direction toward the curve mirror, or the curve specified from the object and the road reflected in the image. The danger may be predicted according to the positional relationship with the area in the mirror.

In the present embodiment, the danger prediction unit 1031A predicts the danger based on the position calculated by the object position calculation unit 1013 and the central axis calculated by the central axis calculation unit 1016. Specifically, the danger may be a degree of danger or a content of danger (for example, contact, collision, entrainment, etc.).

When the position calculated by the object position calculation unit 1013 is to the right of the central axis calculated by the central axis calculation unit 1016, the danger prediction unit 1031A is higher than when the position is to the left of the central axis. Determine the degree of danger. For example, as shown in FIG. 32, if the lower end of the object 63 is located to the right or above the central axis 82 of the road surface region, it can be defined that the object 63 exists in the back in the region of the curved mirror 75. When the object 63 is located in the back in the area of the curve mirror 75, the object 63 is located at a position on the front side of the road when viewed from the vehicle, so that the object 63 collides with the object 63 when the temporarily stopped vehicle starts. There is a danger. Therefore, the danger prediction unit 1031A determines the degree of danger to be high when the object 63 exists in the region of the curve mirror 75. On the other hand, if the lower end of the object 63 is located to the left or below the central axis 82, it can be defined that the object 63 exists in front of the curved mirror region. When the object 63 is in front of the curved mirror 75, the object 63 is located at a position on the back side of the road when viewed from the vehicle, so that the object 63 collides with the object 63 even if the temporarily stopped vehicle starts. The risk of doing it is low. Therefore, the danger prediction unit 1031A can determine the degree of danger to be low when the object 63 is in front of the curved mirror 75.

Hereinafter, this case will be described more specifically with reference to the drawings.

33, 34, and 35 are explanatory views of the degree of danger when the object 64 is located behind the curved mirror 75 in the second embodiment. FIG. 33 shows a scene in which the object 64 is acquiring an image of the curved mirror 75 reflected on the back side from the temporarily stopped vehicle 2a. (A) of FIG. 34 shows the position and the size of the object 64 existing in the curved mirror 75, and (b) corresponds to the enlarged view of the curved mirror 75 of (a) and the object 64. And the relationship with the road are shown. FIG. 35 shows the positional relationship between the object 64 shown in FIG. 34, the vehicle 2a, and the curve mirror 75.

As shown in FIG. 35, it is assumed that the human object 64 is walking in the direction of the intersection on the front side of the road when viewed from the vehicle 2a. Then, when the system 1A mounted on the temporarily stopped vehicle 2a acquires the image 50 _j , the image 50 _{j + 1} , or the image 50 _{j + 2} at the time t _j , the time t _{j + 1} , or the time t _{j + 2} , as shown in FIG. 34. do. In this case, the system 1A, that is, the information processing apparatus 10A, calculates the position of the object 64 _j , the object 64 _{j + 1} , or the object 64 _{j + 2} in the curve mirror 75. In the image 50 _j , the image 50 _{j + 1} , or the image 50 _{j + 2} , the object 64 _j , the object 64 _{j + 1} or 64 _{j + 2} is located on the back side, so that the object 64 _j , 64 _{j + 1} or 64 _{j + 2} is on the front side of the road when viewed from the vehicle 2a. Is located. That is, since there is a risk of collision with the object 64 when the temporarily stopped vehicle 2a starts, the degree of danger is determined to be high.

The system 1A mounted on the temporarily stopped vehicle 2a may acquire images 50 _j , 50 _{j + 1} and 50 _{j + 2} and calculate the position and size of the object 64 _j , 64 _{j + 1} or 64 _{j + 2} . .. In this case, it can be seen that the object 64 approaches the vehicle 2a temporarily stopped at the intersection from the continuity that the size of the object 64 _j , 64 _{j + 1} or 64 _{j + 2} increases. Therefore, the degree of danger to the object 64 _{j + 2} may be determined to be higher than that of the object 64 _{j + 1} or the like. In this way, the danger prediction unit 1031A may determine the degree of danger according to the amount of change in the size of the object corresponding to at least two images continuous in time series. That is, when the object size calculation unit 1014 calculates the size of the object based on the curve mirror, the danger prediction unit 1031A calculates the position of the object in the curve mirror calculated by the object position calculation unit 1013 and the object size. The degree of danger may be predicted based on the size of the object calculated by the unit 1014.

Further, when the system 1A mounted on the temporarily stopped vehicle 2a acquires the image 50 _{j + 2} and the image 50 _{j + 3} , the object 64 does not exist in the curve mirror 75 of the image 50 _{j + 3} . However, since the object 64 _{j + 2} was present in the curve mirror 75 of the image 50 _{j + 2} at the time t _{j + 2} acquired immediately before the continuity, the image 50 _{j + 3} is a blind spot image in which the object 64 can exist in the blind spot of the curve mirror 75. It may be determined that the degree of danger with respect to the image 50 _{j + 3} is higher than that of the object 64 _{j + 2} or the like.

36, 37, and 38 are explanatory views of the degree of danger when the object 64 is located on the front side of the curved mirror 75 in the second embodiment. FIG. 36 shows a scene in which the object 64 is acquiring an image of the curved mirror 75 in which the object 64 is reflected on the front side from the temporarily stopped vehicle 2a. (A) of FIG. 37 shows the position and the size of the object 64 existing in the curved mirror 75, and (b) corresponds to the enlarged view of the curved mirror 75 of (a) and the object 64. And the relationship with the road are shown. FIG. 38 shows the positional relationship between the object 64 shown in FIG. 37, the vehicle 2a, and the curve mirror 75.

As shown in FIG. 38, it is assumed that the human object 64 is walking in the direction of the intersection on the back side of the road when viewed from the vehicle 2a. Then, when the system 1A mounted on the temporarily stopped vehicle 2a acquires the image 50 _k , the image 50 _{k + 1} , or the image 50 _{k + 2} at the time tk, the time tk _{+ 1} , or the time tk _{+ 2} , as shown in FIG. 37. do. In this case, the system 1A, that is, the information processing apparatus 10A, calculates the position of the object 64 _k , the object 64 _{k + 1} , or the object 64 _{k + 2} in the curve mirror 75. In the image 50 k, the image 50 _{k + 1} or the image 50 _{k + 2} , the object 64 _k + 1, the object 64 _{k + 1} or the object 64 _{k + 2} is on the front side, so that the object 64 _k , the object 64 _{k + 1} or the object 64 _{k + 2} is on the road when viewed from the vehicle 2a. Located on the back side. That is, when the temporarily stopped vehicle 2a starts, the risk of collision with the object 64 is low, so the risk is determined to be low.

The system 1A mounted on the temporarily stopped vehicle 2a acquires an image 50 _k , an image 50 _{k + 1} , and an image 50 _{k + 2} , and calculates the position and size of the object 64 _k , the object 64 _{k + 1} , or the object 64 _{k + 2} . You may. In this case, it can be seen that the object 64 approaches the vehicle 2a that is temporarily stopped at the intersection from the continuity that the size of the object 64 _k , the object 64 _{k + 1} , or the object 64 _{k + 2} increases. Therefore, the degree of danger for the object 64 _{k + 2} may be determined higher than that of the object 64 _{k + 1} and the like. In this way, the danger prediction unit 1031A may predict the degree of danger according to the change information of the size of each object reflected in each of at least two images continuous in time series and the position of the object in the curve mirror. That is, the danger prediction unit 1031A may predict the danger based on the position of the object in the curve mirror calculated by the calculation unit 101A and the calculated size of the object.

In the above, the example in which the size change information is the size change amount has been described, but the size change information may be information indicating the size change content. For example, the content of the change in size may be that the size has increased (or decreased), the size has become larger than the predetermined size (or smaller than the predetermined size), and the like.

Further, when the system 1A mounted on the temporarily stopped vehicle 2a acquires the image 50 _{k + 2} and the image 50 _{k + 3} , the object 64 does not exist in the curve mirror 75 of the image 50 _{k + 3} . However, since the object 64 _{k + 2} was present in the curve mirror 75 of the image 50 _{k + 2} at the time t _{k + 2} acquired immediately before the continuity, the image 50 _{k + 3} is a blind spot image in which the object 64 can exist in the blind spot of the curve mirror 75. It may be determined that the degree of danger to the object 64 _{k + 3} is higher than that of the object 64 _{k + 2} or the like.

Further, the danger prediction unit 1031A may determine the degree of danger predicted when the vehicle travels as it is, based on the position of the object in the curve mirror calculated by the calculation unit 101A. For example, the danger prediction unit 1031A determines that when the position of the object calculated by the calculation unit 101A is below the center line of the curve mirror, the risk level is higher than when the position is above the center line of the curve mirror. You may do so. Further, when the position of the object calculated by the calculation unit 101A is to the right of the predetermined position of the curve mirror, the danger prediction unit 1031A is higher than when the position is to the left of the predetermined position of the curve mirror. The degree of danger may be determined. The predetermined position may be the position of the central axis or may be a predetermined position near the center. The center line is, for example, a line passing through the center of a curved mirror, and includes horizontal lines, vertical lines, diagonal lines, and curves. When the center line is a horizontal line, the degree of danger may be determined by the top / bottom, and when the center line is a vertical line, the degree of danger may be determined by the right / left. Further, when the center line is a diagonal line or a curved line, the degree of danger may be determined by one / the other.

<< Information generation unit 1032A >>
The information generation unit 1032A generates driving support information based on the position of the object on the curve mirror calculated by the calculation unit 101A. For example, the information generation unit 1032A may generate driving support information according to the positional relationship between the object and the area in the curved mirror specified from the road reflected in the image. Further, the information generation unit 1032A may generate driving support information based on the position of the object calculated by the calculation unit 101A in the curve mirror and the size of the object calculated by the calculation unit 101A, for example, time. The driving support information may be generated according to the change information of the size of each object reflected in each of at least two consecutive images in the series and the position of the object in the curve mirror.

More specifically, the information generation unit 1032A generates driving support information according to the positional relationship between the position of the object calculated by the object position calculation unit 1013 and the central axis calculated by the central axis calculation unit 1016. You may. For example, the information generation unit 1032A generates driving support information for causing the vehicle to avoid the object when the object is located above the central axis in the image.

Further, the information generation unit 1032A may generate driving support information based on the positional relationship between the position of the object calculated by the calculation unit 101A and the center line of the curve mirror. For example, the information generation unit 1032A may have (i) when the curved mirror is facing to the left when viewed from the vehicle, when the object is located to the right of the center line in the image, or (ii) the curved mirror is the vehicle. When the object is located to the left of the center line in the image when facing to the right, it is sufficient to generate driving support information for the vehicle to avoid the object.

In the present embodiment, the information generation unit 1032A generates driving support information according to the danger predicted by the danger prediction unit 1031A. For example, the information generation unit 1032A may generate vehicle control information based on the danger predicted by the danger prediction unit 1031A. Further, the information generation unit 1032A may generate driving support information according to the danger predicted from the position of the object in the curve mirror in a situation where the vehicle travels in the direction toward the curve mirror.

The information generation unit 1032A may output information indicating the degree of danger predicted by the danger prediction unit 1031A as driving support information. For example, the information generation unit 1032A outputs information indicating the degree of danger determined by the danger prediction unit 1031 according to the amount of change in the size of an object corresponding to at least two consecutive images in time series as driving support information. ..

Further, the information generation unit 1032A includes vehicle control information for suspending the vehicle based on the position of the object calculated by the object position calculation unit 1013 and the determination of the blind spot image by the blind spot determination unit 103. Driving support information may be generated. Specifically, in the information generation unit 1032A, the object recognized by the recognition unit 13 in at least one image out of a plurality of consecutive images in time series is later in time series than at least one image. When the image is not recognized by the recognition unit 13, driving support information may be generated. Further, the information generation unit 1032A may generate driving support information based on the position of the object reflected in the one image and the fact that the object is not recognized by the recognition unit 13 in the subsequent image.

<< Information output unit 1033A >>
The information output unit 1033A outputs the driving support information generated by the information generation unit 1032A.

In the above, the example in which the driving support information is the control command information has been described, but the driving support information may be the presented information. For example, the presented information may be information indicating a danger described later or information indicating a recommended operation for the driver.

Here, an example of the output processing performed by the output processing unit 102A configured as described above will be described with reference to the drawings.

FIG. 39 is a diagram showing an example of a T-shaped intersection in the second embodiment. FIG. 40 is an explanatory diagram showing another example of the output processing performed by the output processing unit 102A in the second embodiment. FIG. 40 shows a state in which the vehicle 2a on which the information processing apparatus 10A or the system 1A of the present embodiment is mounted is temporarily stopped at a T-shaped intersection.

In this case, as shown in FIG. 40, in the output processing unit 102A, that is, the danger prediction unit 1031A, when the size of the object for each time becomes smaller or does not change, the position of the object in the curve mirror 75 is on the back side. However, it is judged that the risk is low even on the front side. Here, the output processing unit 102A, that is, the information generation unit 1032A may output information indicating that the degree of danger predicted by the danger prediction unit 1031A is low. Further, as shown in FIG. 40, the information generation unit 1032A may generate vehicle control information for starting the temporarily stopped vehicle 2a based on the low degree of danger. In the example shown in FIG. 39, the danger prediction unit 1031A can determine that the vehicle 3 is moving away from the vehicle 2a when the size of the vehicle 3 in the curve mirror 75 becomes smaller or does not change. Judge that the degree of risk is low. Then, the information generation unit 1032A may generate information indicating that the determined risk level is low or vehicle control information for starting the temporarily stopped vehicle 2a.

Further, as shown in FIG. 40, the danger prediction unit 1031A determines that the degree of danger is medium when the position of the object in the curve mirror 75 is on the front side and the size of the object in the curve mirror 75 becomes large. do. Here, the information generation unit 1032A may generate information indicating that the degree of danger determined by the danger prediction unit 1031A is medium. Further, as shown in FIG. 40, the information generation unit 1032A may generate vehicle control information for starting the temporarily stopped vehicle 2a while driving slowly, based on the fact that the degree of danger is medium. In the example shown in FIG. 39, when the position of the vehicle 3 in the curve mirror 75 is in front of the vehicle 3 in the curve mirror 75 and the size of the vehicle 3 in the curve mirror 75 becomes large, the vehicle 3 moves to the back side of the road. Since it can be determined that the vehicle is in progress in the direction toward 2a, it is determined that the degree of danger is medium. Then, the information generation unit 1032A may generate information indicating that the determined danger level is medium or vehicle control information for driving the temporarily stopped vehicle 2a in a slow speed.

Further, as shown in FIG. 40, the danger prediction unit 1031A determines that the degree of danger is high when the position of the object in the curve mirror 75 is on the back side and the size of the object in the curve mirror 75 becomes large. do. Here, the information generation unit 1032A may generate information indicating that the degree of danger determined by the danger prediction unit 1031A is high. Further, as shown in FIG. 40, the information generation unit 1032A is for starting the temporarily stopped vehicle 2a after confirming the passage of the vehicle 3 which is the target object by a sensor or the like based on the high degree of danger. Vehicle control information may be generated. In the example shown in FIG. 39, when the size of the vehicle 3 in the curve mirror 75 becomes large, the danger prediction unit 1031 can determine that the vehicle 3 is traveling in the direction toward the vehicle 2a on the front side of the road. , Judge that the degree of danger is high. Then, the information generation unit 1032A generates information indicating that the determined danger level is high, or vehicle control information for starting the temporarily stopped vehicle 2a after confirming the passage of the vehicle 3 with a sensor or the like. You may.

[Operation of system 1A]
Next, the information processing method of the system 1A configured as described above will be described. FIG. 41 is a flowchart showing an outline of the information processing method of the system 1A according to the second embodiment. FIG. 42 is a flowchart showing details of the information processing method of the system 1A in the second embodiment. The same elements as in FIG. 19 are designated by the same reference numerals, and detailed description thereof will be omitted.

First, as shown in FIG. 41, the system 1A detects the curved mirror reflected in the image and the object reflected in the curved mirror based on the image information (S20). Next, the system 1A calculates the position on the curve mirror detected in S20 (S21). Next, the system 1A generates driving support information based on the position calculated in S21 (S22). Then, the system 1A outputs the driving support information generated in S22 (S23).

More specifically, as shown in FIG. 42, in the process of S21, the system 1A performs an object position calculation process of calculating the position of the object recognized by S103 in the curve mirror detected by S102 (S204). .. Next, the system 1A performs an object size calculation process for calculating the size of the object recognized in S103 with reference to the curve mirror detected in S102 (S205). Next, the system 1A performs a road surface area calculation process for calculating the road surface area in the curve mirror detected in S102 (S206). Next, the system 1A performs a central axis calculation process for calculating the central axis of the road surface region calculated in S206 (S207). Since the details of the processes performed in S204 to S207 are as described above, detailed description thereof will be omitted here.

Further, in the process of S22 described with reference to FIG. 41, the system 1A performs a risk prediction process of determining the degree of danger predicted when the vehicle travels as it is, based on the position of the object in the curve mirror calculated in S21. Do (S221). Next, the system 1A generates information indicating the degree of danger determined in S221 as driving support information, or generates vehicle control information for controlling the vehicle based on the degree of danger determined in S221. Information generation processing is performed (S222). Since the details of the processes performed in S221 and S222 are as described above, the description thereof will be omitted here.

Then, the system 1A performs the process of S23 described with reference to FIG. 41. More specifically, in S23, the system 1A performs an information output process for outputting the driving support information and the like generated in S222 (S223).

[Effects of Embodiment 2]
As described above, according to the information processing apparatus 10A or the system 1A according to the second embodiment, the degree of danger can be determined by using the curve mirror in a place where the visibility is poor such as an intersection, and the determined danger can be determined. Vehicle driving support information can be generated and output based on the degree. Thereby, it is possible to support the safe driving of the vehicle equipped with the information processing device 10A.

Specifically, when an object such as a person is reflected on a curved mirror in a place with poor visibility such as an intersection, the object may jump out to the place. In this case, if the temporarily stopped vehicle is started early, there is a risk of collision with the object. In addition, the curved mirror has the characteristic of flipping left and right.

Therefore, when an object such as a person is reflected on the back side of the curved mirror, the side closer to the temporarily stopped vehicle at the place is compared with the case where an object such as a person is reflected on the front side of the curved mirror. It can be seen that the vehicle is moving near the shoulder side of the road. In other words, when an object such as a person is reflected on the back side of the curve mirror, the vehicle that is temporarily stopped is started earlier than when an object such as a person is reflected on the front side of the curve mirror. There is a high risk of collision with the object.

Therefore, in the present embodiment, the degree of danger is determined from the position of the object in the curved mirror reflected in the image obtained from the temporarily stopped vehicle and, if necessary, the size change thereof. In this way, the information processing apparatus 10A according to the second embodiment can determine the degree of danger by using the curve mirror.

For example, when the vehicle during automatic driving is equipped with the information processing device 10A, the vehicle can determine the degree of danger by using the curve mirror as described above, so that the degree of danger can be determined according to the predicted degree of danger. Vehicle control can be performed.

Further, as described above, for example, the information processing apparatus 10A according to the second embodiment generates driving support information for causing the vehicle to avoid the object when the object is located above the central axis in the image. You may do so. As a result, the object can be avoided when the object is close to the vehicle, and the safety of the object and the vehicle can be ensured.

Further, for example, in the information processing apparatus 10A according to the second embodiment, (i) when the curved mirror is directed to the left when viewed from the vehicle, when the object is located to the right of the center line in the image, or. (Ii) When the curved mirror is facing to the right when viewed from the vehicle and the object is located to the left of the center line in the image, even if the vehicle generates driving support information to avoid the object. good. As a result, the object can be avoided when the object is close to the vehicle, and the safety of the object and the vehicle can be ensured. In addition, complicated processing such as specifying the central axis of the road can be omitted, and the processing speed can be improved.

As described above, according to the information processing apparatus 10A or the system 1A according to the second embodiment, it is possible to support the safe driving of the vehicle by using the curved mirror in a place where the visibility is poor such as an intersection.

Although the example in which the vehicle is temporarily stopped has been described above, the second embodiment may be applied to a vehicle moving at a low speed (predetermined speed or less).

(Modification 1)
In the second embodiment, the case where the degree of danger is determined according to the position of the object in the curved mirror and the size of the object if necessary has been described, but the present invention is not limited to this. The recognition unit 13 may recognize the attribute of the object reflected on the curve mirror, and the calculation unit 101 may determine the degree of danger in consideration of the attribute. In this modification, a case where the degree of danger is determined by further considering the attribute related to the moving speed of the object reflected in the curved mirror will be described.

FIG. 43 is an explanatory diagram showing an example of the risk prediction process performed by the output processing unit 102A in the first modification of the second embodiment. The same wording is described in the same contents as in FIG. 40, and detailed description thereof will be omitted. Further, FIG. 43 shows an example of the danger prediction process performed by the output processing unit 102A in the modification 1 at an intersection such as a T-shape. Since the vehicle control information based on the low, medium, or high risk level may be as shown in FIG. 40, the illustration is omitted in FIG. 43.

As shown in FIG. 43, when the object is a person, the danger prediction unit 1031A may determine the same degree of danger as in FIG. 40. On the other hand, when the object is a bicycle, a motorcycle, or an automobile whose moving speed is faster than that of a person, the danger prediction unit 1031A may determine a higher degree of danger according to the moving speed than when the object is a person.

As described above, in this modification, the output processing unit 102A generates driving support information according to the attributes of the object recognized by the recognition unit 13, and outputs the generated driving support information.

(Modification 2)
In the first modification, the case where the degree of danger is determined by further considering the attribute related to the moving speed of the object reflected in the curved mirror has been described, but the present invention is not limited to this. When the object reflected in the curve mirror is a person, the danger prediction unit 1031A may determine the degree of danger by further considering the attribute related to the age of the person. This case will be described below as Modification 2.

FIG. 44 is an explanatory diagram showing an example of risk prediction processing performed by the output processing unit 102A in the second modification of the second embodiment. The same wording is described in the same contents as in FIG. 40, and detailed description thereof will be omitted. Further, FIG. 44 shows an example of the danger prediction process performed by the output processing unit 102A in the modification 2 at an intersection such as a T-shape. Since the vehicle control information based on the low, medium, or high risk level may be as shown in FIG. 40, the illustration is omitted in FIG. 44 as well.

As shown in FIG. 44, when the object is a person and the person is a child or an elderly person, the danger prediction unit 1031A may determine the same degree of danger as in FIG. 40. On the other hand, if the object is a person and the person is something other than a child or the elderly, the risk prediction unit 1031A determines that the risk is higher than when the person is a child or the elderly. good.

As described above, in this modification, when the attribute of the object recognized by the recognition unit 13 is a person, the output processing unit 102A has driving support information according to the information regarding the age of the object recognized by the recognition unit 13. Is generated by changing, and the generated driving support information is output. In the above, the example in which the information regarding the age is the generation of the person has been described, but the information regarding the age may be the age or the age of the person.

(Modification 3)
In the second modification, when the object reflected in the curved mirror is a person, the case where the degree of danger is determined by further considering the attribute related to the age of the person has been described, but the present invention is not limited to this. When the object reflected in the curved mirror is a person, the danger prediction unit 1031A further considers the attribute of whether or not the person is walking while looking at a mobile terminal such as a smartphone, and determines the degree of danger. good. Specifically, inattention behavior is non-forward vision behavior. For example, non-forward vision behavior includes gaze at a mobile terminal such as a smartphone or a book. This case will be described below as Modification 3.

FIG. 45 is an explanatory diagram showing an example of risk prediction processing performed by the output processing unit 102A in the modification 3 of the second embodiment. The same wording is described in the same contents as in FIG. 40, and detailed description thereof will be omitted. Further, FIG. 45 shows an example of the danger prediction process performed by the output processing unit 102A in the modification 1 at an intersection such as a T-shape. Since the vehicle control information based on the low, medium, or high risk level may be as shown in FIG. 40, the illustration is omitted in FIG. 45 as well.

As shown in FIG. 45, when the object is a person and the person does not perform a non-forward-looking behavior in which the person looks at the mobile terminal and performs a moving action, that is, the person does not walk while looking at the mobile terminal. The danger prediction unit 1031A may determine the same degree of danger as in FIG. 40. On the other hand, when the object is a person and the person is non-forward-looking, that is, the person is walking while looking at the mobile terminal, the danger prediction unit 1031A indicates that the person is non-forward-looking. It suffices to determine a higher degree of risk than when not acting. In the above, an example in which the careless behavior is a non-forward vision behavior has been described, but the careless behavior is a behavior in front of the person but looking upward or downward, or a baby carriage or a ball in front of the person. It may be an action of gazing at a specific object.

As described above, in this modification, when the attribute of the object recognized by the recognition unit 13 is a person, the output processing unit 102A determines whether or not the object recognized by the recognition unit 13 is inadvertently acting. The driving support information is changed and generated accordingly, and the generated driving support information is output.

(Embodiment 3)
The curved mirror is also present in a place with poor visibility other than the T-junction or the curved curved road described in the first and second embodiments.

For example, in a place near the exit of a vehicle-only road, there is a high possibility that a car will pop out without slowing down, and there is a high possibility that a person or a vehicle will pop out at the entrance of a shopping mall. Also, for example, at the entrance of a shopping mall, there is a high possibility that people will jump out because there are many people coming and going. In addition, for example, in a place such as a corner where a car is connected to a straight line where it is easy to speed up, the car may swell beyond the lane without slowing down, that is, it may pop out. There is a high risk of accidents in places with poor visibility where curved mirrors such as these exist. That is, depending on the position of the own vehicle or the place where the curved mirror is installed, the degree of danger also differs because the conditions such as the place where many people appear or the place where the speed of the car is high are different.

Therefore, in the third embodiment, an information processing device and the like that can support safe driving of a vehicle that is temporarily stopped or in progress will be described by considering the characteristics of the installation location of the curved mirror. In the following, the vehicle equipped with the system or the information processing device according to the third embodiment will be referred to as a own vehicle.

[System 1B configuration]
FIG. 46 is a block diagram showing an example of the configuration of the system 1B according to the third embodiment. The same elements as those in FIGS. 1 and 26 are designated by the same reference numerals, and detailed description thereof will be omitted.

The system 1B shown in FIG. 46 has a different configuration of the acquisition unit 11B, the detection processing unit 12b, and the information processing device 10B as compared with the system 1 in the first embodiment. Similar to the system 1, the system 1B is also mounted on a vehicle such as an automobile, and by using a curve mirror, it is possible to output driving support information of the vehicle, that is, the own vehicle.

In the present embodiment, in order to support the safe driving of the own vehicle during temporary stop or in progress, the information processing apparatus 10B also takes into consideration the characteristics of the installation location of the curve mirror.

[Acquisition unit 11B]
Similar to the image acquisition unit 11, the acquisition unit 11B acquires image information indicating an image obtained by imaging with an image pickup device mounted on the vehicle. The acquisition unit 11B may acquire map information. Map information is information that indicates a dynamically changing event at a geographic location or geographic static information. Map information includes additional map information indicating at least one of traffic accidents, traffic jams, construction, road conditions and weather on the map.

The acquisition unit 11B is different from the image acquisition unit 11 in the first embodiment in that it can acquire map information, and is the same in other respects. In the acquisition unit 11B, the sensing unit 111 may acquire the map information by communication or the like and record it in the image recording unit 112.

[Detection processing unit 12b]
The detection processing unit 12b detects the curved mirror and the object reflected on the curved mirror. In the present embodiment, the detection processing unit 12b detects the curve mirror around the vehicle based on the map information or the image information indicating the image obtained by the image pickup of the image pickup device mounted on the vehicle.

As shown in FIG. 46, the detection processing unit 12b includes a detection unit 12B and a recognition unit 13B.

<Detection unit 12B>
The detection unit 12B detects the curved mirror. More specifically, the detection unit 12B detects the curved mirror around the vehicle based on the image information. For example, the detection unit 12B detects the curve mirror around the vehicle based on the image information by using the identification information of the curve mirror or the classifier of the curve mirror. The method of detecting the curve mirror around the vehicle based on the image information is the same as the method of detecting the curve mirror in the image in the first embodiment.

Further, the detection unit 12B may detect the curve mirror around the vehicle based on the map information. For example, the detection unit 12B detects the curve mirror around the vehicle based on the position of the vehicle and the installation location of the curve mirror indicated by the map information.

<Recognition unit 13B>
The recognition unit 13B recognizes an object reflected on the curved mirror based on image information indicating an image obtained by imaging by an image pickup device mounted on a vehicle. Further, the recognition unit 13B recognizes the surrounding environment of the curve mirror based on the image information by using the identification information of the curve mirror or the discriminator of the curve mirror. In this way, the recognition unit 13B recognizes the object in the curved mirror and the surrounding environment of the curved mirror reflected in the image acquired by the acquisition unit 11B.

The acquisition unit 11B and the detection processing unit 12b may be configured in the information processing device 10B as in the first and second embodiments.

[Information processing device 10B]
The information processing apparatus 10B according to the third embodiment outputs driving support information of the own vehicle being suspended or in progress by using the curve mirror. In the present embodiment, in order to support the safe driving of the own vehicle during temporary stop or in progress, the information processing apparatus 10B also takes into consideration the characteristics of the installation location of the curve mirror. This is because, as described above, it is possible to consider different degrees of risk depending on the position of the own vehicle or the installation location of the curved mirror.

More specifically, as shown in FIG. 46, the information processing apparatus 10B includes an output processing unit 102B, a blind spot determination unit 103, and a determination unit 104. The information processing device 10B shown in FIG. 46 has an additional configuration of the determination unit 104 and a different configuration of the output processing unit 102B as compared with the information processing device 10 in the first embodiment. Hereinafter, details such as the configuration of the determination unit 104 and the output processing unit 102B will be described.

<Judgment unit 104>
FIG. 47 is a diagram showing an example of the functional configuration of the determination unit 104 in the third embodiment.

As shown in FIG. 47, the determination unit 104 includes a feature determination unit 1041, a first risk determination unit 1042, and a second risk determination unit 1043.

<< Feature determination unit 1041 >>
The feature determination unit 1041 determines the characteristics of the installation location of the curved mirror. The features include the mode of the road at the installation site of the curved mirror or the mode of passage of an object at the installation site.

More specifically, the feature determination unit 1041 determines the characteristics of the installation location of the curved mirror based on the map information. For example, the feature determination unit 1041 determines the feature based on the map information around the curve mirror. The feature determination unit 1041 may determine the mode of the road or the mode of passage of the object from the additional map information.

Further, the feature determination unit 1041 may determine the characteristics of the installation location of the curved mirror based on the image information. For example, the feature determination unit 1041 may determine the feature based on the surrounding environment of the curved mirror. The feature determination unit 1041 may determine the mode of the road or the mode of passage of an object from the surrounding environment of the curved mirror.

Further, the feature determination unit 1041 may determine the feature based on the object reflected on the curved mirror. In this case, the feature determination unit 1041 may determine that the object is reflected in the curve mirror or the amount of the object reflected in the curve mirror as the feature.

The feature determination unit 1041 further has an installation location acquisition unit that acquires information on the characteristics of the installation location of the curved mirror reflected in the image obtained by continuously photographing the front of the own vehicle in chronological order. You may. The installation location acquisition unit is not limited to the case where the feature determination unit 1041 has, and the detection processing unit 12b or the acquisition unit 11B may have the installation location acquisition unit.

For example, the installation location acquisition unit acquires the installation location of the curved mirror reflected in the image obtained by continuously shooting the front of the own vehicle in chronological order from the GPS (Global Positioning System) information indicating the position of the own vehicle. can do. In this case, the installation location acquisition unit is based on the acquired installation location and the local dynamic map in which information such as traffic accident frequent occurrence points, traffic congestion information, road construction information, accident site and road surface information are embedded, and the installation location of the curve mirror. Get information about the features of.

The characteristic of the installation location of the curved mirror is the mode of the road described above. Specifically, the mode of the road is the location where the accident is likely to occur or where the accident actually occurred, the degree of congestion of the road such as the number of vehicles or the presence or absence of traffic congestion, the road surface condition such as leaf fall, rock fall, snow cover or freezing, or Whether or not there is construction work. For example, a place where an accident is likely to occur is near the exit of a vehicle-only road, the entrance or entrance of a shopping mall, or a corner connected to a straight line where a car can easily speed up.

FIG. 48 is an explanatory diagram showing an example of a dynamic map. The local dynamic map is a part of the dynamic map including the vicinity of a predetermined place, and the dynamic information that changes every moment around the position of the own vehicle is added to the high-definition map data that is static information such as roads and features. It is map information that has been enhanced by superimposing it. As shown in FIG. 48, the local dynamic map includes quasi-static data and quasi-dynamic data on the bottom layer basic data which is static information such as road surface information, lane information, and three-dimensional structure. Dynamic data is superimposed. Here, the quasi-static data includes, for example, traffic regulation information, road construction information, wide area weather information, and the like, and the quasi-dynamic data includes, for example, accident information, traffic congestion information, narrow area weather information, and the like. The dynamic data includes ITS (Intelligent Transport Systems) look-ahead information such as peripheral vehicle information, pedestrian information, and signal information.

In addition, the installation location acquisition unit installs the curve mirror from the installation location acquired by GPS information and the surrounding environment of the curve mirror reflected in the image obtained by taking a picture with the in-vehicle camera installed in the own vehicle. You may get information about the characteristics of the place. Further, the characteristic of the installation place of the curved mirror may be the mode of passage of an object at the above-mentioned installation place. Specifically, the mode of passage of an object at the installation site is the presence / absence or degree of entry / exit of a person or a vehicle.

Here, the information regarding the characteristics of the installation location of the curved mirror is the information indicating whether or not the location is likely to cause an accident obtained by using the map information or the captured image as described above.

49A to 49E are examples of the surrounding environment of the curved mirror in the third embodiment. The surrounding environment is an entrance / exit of a commercial facility such as a shopping center as shown in FIGS. 49A to 49E. Since the entrance / exit of the commercial facility has variations as shown in FIGS. 49A to 49E, the installation location acquisition unit is a commercial facility obtained by learning (deep learning, etc.) by inputting an image showing the entrance / exit of the commercial facility. It may be equipped with an entrance / exit recognizer. As a result, the installation location acquisition unit can acquire information indicating an accident-prone location such as a warehousing entrance or entrance of a shopping mall from an image showing the surrounding environment of the curved mirror reflected in the image.

FIG. 50 is a diagram showing an example of the map 86 used by the installation location acquisition unit in the third embodiment.

The installation location acquisition unit may acquire information on the characteristics of the installation location of the curved mirror from the surrounding environment of the curved mirror reflected in the image after acquiring the map 86 showing the current position of the own vehicle in FIG. 50. When the current position of the own vehicle is near a commercial facility 860 such as a shopping center on the map 86 shown in FIG. 50, and then the current position of the own vehicle is an indoor parking lot of the commercial facility 860 for which GPS information cannot be obtained, it is commercial. This is because it is possible to obtain that the current position of the own vehicle is in the facility 860. As a result, the installation location acquisition unit can more accurately acquire information indicating a location where an accident is likely to occur, such as an entrance or exit of the commercial facility 860, from the surrounding environment of the curved mirror reflected in the image.

When the current position of the own vehicle is an outdoor parking lot of a shopping center where GPS information can be acquired, the installation location acquisition unit may acquire information on the characteristics of the installation location of the curved mirror from the geographical information.

<< First danger level determination unit 1042 >>
The first risk level determination unit 1042 determines the first risk level indicating the high possibility of an accident of the own vehicle based on the information regarding the characteristics of the installation location of the curved mirror acquired by the installation location acquisition unit. .. The first risk level determination unit 1042 may determine the first risk level based on the information regarding the characteristics of the installation location of the curved mirror determined (acquired) based on the current position of the own vehicle and the map information. , The first degree of danger may be determined based on the information regarding the characteristics of the installation location of the curved mirror determined (acquired) based on the surrounding environment of the curved mirror reflected in the image.

First, with reference to FIGS. 51 to 54, an example of a method in which the first risk level determination unit 1042 determines the first risk level based on geographic information will be described. 51 to 54 are diagrams showing an example of a local dynamic map used by the first risk determination unit 1042 in the third embodiment for determination. In FIGS. 51 to 54, similar elements are designated by the same reference numerals.

FIG. 51 shows an example of the local dynamic map 86t of the commuting time zone acquired by the current position of the own vehicle. In FIG. 51, it is assumed that the road 861, the road 862, the highway 863, and the road 864 indicate a road on which the own vehicle may travel. Further, it is assumed that the local dynamic map 86t includes information on dynamically changing traffic accidents and traffic jams, and information indicating that many people commute through the road 861.

In this case, the first risk level determination unit 1042 determines that the first risk level of the road 861 is high, assuming that the possibility of an accident of the own vehicle is the highest on the hatched road 861. Further, the first danger level determination unit 1042 considers that the first danger is that the accident of the own vehicle is the next highest on the hatched road 862 which is the road 862 which is connected to the road 861 and passes through the commercial facility 865. Judge the degree as medium. The first risk level determination unit 1042 determines that the first risk level is low on the highways 863 and the road 864 other than the roads 861 and 862, assuming that the possibility of an accident of the own vehicle is low.

FIG. 52 shows _an example of the local dynamic map 86t1 acquired by the current position of the own vehicle during the long holidays. It is assumed that the local dynamic map 86t ₁ includes information indicating that there are many traffic jams and accidents on the expressway 863 as dynamically changing traffic accidents and traffic jam information. In this case, the first risk level determination unit 1042 determines that the first risk level of the highway 863 is high, assuming that the possibility of an accident of the own vehicle is the highest on the hatched highway 863. Further, the first danger level determination unit 1042 determines that the first risk level of the hatched road 862 and the road 864 connected to the expressway 863 is medium because the possibility of an accident of the own vehicle is the next highest. .. The first risk level determination unit 1042 determines that the first risk level is low on the roads other than the expressway 863, the road 862, and the road 864, assuming that the possibility of an accident of the own vehicle is low.

Further, FIG. 53 shows an example of the local dynamic map 86t ₂ of the weekend afternoon time zone acquired by the current position of the own vehicle. In the local dynamic map 86t ₂ , as dynamically changing traffic accident and congestion information, there are many congestions and accidents on the graph showing the time zone when the commercial facility 865 such as a shopping center is congested and the road 862 connecting to the commercial facility 865. It is assumed that the information indicating the above is included.

In this case, the first risk level determination unit 1042 determines that the first risk level of the road 862 is high, assuming that the possibility of an accident of the own vehicle is the highest on the hatched road 862. Further, the first risk level determination unit 1042 determines that the first risk level is medium, assuming that the possibility of a vehicle accident occurring on the hatched road 861 connected to the road 862 is the next highest. The first risk level determination unit 1042 determines that the first risk level of the highway 863 and the road 864 other than the road 861 and the road 862 is low because the possibility of an accident of the own vehicle is low.

Further, FIG. 54 shows an example of the local dynamic map _86t3 at the time of the event acquired by the current position of the own vehicle. In addition, the local dynamic map _86t3 includes information indicating that there are many traffic jams and accidents on the road 862 connected to the concert venue 866 where an event such as a concert occurred, as dynamically changing traffic accident and traffic jam information. And.

In this case, the first risk level determination unit 1042 determines that the first risk level of the road 862 is high, assuming that the possibility of an accident of the own vehicle is the highest on the hatched road 862. Further, the first risk level determination unit 1042 determines that the first risk level is medium, assuming that the possibility of a vehicle accident occurring on the hatched road 861 connected to the road 862 is the next highest. The first danger level determination unit 1042 determines that the first risk level of the highways 863 and the road 864 other than the road 861 and the road 862 is low because the possibility of an accident of the own vehicle is low.

In this way, the first risk determination unit 1042 is geographic information indicating the current position of the own vehicle and the traffic accident and traffic congestion information that dynamically changes depending on the current time, season, current event, and the like. The first degree of danger can be determined based on.

Next, an example of a method in which the first risk level determination unit 1042 determines the first risk level based on the surrounding environment of the curved mirror reflected in the image will be described. Here, it is assumed that the installation location acquisition unit acquires information on the characteristics of the installation location of the curved mirror such as the entrance / exit of a shopping center or a parking lot from the surrounding environment of the curved mirror reflected in the image.

If the current position of the own vehicle is estimated to be the parking lot of a commercial facility such as a shopping center from the information on the characteristics of the installation location of the curved mirror, the first risk determination unit 1042 causes an accident of the own vehicle. Assuming that the possibility is medium, the first degree of danger is determined to be medium. This is because many cars are parked in the parking lot, and the movement of the own vehicle in the parking lot is poor. On the other hand, the first risk determination unit 1042 causes an accident of the own vehicle when the current position of the own vehicle is an entrance / exit or a warehousing entrance of a commercial facility such as a shopping center from the information regarding the characteristics of the installation location of the curved mirror. As the possibility is high, the first risk is judged to be high. This is because many people are moving at the entrances and exits of commercial facilities, and there is a high possibility of collision between their own vehicle and people. In addition, there is a high possibility that the vehicle or a motorcycle will collide with a person because the entrance of the shopping center is usually inclined and the vehicle or motorcycle may accelerate or jump out after being stopped.

<< Second danger level determination unit 1043 >>
The second danger level determination unit 1043 determines the risk level predicted when the own vehicle travels based on the first risk level determined by the first risk level determination unit 1042 and the presence / absence of an object reflected in the curve mirror. Determine a second degree of risk.

When there is an object reflected in the curve mirror, the second risk degree determination unit 1043 determines the second risk degree in consideration of the risk of collision with the object when the own vehicle travels. More specifically, the second risk determination unit 1043 sets the second risk to medium when the first risk determined by the first risk determination unit 1042 is small and there is an object reflected in the curved mirror. Judgment is made, and if there is no object reflected in the curve mirror, the second risk is judged to be low. When the first risk level determined by the first risk level determination unit 1042 is medium, the second risk level determination unit 1043 determines that the second risk level is medium if there is an object reflected in the curve mirror, and the curve mirror. If there is no object reflected in, the second risk is judged to be low. Further, when the first danger degree determined by the first danger degree determination unit 1042 is large, the second danger degree determination unit 1043 determines that the second danger degree is high if there is an object reflected in the curved mirror. If there is no object reflected in the curved mirror, the second risk is judged to be medium.

For example, if the position of the own vehicle is at the warehousing port of a shopping mall, the car or motorcycle may accelerate or jump out after temporarily stopping at the warehousing port. 1 It is determined that the degree of danger is medium. When there is a vehicle reflected in the curved mirror, the second danger level determination unit 1043 determines that the second risk level is medium because there is a high possibility of a collision between the own vehicle located near the warehousing port and the vehicle. On the other hand, the second danger level determination unit 1043 determines that the second risk level is low because the possibility of collision between the own vehicle and the vehicle is low when there is no vehicle reflected in the curve mirror.

Further, for example, when the position of the own vehicle is near the entrance / exit of the shopping mall, many people come and go in the vicinity of the entrance / exit, and the probability of accident occurrence increases. Is determined to be large. When there is a person reflected in the curved mirror, the second danger level determination unit 1043 determines that the second risk level is high because there is a high possibility of a collision between the own vehicle located near the doorway and the person. On the other hand, the second danger level determination unit 1043 determines that the second risk level is medium because the possibility of collision between the own vehicle and a person is low when there is no person reflected in the curve mirror.

The second risk level determination unit 1043 may determine the second risk level based only on the first risk level determined by the first risk level determination unit 1042. In this case, the second risk level determination unit 1043 may determine that the second risk level is high if the first risk level determined by the first risk level determination unit 1042 is large. Similarly, if the first risk level determined by the first risk level determination unit 1042 is medium or small, the second risk level determination unit 1043 may determine the second risk level as medium or low.

In this way, the determination unit 104 determines the second degree of danger as a feature of the installation location of the curved mirror reflected in the image obtained by continuously photographing the front of the own vehicle in time series.

For example, if the position of the own vehicle is near a motorway or a highway entrance / exit, the vehicle leaving the vehicle exclusive road may not slow down in the vicinity of the entrance / exit, and the probability of an accident may increase. Therefore, the first risk level determination unit 1042 may determine that the first risk level is high. and. When there is a vehicle reflected in the curve mirror, the second danger level determination unit 1043 may determine that the second risk level is high because the possibility of collision between the own vehicle and a person is high. On the other hand, when there is no vehicle reflected in the curve mirror, the second danger level determination unit 1043 may determine that the second danger level is medium because the possibility of collision between the own vehicle and a person is low.

In the above, an example in which the presence or absence of an object reflected in the curved mirror is used for determining the second risk level has been described, but the type, number, number, or density of objects reflected in the curved mirror is used for determining the second risk level. Etc. may be used.

<Output processing unit 102B>
The output processing unit 102B generates driving support information based on the characteristics determined by the determination unit 104, and outputs the generated driving support information. Here, the driving support information may include control command information of the behavior of the vehicle, or may include presentation information presented to the occupants of the vehicle. The presented information may include information that presents the hazards predicted from the characteristics at the location of the curved mirror.

FIG. 55 is a diagram showing an example of the functional configuration of the output processing unit 102B according to the third embodiment.

In the present embodiment, the output processing unit 102B includes an information generation unit 1032B and an information output unit 1033B, as shown in FIG. 55.

<< Information generation unit 1032B >>
The information generation unit 1032B generates driving support information based on the characteristics determined by the determination unit 104. The information generation unit 1032B may generate driving support information according to the determined mode of the road or the mode of passage of an object. For example, the information generation unit 1032B generates driving support information for decelerating, stopping, or detouring the vehicle when the determined road mode or the passing mode of the object is a mode that hinders the safe running of the vehicle. Further, for example, the information generation unit 1032B decelerates, stops, or detours the vehicle according to the fact that the object is reflected in the curve mirror or the amount of the object reflected in the curve mirror determined by the determination unit 104. You may generate driving support information to make it.

The information generation unit 1032B may generate driving support information according to the danger predicted from the characteristics.

Further, as in the second embodiment, the output processing unit 102B may further generate driving support information when the above-mentioned blind spot image is determined. For example, the output processing unit 102B may generate information for suspending the vehicle for a certain period of time as driving support information when the blind spot image is determined by the blind spot determination unit 103.

<< Information output unit 1033B >>
The information output unit 1033B outputs the driving support information generated by the information generation unit 1032B.

Hereinafter, an example of the output processing performed by the output processing unit 102B configured as described above will be described with reference to the drawings. 56 and 57 are explanatory views showing an example of output processing performed by the output processing unit 102B in the third embodiment. FIG. 56 shows an example of vehicle control information of the own vehicle output by the output processing unit 102B in the third embodiment on a curved curved road, and FIG. 57 shows output processing in the third embodiment at a T-shaped intersection. An example of the vehicle control information of the own vehicle output by the unit 102B is shown. In addition to the second degree of danger and the vehicle control information thereof, FIGS. 56 and 57 also show the first degree of danger based on the location where the curve mirror is installed and the presence or absence of an object reflected in the curve mirror.

Therefore, when the own vehicle is located on the curved curved road, the output processing unit 102B controls the vehicle as shown in FIG. 56 based on the second danger level determined by the second risk level determination unit 1043. Vehicle control information for this purpose may be generated and output. On the other hand, when the own vehicle is located at the T-shaped intersection, the output processing unit 102B controls the vehicle as shown in FIG. 57 based on the second risk level determined by the second risk level determination unit 1043. Vehicle control information for this purpose may be generated and output.

Since the relationship between the second risk level and the vehicle control information is the same as the relationship between the risk level and the vehicle control information described with reference to FIGS. 16 and 17, detailed description here will be omitted.

[Operation of system 1B]
Next, the information processing method of the system 1B configured as described above will be described. FIG. 58 is a flowchart showing an outline of the information processing method of the system 1B according to the third embodiment. FIG. 59 is a flowchart showing details of the information processing method of the system 1B in the third embodiment. The same elements as those in FIG. 19 and the like are designated by the same reference numerals, and detailed description thereof will be omitted.

As shown in FIG. 58, first, the system 1B acquires map information or image information indicating an image obtained by imaging an image pickup device mounted on a vehicle (S30). Next, the system 1B detects the curve mirror around the vehicle based on the map information or the image information acquired in S30 (S31). Next, the system 1B determines the characteristics of the installation location of the curved mirror based on the map information or the image information (S32). Next, the system 1B generates driving support information based on the feature determined in S32 (S33). Then, the system 1B outputs the driving support information generated in S33 (S34).

More specifically, as shown in FIG. 59, first, the system 1B performs the process of S30 described with reference to FIG. 58. More specifically, in the process of S30, the system 1B performs an acquisition process of acquiring map information or image information indicating an image obtained by imaging of an image pickup device mounted on a vehicle (S301).

Next, the system 1B performs the process of S31 described with reference to FIG. 58. More specifically, in the process of S31, the system 1B performs a detection process of detecting a curve mirror around the vehicle based on the map information or the image information acquired in S301 (S302). Further, the system 1B performs a recognition process for recognizing an object reflected on the curved mirror based on the image information acquired in S302 (S303).

Next, the system 1B performs the process of S32 described with reference to FIG. 58. More specifically, in the process of S32, the system 1B performs a feature determination process for determining the feature of the installation location of the curved mirror based on the map information or the image information (S304). Next, the system 1B performs a first risk level determination process for determining a first risk level indicating the high possibility that an accident of the own vehicle will occur based on the information regarding the characteristics of the installation location of the curved mirror (S305). .. Next, the system 1B determines the second risk, which is the predicted risk when the own vehicle travels, based on the first risk determined in S305 and the presence / absence of the object reflected in the curve mirror. 2 Perform the risk determination process (S306). Since the details of the processes performed in S304 to S306 are as described above, detailed description thereof will be omitted here.

Further, in the process of S33, the system 1B performs an information generation process of generating as driving support information according to the second risk degree determined in S306 (S307). Since the details of the processing performed in S307 are as described above, the description thereof is omitted here.

Then, the system 1B performs the process of S34 described with reference to FIG. 57. More specifically, in S34, the system 1B performs information output processing for outputting the driving support information and the like generated in S307 (S308).

[Effects of Embodiment 3 and the like]
As described above, according to the information processing apparatus 10B or the system 1B according to the third embodiment, the curved mirror can be used even in a place with poor visibility such as a parking lot or an entrance / exit of a commercial facility in addition to an intersection. It is possible to determine the degree of danger (that is, the second degree of danger) when the vehicle travels. Then, since the driving support information of the vehicle can be generated and output based on the determined second degree of danger, it is possible to support the safe driving of the vehicle equipped with the information processing device 10B.

Specifically, if the characteristics of the location where the curved mirror is installed, that is, when the location where the curved mirror is installed is an intersection or parking lot, the entrance / exit of a commercial facility, the vicinity of the exit of a vehicle-only road, etc., the visibility is poor and an accident is likely to occur. There is also the possibility of an accident. That is, if a vehicle located near this location is advanced, there is a risk that an accident may occur in the own vehicle. Further, if the place where the curved mirror is installed is a place where the visibility is poor and an accident is likely to occur, and if an object is reflected on the curved mirror, there is a higher risk that an accident will occur in the own vehicle.

Therefore, in the present embodiment, it is predicted when the vehicle travels from the characteristics of the installation location of the curved mirror reflected in the image that can be obtained from the vehicle that is temporarily stopped or in progress and the presence or absence of the object reflected in the curved mirror. The second degree of danger, which is the degree of danger, is determined.

As described above, the information processing apparatus 10B according to the third embodiment can determine the second risk level), which is the risk level when the vehicle travels, by using the curve mirror.

As described above, the second risk level, which is a risk level predicted when the vehicle is moving, is only from the characteristics of the installation location of the curved mirror reflected in the image that can be obtained from the vehicle that is temporarily stopped or in progress. May be determined.

Further, for example, when the vehicle during automatic driving is equipped with the information processing device 10B, the vehicle can determine the second danger level by using the curve mirror as described above, so that the determined second danger degree can be determined. Vehicle control can be performed according to the degree.

As described above, according to the information processing apparatus 10B or the system 1B according to the third embodiment, it is possible to support the safe driving of the vehicle by using the curve mirror in a place where the visibility is poor.

(Modification 1)
In the third embodiment, the characteristics of the installation location of the curved mirror, the presence or absence of an object reflected on the curved mirror, and the case of determining the second danger level, which is the predicted danger level when the vehicle travels, have been described. Not exclusively. The recognition unit 13B may recognize the attribute of the object reflected on the curve mirror, and the determination unit 104 may determine the degree of danger in consideration of the attribute. In this modification, a case where the degree of danger is determined by further considering the attribute related to the moving speed of the object reflected in the curved mirror will be described.

FIG. 60 is an explanatory diagram showing an example of a second risk determination process performed by the second risk determination unit 1043 in the first modification of the third embodiment. The same wording is described in the same contents as those in FIGS. 56 and 57, and detailed description thereof will be omitted. Further, since the vehicle control information of the own vehicle based on the second risk level of low, medium or high may be as shown in FIG. 56 or FIG. 57, the illustration is omitted in FIG. 60.

As shown in FIG. 60, when the object is a person, the second risk level determination unit 1043 may determine the second risk level similar to those in FIGS. 56 and 57. On the other hand, when the object is a bicycle, a motorcycle or a car whose moving speed is faster than that of a person, the second danger level determination unit 1043 has a second risk degree higher than that when the object is a person according to the moving speed. Should be determined.

As described above, in this modification, the second risk level determination unit 1043 determines the second risk level changed according to the attribute of the object recognized by the recognition unit 13B. As a result, the output processing unit 102B can generate driving support information according to the attributes of the object recognized by the recognition unit 13B, and can output the generated driving support information.

(Modification 2)
In the first modification, the case where the second danger level is determined by further considering the attribute related to the moving speed of the object reflected in the curved mirror has been described, but the present invention is not limited to this. When the object reflected in the curved mirror is a person, the second risk may be determined by further considering the attribute related to the person's age. This case will be described below as Modification 2.

FIG. 61 is an explanatory diagram showing an example of a second risk determination process performed by the second risk determination unit 1043 in the second modification of the third embodiment. The same wording is described in the same contents as those in FIGS. 56 and 57, and detailed description thereof will be omitted. Further, since the vehicle control information of the own vehicle based on the second degree of danger of low, medium or high may be as shown in FIG. 56 or FIG. 57, the illustration is omitted in FIG. 60 as well.

As shown in FIG. 61, when the object is a person and the person is a child or an elderly person, the second risk level determination unit 1043 determines the second risk level similar to that in FIGS. 56 and 57. do it. On the other hand, if the object is a person and the person is something other than a child or the elderly, the second risk determination unit 1043 will have a second higher than when the person is a child or the elderly. The degree of danger may be determined.

As described above, in this modification, when the attribute of the object recognized by the recognition unit 13B is a person, the second risk determination unit 1043 changes it according to the age of the object recognized by the recognition unit 13B. The second degree of danger is determined. As a result, when the attribute of the object recognized by the recognition unit 13B is a person, the output processing unit 102B generates and generates the driving support information by changing the driving support information according to the age of the object recognized by the recognition unit 13B. Driving support information can be output. In the above, the example in which the information regarding the age is the generation of the person has been described, but the information regarding the age may be the age or the age of the person.

(Modification 3)
In the second modification, when the object reflected in the curved mirror is a person, the case where the second degree of danger is determined by further considering the attribute related to the age of the person has been described, but the present invention is not limited to this. When the object reflected in the curve mirror is a person, the second danger level determination unit 1043 further considers the attribute of whether or not the person is walking while looking at a mobile terminal such as a smartphone. May be determined. Specifically, inattention behavior is non-forward vision behavior. For example, non-forward vision behavior includes gaze at a mobile terminal such as a smartphone or a book. This case will be described below as Modification 3.

FIG. 62 is an explanatory diagram showing an example of the second risk level determination process performed by the second risk level determination unit 1043 in the third modification of the third embodiment. The same wording is described in the same contents as those in FIGS. 56 and 57, and detailed description thereof will be omitted. Further, since the vehicle control information of the own vehicle based on the second risk level of low, medium or high may be as shown in FIG. 56 or FIG. 57, the illustration is omitted in FIG. 60.

As shown in FIG. 62, when the object is a person and the person does not perform a non-forward-looking behavior in which the person looks at the mobile terminal and performs a moving action, that is, the person does not walk while looking at the mobile terminal. The second risk level determination unit 1043 may determine the same second risk level as in FIGS. 56 and 57. On the other hand, when the object is a person and the person is not looking forward, that is, the person is walking while looking at the mobile terminal, the second risk determination unit 1043 is not the person. It suffices to determine the second risk level, which is higher than that when the person is not looking forward. In the above, an example in which the careless behavior is a non-forward vision behavior has been described, but the careless behavior is a behavior in front of the person but looking upward or downward, or a baby carriage or a ball in front of the person. It may be an action of gazing at a specific object.

As described above, in the present modification, if the attribute of the object recognized by the recognition unit 13B is a person, the second risk determination unit 1043 is inadvertently acting on the object recognized by the recognition unit 13B. The second degree of danger changed according to whether or not is determined. As a result, when the attribute of the object recognized by the recognition unit 13B is a person, the output processing unit 102B changes the driving support information according to whether or not the object recognized by the recognition unit 13B is inadvertently acting. It can be generated and output.

As described above, according to the information processing apparatus and information processing method of the present disclosure, it is possible to support safe driving of a vehicle by using a curved mirror installed in a place with poor visibility such as an intersection. can. In countries other than Japan, curved mirrors may not be widely used, but from the viewpoint of preventing traffic accidents, some equipment for preventing traffic accidents may be installed in places with poor visibility in the future. There is sex. Here, if the information processing apparatus or the like according to each embodiment of the present disclosure is used, it is possible to improve traffic safety with an inexpensive curved mirror without making the equipment to be arranged expensive equipment (for example, radar). Can be done.

In the above-described embodiment and the like, the shape of the curved mirror has been described as a circular shape or a rectangular shape, but the shape is not limited to this. The scope of the present disclosure also includes cases where the shape of the installed curved mirror is deformed for some reason and the surface of the mirror is uneven, or the mirror surface of the curved mirror is cloudy. In this case, reliability is introduced in the recognition of the object reflected in the curve mirror, and if the reliability is equal to or less than the threshold value, the information of the object reflected in the curve mirror is considered unreliable and the information processing method of the present disclosure is not used. May be good.

Although the information processing apparatus and the information processing method of the present disclosure have been described above based on the embodiments, the present invention is not limited to the embodiments. As long as it does not deviate from the gist of the present invention, one or more of the present embodiments may be modified by those skilled in the art, or may be constructed by combining components in different embodiments. It may be included in the range of the aspect of. For example, the following cases are also included in the present invention.

(1) Each of the above devices is specifically a computer system including a microprocessor, ROM, RAM, hard disk unit, display unit, keyboard, mouse and the like. A computer program is stored in the RAM or the hard disk unit. When the microprocessor operates according to the computer program, each device achieves its function. Here, a computer program is configured by combining a plurality of instruction codes indicating commands to a computer in order to achieve a predetermined function.

(2) A part or all of the components constituting each of the above devices may include one system LSI (Large Scale Integration). A system LSI is a super-multifunctional LSI manufactured by integrating a plurality of components on one chip, and specifically, is a computer system including a microprocessor, ROM, RAM, and the like. .. A computer program is stored in the RAM. When the microprocessor operates according to the computer program, the system LSI achieves its function.

(3) A part or all of the components constituting each of the above devices may be provided with a removable IC card or a single module in each device. The IC card or module is a computer system including a microprocessor, ROM, RAM, and the like. The IC card or the module may include the above-mentioned super multifunctional LSI. When the microprocessor operates according to a computer program, the IC card or the module achieves its function. This IC card or this module may have tamper resistance.

(4) The present disclosure may be the method shown above. Further, it may be a computer program that realizes these methods by a computer, or it may be a digital signal composed of the computer program.

(5) In the present disclosure, the computer program or a recording medium capable of reading the digital signal by a computer, for example, a flexible disk, a hard disk, a CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD ( It may be recorded on a Blu-ray (registered trademark) Disc), a semiconductor memory, or the like. Further, it may be the digital signal recorded on these recording media.

(6) Further, the present disclosure may transmit the computer program or the digital signal via a telecommunication line, a wireless or wired communication line, a network typified by the Internet, data broadcasting, or the like.

(7) Further, the present disclosure is a computer system including a microprocessor and a memory, in which the memory stores the computer program, and the microprocessor may operate according to the computer program. ..

(8) Further, another independent computer by recording and transferring the program or the digital signal on the recording medium, or by transferring the program or the digital signal via the network or the like. It may be carried out by the system.

The present disclosure can be used for an in-vehicle camera mounted on a vehicle for autonomous driving, a system such as CAN (Controller Area Network), an information processing device or system for providing driving support, and the like.

1, 1A, 1B system 2, 2a, 3 Vehicle 10, 10A, 10B Information processing device 11 Image acquisition unit 11B Acquisition unit 12, 12B Detection unit 12a, 12b Detection processing unit 13, 13B Recognition unit 50, 50a, 50 _j , 50 _{j + 1} , 50 _{j + 2} , 50 _{j + 3} , 50 _k , 50 _{k + 1} ,
50 _{k + 2} , 50 _{k + 3} , 50 _m , 50 _{m + 1} 50 _{m + 2} , 50 _{m + 3} , 50 _n , 50 _{n + 1} , 50 _{n + 2} images 51, 51 _m , 51 _{m + 1} , 51 _{m + 2} , 51 _{n + 1} , 51 _{n + 2} , 51', 51'n _{+ 1} , 51'n _{+ 2} , 71, 72, 73, 74, 75 Curved mirror 51a Edge 52 Mirror shape filter 52a Matching area 53, 53a Position 54, 54a Similarity map 60, 60 _m , 60 _{m + 1} , 60 _{m + 2} , 60 _{n + 1} , 60 _{n + 2} , 60', 60' _{n + 1} , 60'n _{+ 2} people 61 _{m + 1} , 61 _{m + 2} animal body 64, 64 _m , 64 _{m + 1} , 64 _{m + 2} object 80 road surface area 82 central axis 86 map 86 _t , 86 _t1 , 86 _t2 , 86 _t3 Local dynamic map 101, 101A Calculation unit 102, 102A, 102B Output processing unit 103 Blind spot determination unit 104 Judgment unit 111 Sensing unit 112 Image recording unit 860, 865 Commercial facility 861, 862, 864 Road 863 Expressway 866 Concert venue 1011 Difference calculation unit 1012, 1014 Object size calculation unit 1013 Object position calculation unit 1015 Road surface area calculation unit 1016 Central axis calculation unit 1031, 1031A Danger prediction unit 1032, 1032A, 1032B Information generation unit 1033, 1033A, 1033B Information output unit 1041 Features Judgment unit 1042 1st risk level judgment unit 1043 2nd risk level judgment unit

Claims (18)

An acquisition unit that acquires at least map information indicating the installation location of the curved mirror and image information indicating an image obtained by continuously photographing the front or rear of the vehicle in time series by an image pickup device mounted on the vehicle.
A detection unit that detects a curve mirror existing around the vehicle based on the map information and the image information, and a detection unit.
The map information or the feature including the mode of the road at the installation location, which is a feature of the installation location of the curve mirror, which is a feature that can predict whether or not the detected installation location of the curve mirror is a place where an accident is likely to occur. A feature determination unit that determines based on the image information,
A generator that generates driving support information for decelerating, stopping, or detouring the vehicle according to the degree of danger obtained by the determined characteristics.
It is equipped with an output unit that outputs the generated driving support information.
The aspect of the road is at least one of the degree of congestion of the road, the road surface condition of the road, and the presence or absence of construction of the road.
Information processing equipment. The map information includes the mode of the road at the installation location.
The detection unit detects the curve mirror based on the position of the vehicle and the installation location of the curve mirror indicated by the map information.
The feature determination unit determines the feature based on the map information around the detected curve mirror.
The information processing apparatus according to claim 1. The map information includes additional map information indicating at least one of traffic accident, traffic jam, construction, road surface condition and weather on the map.
The features include aspects of the road at the location of the curved mirror.
The feature determination unit determines the mode of the road from the additional map information, and determines the mode of the road.
The generation unit generates the driving support information according to the determined mode of the road.
The information processing apparatus according to claim 2. The detection unit detects the curve mirror based on the image information by using the identification information of the curve mirror or the discriminator of the curve mirror, recognizes the surrounding environment of the curve mirror, and recognizes the surrounding environment of the curve mirror.
The feature determination unit determines the feature based on the surrounding environment of the curve mirror.
The information processing apparatus according to any one of claims 1 to 3. The features include aspects of the road at the location of the curved mirror.
The feature determination unit determines the mode of the road from the surrounding environment of the curve mirror, and determines the mode of the road.
The generation unit generates the driving support information according to the determined mode of the road.
The information processing apparatus according to claim 4. The generation unit generates the driving support information for decelerating, stopping, or detouring the vehicle when the determined aspect of the road interferes with the safe running of the vehicle.
The information processing apparatus according to claim 3 or 5. The feature determination unit further determines the feature based on the object reflected in the curve mirror.
The information processing apparatus according to any one of claims 2 to 6. The feature determination unit generates the driving support information for decelerating, stopping, or detouring the vehicle depending on whether an object is reflected in the curved mirror or the amount of the object reflected in the curved mirror. ,
The information processing apparatus according to claim 7. The generation unit generates the driving support information according to the danger predicted from the characteristics.
The information processing apparatus according to any one of claims 1 to 8. The driving support information includes control command information of the behavior of the vehicle.
The information processing apparatus according to any one of claims 1 to 9. The driving support information includes presentation information presented to the occupants of the vehicle.
The information processing apparatus according to any one of claims 1 to 10. The presented information includes information presenting the hazards predicted from the characteristics at the location of the curved mirror.
The information processing apparatus according to claim 11. The information processing device is
A recognition unit for recognizing an object reflected on the curved mirror is provided.
The information processing apparatus according to any one of claims 1 to 12. The generation unit generates the driving support information according to the attribute of the object recognized by the recognition unit.
The information processing apparatus according to claim 13. When the attribute of the object recognized by the recognition unit is a person, the generation unit changes the driving support information according to the information regarding the age of the object recognized by the recognition unit.
The information processing apparatus according to claim 13. When the attribute of the object recognized by the recognition unit is a person, the generation unit changes the driving support information according to whether or not the object recognized by the recognition unit is inadvertently acting. Generate
The information processing apparatus according to claim 13. The processor acquires map information including at least the location of the curved mirror and image information indicating an image obtained by continuously taking a time-series image of the front or rear of the vehicle by an image pickup device mounted on the vehicle. ,
The processor detects the curved mirror existing around the vehicle based on the map information and the image information, and determines the curve mirror.
The processor can predict whether or not the detected location of the curved mirror is likely to cause an accident. The feature of the location of the curved mirror is the feature including the mode of the road at the location. Judgment based on map information or the image information
The processor generates driving assistance information for decelerating, stopping or detouring the vehicle according to the degree of danger obtained by the determined characteristics.
Output the generated driving support information and output
The aspect of the road is at least one of the degree of congestion of the road, the road surface condition of the road, and the presence or absence of construction of the road.
Information processing method. A computer-readable program that performs information processing methods
The map information including at least the installation location of the curved mirror and the image information indicating the image obtained by continuously photographing the front or the rear of the vehicle in time series by the image pickup device mounted on the vehicle are acquired.
Based on the map information and the image information, the curve mirror existing around the vehicle is detected, and the curve mirror is detected.
The map information or the feature including the mode of the road at the installation location, which is a feature of the installation location of the curve mirror, which is a feature that can predict whether or not the detected installation location of the curve mirror is a place where an accident is likely to occur. Judgment is made based on the image information.
The aspect of the road is at least one of the degree of congestion of the road, the road surface condition of the road, and the presence or absence of construction of the road.
The driving support information for decelerating, stopping or detouring the vehicle is generated according to the degree of danger obtained from the determined characteristics.
Output the generated driving support information,
program.

Images