JP7466776B2 - Notification control device, information distribution server, and notification control method - Google Patents

Description

The present disclosure relates to an alarm control device and an alarm control method that notify a driver of a dangerous situation that occurs while driving when the driver fails to recognize the situation, and an information distribution server that collects information from the alarm control device and distributes necessary information to the alarm control device.

Conventionally, technologies have been disclosed that notify drivers of various information when a dangerous situation occurs as a result of high-risk driving, but the driver is not aware of the dangerous situation (see, for example, Patent Documents 1 and 2).

JP 2007-65997 A JP 2010-15451 A

In conventional technology, when a danger avoidance action is taken by the autonomous driving of the vehicle itself or another vehicle while the driver is not aware of the occurrence of a dangerous situation, the driver is not aware that a dangerous situation has occurred. Here, a danger avoidance action refers to an action taken to avoid a dangerous situation when the dangerous situation occurs. Therefore, the driver may fall into a similar dangerous situation again. Thus, conventional technology has not been able to appropriately alert the driver to dangerous situations that the driver did not recognize.

The present disclosure has been made to solve such problems, and aims to provide an alarm control device, an information distribution server, and an alarm control method that can appropriately alert a driver to a dangerous situation that the driver has not recognized.

In order to solve the above problems, the notification control device according to the present disclosure includes a driver state recognition unit that recognizes the state of the driver of the host vehicle, a surrounding situation recognition unit that recognizes the surrounding situation of the host vehicle, a host vehicle driving information acquisition unit that acquires host vehicle driving information, which is information related to the driving of the host vehicle, a control unit that determines whether or not a danger avoidance action is performed through automatic driving of the host vehicle when a dangerous situation occurs while the host vehicle is traveling and the driver of the host vehicle is not aware of the dangerous situation, based on the state of the driver of the host vehicle recognized by the driver state recognition unit, the surrounding situation recognized by the surrounding situation recognition unit, and the host vehicle driving information acquired by the host vehicle driving information acquisition unit, and a notification unit that, when the control unit determines that a danger avoidance action is performed through automatic driving of the host vehicle when the driver of the host vehicle is not aware of the dangerous situation, notifies the driver of the host vehicle of the dangerous situation when a danger avoidance action is performed through automatic driving of the host vehicle.

According to the present disclosure, when a dangerous situation occurs while the vehicle is traveling and the driver of the vehicle is not aware of the dangerous situation, if it is determined that a danger avoidance action has been taken by the autonomous driving of the vehicle, the driver of the vehicle is notified of the dangerous situation at the time the danger avoidance action was taken by the autonomous driving of the vehicle, thereby making it possible to appropriately notify the driver of a dangerous situation that the driver did not recognize.

The objectives, features, aspects, and advantages of the present disclosure will become more apparent from the following detailed description and accompanying drawings.

1 is a block diagram showing an example of the configuration of a notification control device according to a first embodiment; 1 is a block diagram showing an example of the configuration of a notification control device according to a first embodiment; 4 is a flowchart showing an example of the operation of the notification control device according to the first embodiment. FIG. 11 is a diagram showing an example of notification according to the first embodiment. 13 is a block diagram showing an example of the configuration of a notification control device according to a second modification of the first embodiment. FIG. FIG. 13 is a diagram showing an example of notification according to the second modification of the first embodiment. FIG. 13 is a diagram showing an example of notification according to variant example 3 of embodiment 1. FIG. 13 is a diagram showing an example of notification according to variant example 4 of embodiment 1. FIG. 13 is a diagram showing an example of notification according to variant example 4 of embodiment 1. 11 is a block diagram showing an example of the configuration of a notification control device according to a second embodiment. FIG. 11 is a flowchart showing an example of the operation of the notification control device according to the second embodiment. 2 is a diagram illustrating an example of a hardware configuration of a notification control device according to the first embodiment. 2 is a diagram illustrating an example of a hardware configuration of a notification control device according to the first embodiment.

<First embodiment>
<Configuration>
Fig. 1 is a block diagram showing an example of the configuration of a notification control device 1 according to embodiment 1. Note that Fig. 1 shows the minimum necessary configuration of the notification control device according to embodiment 1.

The notification control device 1 includes a driver state recognition unit 2, a surrounding condition recognition unit 3, a vehicle driving information acquisition unit 4, an other vehicle driving information acquisition unit 5, a control unit 6, and a notification unit 7.

The driver state recognition unit 2 recognizes the state of the driver of the vehicle. The surrounding situation recognition unit 3 recognizes the surrounding situation of the vehicle.

The host vehicle driving information acquisition unit 4 acquires host vehicle driving information, which is information related to the driving of the host vehicle. The other vehicle driving information acquisition unit 5 acquires other vehicle driving information, which is information related to the driving of other vehicles.

The control unit 6 determines, based on the state of the driver of the host vehicle recognized by the driver state recognition unit 2, the surrounding conditions recognized by the surrounding condition recognition unit 3, the host vehicle driving information acquired by the host vehicle driving information acquisition unit 4, and the other vehicle driving information acquired by the other vehicle driving information acquisition unit 5, whether a danger avoidance action has been taken by the automatic driving of the host vehicle or a danger avoidance action has been taken by the other vehicle when a dangerous situation occurs while the host vehicle is traveling and the driver of the host vehicle is not aware of the dangerous situation.

When the control unit 6 determines that a danger avoidance operation has been performed by the autonomous driving of the vehicle itself or by another vehicle, the notification unit 7 notifies the driver of the vehicle itself of the dangerous situation when the danger avoidance operation has been performed by the autonomous driving of the vehicle itself or by another vehicle.

Next, other configurations of the notification control device including the notification control device 1 shown in Figure 1 will be described.

Figure 2 is a block diagram showing an example of the configuration of an alarm control device 8 relating to another configuration.

The notification control device 8 includes a driver state recognition unit 2, a surrounding situation recognition unit 3, a vehicle driving information acquisition unit 4, another vehicle driving information acquisition unit 5, a control unit 6, a notification unit 7, and a situation memory unit 9. The notification control device 8 is connected to a driver state detection device 10, a surrounding situation detection device 11, a photographing device 12, an automatic driving control device 13, a communication device 14, and a notification device 15. The notification control device 8, the driver state detection device 10, the surrounding situation detection device 11, the photographing device 12, the automatic driving control device 13, the communication device 14, and the notification device 15 are mounted on the vehicle. The vehicle is not limited to a drive system such as an engine-driven vehicle, an electric vehicle, or a fuel cell vehicle.

The notification control device 8 can be applied not only to in-vehicle navigation devices, i.e., car navigation devices, but also to navigation devices constructed as a system by appropriately combining a PND (Portable Navigation Device) that can be mounted on a vehicle and a server provided outside the vehicle, or to devices other than navigation devices constructed as a system. In this case, each function or each component of the notification control device 8 is distributed and arranged among the functions that construct the above-mentioned system.

The driver state recognition unit 2 obtains the detection result of the state of the driver of the host vehicle from the driver state detection device 10, and recognizes the state of the driver of the host vehicle based on the detection result. Furthermore, when a dangerous situation occurs while the host vehicle is traveling, the driver state recognition unit 2 determines whether or not the driver of the host vehicle has recognized the dangerous situation. When determining whether or not the driver of the host vehicle has recognized the dangerous situation, the driver state recognition unit 2 may use not only the detection result obtained from the driver state detection device 10, but also the detection result obtained from the surrounding situation detection device 11.

For example, the driver state recognition unit 2 recognizes that the driver of the vehicle has recognized a dangerous situation when another vehicle is present in the line of sight of the driver of the vehicle, based on the line of sight of the driver of the vehicle detected by the driver state detection device 10 and the relative position of the other vehicle present around the vehicle detected by the surrounding situation detection device 11. At this time, the other vehicle detected by the surrounding situation detection device 11 is the vehicle involved in the dangerous situation when the dangerous situation occurs. On the other hand, when there is no other vehicle in the line of sight of the driver of the vehicle, it recognizes that the driver of the vehicle does not recognize the dangerous situation. Note that the line of sight of the driver of the vehicle detected by the driver state detection device 10 includes not only the direct line of sight, which is the actual line of sight, but also the line of sight when looking at the rearview mirror or electronic mirror.

The driver state detection device 10 is, for example, a DMS (Driver Monitoring System) and detects the state of the driver of the vehicle. The driver state detection device 10 detects the state of the driver of the vehicle while driving, such as the driver's line of sight, alertness (physiology), facial expression (psychology), and line of sight. The driver state detection device 10 outputs the detection result of the driver's state of the vehicle to the driver state recognition unit 2.

The surrounding situation recognition unit 3 acquires the detection result of the surrounding situation of the host vehicle from the surrounding situation detection device 11, and recognizes the surrounding situation of the host vehicle based on the detection result. The surrounding situation recognition unit 3 also recognizes a dangerous situation that occurs while the host vehicle is traveling based on the detection result acquired from the surrounding situation detection device 11.

The surrounding condition detection device 11 detects the surrounding conditions of the vehicle. The surrounding condition detection device 11 detects various surrounding conditions related to traffic conditions, including, for example, the distance and relative positional relationship between the vehicle and other vehicles.

The host vehicle driving information acquisition unit 4 acquires host vehicle driving information, which is information related to the driving of the host vehicle. Specifically, the host vehicle driving information acquisition unit 4 acquires information related to the automatic driving of the host vehicle from the automatic driving control device 13 as the host vehicle driving information. The host vehicle driving information acquisition unit 4 also acquires information related to manual driving by the driver of the host vehicle as the host vehicle driving information.

The automatic driving control device 13 controls the driving control system of the host vehicle to perform automatic driving. In addition, when a dangerous situation occurs while the host vehicle is traveling, the automatic driving control device 13 outputs information to the host vehicle driving information acquisition unit 4 that a danger avoidance operation has been performed by driving operation intervention. Here, the danger avoidance operation by driving operation intervention means that when a dangerous situation occurs during manual driving by the driver of the host vehicle, the automatic driving control device 13 automatically switches from manual driving by the driver to automatic driving by the automatic driving control device 13 to perform a danger avoidance operation. In addition to the danger avoidance operation by driving operation intervention, the automatic driving control device 13 may perform a danger avoidance operation when a dangerous situation occurs during automatic driving by the automatic driving control device 13. In this case, the automatic driving control device 13 outputs information to the host vehicle driving information acquisition unit 4 that a danger avoidance operation has been performed. In this way, the host vehicle driving information includes information indicating that a danger avoidance operation has been performed by automatic driving of the host vehicle and information indicating that a danger avoidance operation has been performed by manual driving of the driver of the host vehicle.

The other vehicle driving information acquisition unit 5 acquires other vehicle driving information, which is information related to the driving of the other vehicle. Specifically, the other vehicle driving information acquisition unit 5 acquires other vehicle driving information, which is information related to the driving of the other vehicle, from the other vehicle via the communication device 14. The other vehicle driving information includes information indicating that a danger avoidance operation has been performed by the other vehicle. Examples of danger avoidance operations by the other vehicle include danger avoidance operations by automatic driving of the other vehicle and danger avoidance operations by the driver of the other vehicle. Here, the danger avoidance operation by automatic driving of the other vehicle refers to an operation in which the other vehicle avoids a dangerous situation by automatic driving under the control of an automatic driving control device mounted on the other vehicle. The danger avoidance operation by the driver of the other vehicle refers to an operation in which the other vehicle avoids a dangerous situation by manually driving the other vehicle.

The control unit 6 determines, based on the state of the driver of the host vehicle recognized by the driver state recognition unit 2, the surrounding conditions recognized by the surrounding condition recognition unit 3, the host vehicle driving information acquired by the host vehicle driving information acquisition unit 4, and the other vehicle driving information acquired by the other vehicle driving information acquisition unit 5, whether a danger avoidance action has been taken by the automatic driving of the host vehicle or a danger avoidance action has been taken by the other vehicle when a dangerous situation occurs while the host vehicle is traveling and the driver of the host vehicle is not aware of the dangerous situation.

Specifically, when the surrounding situation recognition unit 3 recognizes that a dangerous situation has occurred while the host vehicle is traveling, the control unit 6 determines that a dangerous situation has occurred while the host vehicle is traveling. In addition, when the driver state recognition unit 2 recognizes that the driver of the host vehicle is not aware of the dangerous situation, the control unit 6 determines that the driver of the host vehicle is not aware of the dangerous situation.

Furthermore, the control unit 6 determines whether a danger avoidance operation has been performed by the autonomous driving of the own vehicle or a danger avoidance operation by another vehicle based on the own vehicle driving information acquired by the own vehicle driving information acquisition unit 4 and the other vehicle driving information acquired by the other vehicle driving information acquisition unit 5.

The danger avoidance operation by the automatic driving of the host vehicle refers to an operation in which, when a dangerous situation occurs, the dangerous situation is avoided by the automatic driving under the control of the automatic driving control device 13. The danger avoidance operation by the automatic driving of the host vehicle is assumed to be performed, for example, in the following situations. Note that the following is merely an example, and is not limited to the following as long as the danger avoidance operation by the automatic driving of the host vehicle is performed.
- When the distance between the vehicle and another object (a moving body including another vehicle, or a non-moving body) becomes less than a predetermined distance, the automatic driving control device 13 automatically activates at least one of the brake, accelerator, and steering operations to avoid a dangerous situation.
When the vehicle is about to run a red light, the automatic driving control device 13 automatically performs at least one of braking, accelerating, and steering to avoid a dangerous situation.
- When the driver of the vehicle is opening the door and a moving object is approaching from behind, and it is predicted that the moving object will collide with the door or the driver of the vehicle if the situation continues, the automatic driving control device 13 automatically activates at least one of the brake, accelerator, and steering operations to avoid a dangerous situation.

In addition, the host vehicle driving information acquisition unit 4 may acquire information regarding the autonomous driving of the host vehicle by a method other than acquiring the host vehicle driving information from the autonomous driving control device 13. For example, the control unit 6 may compare information regarding the driver's operation of the actuator with information from a speed sensor and an acceleration sensor that detect the physical movement of the host vehicle, and determine that the host vehicle is traveling autonomously if the host vehicle performs at least one of steering, accelerating, and braking even though the driver has not operated the actuator. In this case, the host vehicle driving information acquisition unit 4 acquires information regarding the driver's operation of the actuator, and information from a speed sensor and an acceleration sensor that detect the physical movement of the host vehicle.

The danger avoidance action by another vehicle refers to an action taken by another vehicle to avoid a dangerous situation when a dangerous situation occurs. The danger avoidance action by another vehicle is assumed to be taken, for example, in the following situations. Note that the following is merely an example, and the situation is not limited to the following as long as the danger avoidance action is taken by the other vehicle.
- When the distance between your vehicle and another vehicle falls below a predetermined distance, the other vehicle uses autonomous driving to avoid a dangerous situation.
When the distance between the vehicle and another vehicle becomes less than a predetermined distance, the driver of the other vehicle avoids a dangerous situation.
- When a collision between the vehicle and another vehicle other than the above is expected, a dangerous situation is avoided by the automatic driving of the other vehicle or the driver of the other vehicle. An example of a collision between the vehicle and another vehicle is expected when the distance between the vehicle and the other vehicle is still far, but the driver of the other vehicle is asleep, and there is a high possibility of a collision if the situation continues. The state of the driver of the other vehicle can be detected by image recognition of the image of the driver of the other vehicle captured by the surrounding situation detection device 11 or the image capture device 12. Another example of a collision between the vehicle and the other vehicle is expected when the distance between the vehicle and the other vehicle is still far, but the other vehicle is in a runaway state.
- When a collision between the driver of the vehicle and another vehicle is anticipated, the dangerous situation is avoided by the autonomous driving of the other vehicle or by the driver of the other vehicle.
- When a collision between your vehicle and the driver of another vehicle is anticipated, and a dangerous situation is avoided by the autonomous driving of the other vehicle or the driver of the other vehicle.

The other vehicle driving information acquisition unit 5 may acquire information related to the driving of the other vehicle by a method other than acquiring the other vehicle driving information from the other vehicle via the communication device 14. For example, the control unit 6 may estimate the driving behavior of the other vehicle based on the surrounding conditions detected by the surrounding condition detection device 11. The control unit 6 may also detect the behavior of the driver of the other vehicle by performing image recognition on an image of the driver of the other vehicle captured by the surrounding condition detection device 11 or the image capture device 12, and determine that the other vehicle is performing automatic driving if the driver of the other vehicle is not operating an actuator.

Returning to the explanation of FIG. 2, the situation memory unit 9 stores video of a dangerous situation captured by the imaging device 12. The video captured by the imaging device 12 is stored in the situation memory unit 9 in a format that can be played on a television or video and audio playback software. The video may be subjected to processing such as image processing, and audio may be stored in the situation memory unit 9 along with the video. The situation memory unit 9 may be provided outside the alarm control device 8. In this case, the situation memory unit 9 and the imaging device 12 may be configured as one unit, for example like a drive recorder. The situation memory unit 9 may also store the surrounding situation of the vehicle recognized by the surrounding situation recognition unit 3.

The image capturing device 12 captures images of dangerous situations that occur while the vehicle is traveling. The image capturing device 12 and the surrounding situation detection device 11 may be configured as one unit.

When the control unit 6 determines that a danger avoidance operation has been performed by the autonomous driving of the vehicle itself or by another vehicle, the notification unit 7 notifies the driver of the vehicle itself of a dangerous situation when the danger avoidance operation has been performed by the autonomous driving of the vehicle itself or by another vehicle. Specifically, the notification unit 7 notifies the driver of the vehicle itself of the dangerous situation stored in the situation memory unit 9 using the notification device 15.

The alarm device 15 notifies of a dangerous situation according to instructions from the alarm unit 7. Examples of the alarm device 15 include a head up display (HUD), a center information display (CID), a light emitting diode (LED), a speaker, and a device that vibrates the seat, but any alarm means may be used as long as it can notify the driver of the vehicle.

<Operation>
Fig. 3 is a flowchart showing an example of the operation of the notification control device 8. In Fig. 3, it is assumed that the host vehicle is traveling when the operation starts.

In step S11, the surrounding condition recognition unit 3 obtains the detection results of the surrounding conditions of the vehicle from the surrounding condition detection device 11, and recognizes the surrounding conditions of the vehicle based on the detection results.

In step S12, the driver state recognition unit 2 obtains the detection result of the state of the driver of the vehicle from the driver state detection device 10, and recognizes the state of the driver of the vehicle based on the detection result.

In step S13, the control unit 6 determines whether or not driving of the host vehicle has ended. For example, the control unit 6 may determine that driving of the host vehicle has ended when the engine is turned off if the host vehicle is an engine-driven vehicle, or when the motor power source for driving the vehicle is turned off if the host vehicle is an electric vehicle. The control unit 6 may also determine that driving of the host vehicle has ended when the host vehicle arrives at a preset destination. If driving of the host vehicle has ended, the process proceeds to step S18. On the other hand, if driving of the host vehicle has not ended, the process proceeds to step S14.

In step S14, the control unit 6 determines whether or not a danger avoidance operation has been performed by the automatic driving of the host vehicle or by the other vehicle, based on the host vehicle driving information acquired by the host vehicle driving information acquisition unit 4 and the other vehicle driving information acquired by the other vehicle driving information acquisition unit 5. If a danger avoidance operation has been performed by the automatic driving of the host vehicle or by the other vehicle, the process proceeds to step S15. On the other hand, if a danger avoidance operation has not been performed by the automatic driving of the host vehicle or by the other vehicle, the process returns to step S11.

In step S15, the control unit 6 determines whether the driver of the host vehicle has performed a danger avoidance operation. Specifically, when the host vehicle driving information acquisition unit 4 acquires information indicating that the driver of the host vehicle has performed a danger avoidance operation by manual driving, the control unit 6 determines that the driver of the host vehicle has performed a danger avoidance operation. Here, since it is clear that the driver of the host vehicle has not performed a danger avoidance operation when a danger avoidance operation is performed by another vehicle, the processing of step S15 is executed only when a danger avoidance operation is performed by the host vehicle. In other words, when a danger avoidance operation is performed by another vehicle, the processing of step S15 is omitted and the process proceeds to step S16. If the driver of the host vehicle has not performed a danger avoidance operation, the process proceeds to step S16. On the other hand, if the driver of the host vehicle has performed a danger avoidance operation, the process returns to step S11.

In step S16, the driver state recognition unit 2 determines whether or not the driver of the host vehicle is aware of the dangerous situation based on the detection result of the driver's state of the host vehicle obtained from the driver state detection device 10. If the driver of the host vehicle is not aware of the dangerous situation, the process proceeds to step S17. At this time, the driver state recognition unit 2 notifies the control unit 6 that the driver of the host vehicle is not aware of the dangerous situation. On the other hand, if the driver of the host vehicle is aware of the dangerous situation, the process returns to step S11.

In step S17, the situation memory unit 9 stores the video of the dangerous situation captured by the image capture device 12 in accordance with the instructions of the control unit 6. At this time, the situation memory unit 9 may store the dangerous situation recognized by the surrounding situation recognition unit 3.

In step S18, if it is determined in step S13 that driving of the vehicle has ended, the notification unit 7 notifies the driver of the vehicle of the dangerous situation stored in the situation memory unit 9 in accordance with instructions from the control unit 6. For example, as shown in FIG. 4, the notification content may be displayed on the HUD, which is the notification device 15. In the example of FIG. 4, it is notified that a danger avoidance operation has been detected. It is also notified that the danger avoidance operation is a sudden steering operation of another vehicle. In FIG. 4, the avoidance category indicates what entity performed what danger avoidance operation.

The timing of the notification to the driver of the vehicle is not limited to the end of driving the vehicle. For example, the notification may be made after a predetermined time has elapsed since the occurrence of a dangerous situation, or when the driving load around the vehicle is reduced, for example, when the density of other vehicles around the vehicle is equal to or less than a predetermined density, or when there is a sufficient distance between the vehicle and other vehicles. The notification may also be made when the driver starts driving again after the driving operation has been terminated.

<Modification 1>
In the above, the case where both the vehicle driving information and the other vehicle driving information are acquired has been described, but only one of the information may be acquired. For example, when only the vehicle driving information is acquired, the other vehicle driving information acquisition unit 5 and the communication device 14 shown in FIG. 2 are not required.

<Modification 2>
FIG. 5 is a block diagram showing an example of the configuration of the notification control device 16 according to the second modification.

The notification control device 16 is characterized by having a location information acquisition unit 17 and a map information acquisition unit 18. The location information acquisition unit 17 is connected to a positioning device 19, and the map information acquisition unit 18 is connected to a map DB (Database) storage device 20. The positioning device 19 and the map DB storage device 20 are mounted on the vehicle. Other configurations and basic operations are similar to those of the notification control device 8 shown in Figure 2, so detailed explanations will be omitted here.

The position information acquisition unit 17 acquires the current position information of the vehicle from the positioning device 19. The positioning device 19 receives signals from, for example, a GPS (Global Positioning System) satellite to determine the current position of the vehicle.

The map information acquisition unit 18 acquires map information, including the roads on which the vehicle is traveling, from the map DB storage device 20. The map DB storage device 20 stores map information including roads. The map DB storage device 20 may be provided outside the vehicle.

The situation memory unit 9 stores location information where a dangerous situation has occurred based on the current location information of the vehicle acquired by the location information acquisition unit 17 and the map information acquired by the map information acquisition unit 18. The situation memory unit 9 stores the location information where a dangerous situation has occurred in association with at least one of the video of the dangerous situation captured by the image capture device 12 and the dangerous situation recognized by the surrounding situation recognition unit 3.

As shown in Fig. 6, for example, the notification unit 7 displays a map including the location where a dangerous situation occurred, i.e., the location where a danger avoidance action was taken by the autonomous driving of the vehicle or another vehicle, on the notification device 15. Note that, in the example of Fig. 6, the notification device 15 is a CID, but is not limited to a CID.

In addition to the above, a map including marking objects indicating the location where a dangerous situation has occurred may be displayed on the alarm device 15, and when any marking object is selected, a display such as that shown in Figure 6 or an image of the dangerous situation may be displayed.

In addition to storing information about a single driving session, dangerous situations that occurred going back a predetermined number of days or a predetermined distance may be stored, and information such as the date and time when dangerous situations occurred in the past may be added to the report.

<Modification 3>
As shown in Fig. 7, the positional relationship between the host vehicle and another vehicle, and whether a danger avoidance action has been taken by the host vehicle or another vehicle may be notified. In Fig. 7, the vehicle object indicates the host vehicle, and the hatched areas indicate the positions of the other vehicles.

Specifically, the control unit 6 determines whether the host vehicle or the other vehicle has performed a danger avoidance operation based on a time series of the relative positional relationship between the host vehicle and the other vehicle recognized by the surrounding situation recognition unit 3.

Based on the recognition results of the surrounding situation recognition unit 3, it is determined whether a danger avoidance operation has been performed by the autonomous driving of the other vehicle or by the driver of the other vehicle. Whether or not a danger avoidance operation has been performed by the autonomous driving of the host vehicle can be determined from the host vehicle driving information acquired by the host vehicle driving information acquisition unit 4.

In the example of Figure 7, the alarm device 15 is shown as a HUD, but is not limited to a HUD.

<Modification 4>
As shown in Fig. 8, a pattern of past dangerous situations may be notified. Specifically, the control unit 6 generates a pattern of past dangerous situations not recognized by the driver of the vehicle (a pattern in which a dangerous situation not recognized by the driver of the vehicle is likely to occur) based on a plurality of dangerous situations not recognized by the driver of the vehicle in the past.

In FIG. 8, "Data," "Analysis Results," and "Patterns in which dangerous situations are likely to occur" indicate, for example, operational transitions from a menu screen. In the example of FIG. 8, patterns in which dangerous situations are likely to occur are shown in a ranking format, but this is not limiting.

As shown in Fig. 9, the notification unit 7 may notify the driver that a situation in which danger is likely to occur while the vehicle is traveling has occurred. In other words, the notification unit 7 may notify the driver that the current traveling situation of the vehicle is the same as the pattern of a past dangerous situation.

Note that in the examples of Figures 8 and 9, the alarm device 15 is shown as a CID, but is not limited to a CID.

<Modification 5>
In the above, a case has been described in which the driver state recognition unit 2 determines whether or not the driver of the vehicle is aware of a dangerous situation based on the line of sight of the driver of the vehicle, but this is not limited to this.

The driver state recognition unit 2 may determine whether the driver of the vehicle is aware of a dangerous situation based on other information such as the degree of arousal (physiology), facial expression (psychology), and gaze. For example, the driver state recognition unit 2 may determine that the driver of the vehicle is aware of a dangerous situation when the driver's facial expression shows fear or surprise, or when the driver's palpitations or cold sweat are recognized. In addition, the driver state recognition unit 2 may determine that the driver of the vehicle is aware of a dangerous situation when the driver's limbs are twitching, or when the driver closes his or her eyes.

<Modification 6>
The driver state recognition unit 2 may determine whether the driver of the host vehicle recognizes the dangerous situation based on the actions of the driver of the host vehicle when the dangerous situation occurs. The actions of the driver of the host vehicle include a danger avoidance action by the driver of the host vehicle and an action by the driver of the host vehicle attempting to avoid the danger. When the driver of the host vehicle is performing an action to avoid the danger, the driver state recognition unit 2 may determine that the driver recognizes the dangerous situation based on the actions of the driver up to the actuator operation.

＜Effects＞
From the above, according to the first embodiment, it is possible to notify the driver that a dangerous situation has occurred when the driver does not recognize that a dangerous situation has been avoided by a danger avoidance operation by the autonomous driving of the vehicle itself or a danger avoidance operation by another vehicle. In other words, it is possible to appropriately notify the driver of a dangerous situation that the driver did not recognize.

<Embodiment 2>
<Configuration>
FIG. 10 is a block diagram showing an example of the configuration of a notification control device according to the second embodiment.

As shown in Fig. 10, the notification control device 21 is characterized by having a communication unit 22. The rest of the configuration and basic operation are similar to those of the notification control device 16 shown in Fig. 5, so detailed explanations are omitted here.

The communication unit 22 transmits the dangerous situation stored in the situation memory unit 9 as danger information to the information distribution server 23 via the communication device 14. The communication unit 22 has an other vehicle driving information acquisition unit 5.

The information distribution server 23 collects danger information transmitted from the notification control device 21 and distributes the collected danger information to the notification control device 21 as appropriate.

Note that the notification control device 21 shown in Figure 10 is based on the notification control device 16 shown in Figure 5, but may also be based on the notification control device 8 shown in Figure 2.

<Operation>
Fig. 11 is a flowchart showing an example of the operation of the notification control device 21. Note that the processes other than step S28 in Fig. 11 are similar to steps S11 to S18 in Fig. 3, and therefore will not be described here. Here, step S28 will be described.

In step S28, the communication unit 22 transmits the dangerous situation stored in the situation memory unit 9 as danger information to the information distribution server 23 via the communication device 14. Specifically, the communication unit 22 converts the dangerous situation stored in the situation memory unit 9 into an appropriate format according to an instruction from the control unit 6, and transmits the converted information as danger information to the information distribution server 23. The danger information may include only video and audio related to the dangerous situation, but may also include location information of the location where the dangerous situation occurred and detailed information of the dangerous situation (such as the object, the cause of occurrence, the time of occurrence, and the state of the driver of the vehicle).

The information distribution server 23 collects and manages the danger information transmitted from the notification control device 21. The danger information managed by the information distribution server 23 may be viewable from outside, for example, by the driver of the vehicle.

<Modification 1>
Based on the collected danger information, the information distribution server 23 may create a danger map showing on a map the location where a dangerous situation occurred without the driver of the vehicle being aware of it (the location where a danger avoidance action was taken by the automatic driving of the vehicle or by another vehicle), and distribute the created danger map to the outside (e.g., the alarm control device 21).

<Modification 2>
The information distribution server 23 may store the vehicle equipped with the notification control device 21 that transmitted the danger information as a vehicle requiring caution.

The information distribution server 23 may notify vehicles in the vicinity of the vehicle requiring attention to keep a distance from the vehicle requiring attention, or may perform automatic driving control to keep a distance from the vehicle requiring attention. Methods for keeping a distance from the vehicle requiring attention include, for example, keeping a distance of at least a predetermined distance between the vehicles, or presenting an alternative route to move away from the vehicle requiring attention.

＜Effects＞
As described above, according to the second embodiment, in addition to the effects of the first embodiment, it becomes possible to manage a greater number of dangerous situations in more detail. Furthermore, it becomes possible to view the information managed in this way from outside.

<Hardware Configuration>
The functions of the driver state recognition unit 2, the surrounding situation recognition unit 3, the vehicle driving information acquisition unit 4, the other vehicle driving information acquisition unit 5, the control unit 6, and the notification unit 7 in the notification control device 8 described in the first embodiment are realized by a processing circuit. That is, the notification control device 8 recognizes the state of the driver of the vehicle, recognizes the surrounding situation of the vehicle, acquires the vehicle driving information, acquires the other vehicle driving information, and when a dangerous situation occurs during the driving of the vehicle and the driver of the vehicle does not recognize the dangerous situation, determines whether or not a danger avoidance operation due to the automatic driving of the vehicle or a danger avoidance operation by the other vehicle has been performed, and when it is determined that a danger avoidance operation due to the automatic driving of the vehicle or a danger avoidance operation by the other vehicle has been performed, the notification control device 8 includes a processing circuit for notifying the driver of the vehicle of a dangerous situation when a danger avoidance operation due to the automatic driving of the vehicle or a danger avoidance operation by the other vehicle has been performed. The processing circuit may be dedicated hardware or may be a processor (also called a CPU, a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor)) that executes a program stored in a memory.

12, the processing circuit 24 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. Each function of the driver state recognition unit 2, the surrounding situation recognition unit 3, the vehicle driving information acquisition unit 4, the other vehicle driving information acquisition unit 5, the control unit 6, and the notification unit 7 may be realized by the processing circuit 24, or each function may be realized by a single processing circuit 24.

When the processing circuit 24 is the processor 25 shown in FIG. 13, the functions of the driver state recognition unit 2, the surrounding situation recognition unit 3, the vehicle driving information acquisition unit 4, the other vehicle driving information acquisition unit 5, the control unit 6, and the notification unit 7 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 26. The processor 25 realizes each function by reading and executing the program recorded in the memory 26. That is, the notification control device 8 includes a memory 26 for storing a program that results in the execution of a step of recognizing the state of the driver of the vehicle, a step of recognizing the surrounding situation of the vehicle, a step of acquiring the vehicle driving information, a step of acquiring the other vehicle driving information, a step of determining whether or not a danger avoidance operation by the automatic driving of the vehicle or a danger avoidance operation by the other vehicle has been performed when a dangerous situation occurs during the driving of the vehicle and the driver of the vehicle does not recognize the dangerous situation, and a step of notifying the driver of the vehicle of the dangerous situation when the danger avoidance operation by the automatic driving of the vehicle or the danger avoidance operation by the other vehicle has been performed when it is determined that the danger avoidance operation by the automatic driving of the vehicle or the danger avoidance operation by the other vehicle has been performed. It can also be said that these programs cause the computer to execute the procedures or methods of the driver state recognition unit 2, the surrounding situation recognition unit 3, the vehicle driving information acquisition unit 4, the other vehicle driving information acquisition unit 5, the control unit 6, and the notification unit 7. Here, the memory may be, for example, a non-volatile or volatile semiconductor memory such as a random access memory (RAM), a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM), or an electrically erasable programmable read only memory (EEPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a digital versatile disk (DVD), or any storage medium to be used in the future.

In addition, with regard to each function of the driver state recognition unit 2, surrounding condition recognition unit 3, host vehicle driving information acquisition unit 4, other vehicle driving information acquisition unit 5, control unit 6, and notification unit 7, some functions may be realized by dedicated hardware and other functions may be realized by software or firmware.

Thus, the processing circuitry can realize each of the above-mentioned functions through hardware, software, firmware, or a combination of these.

Note that while the above describes the hardware configuration of the notification control device 8 described in embodiment 1, the same applies to the hardware configuration of the notification control device 16 shown in Figure 5 and the hardware configuration of the notification control device 21 shown in Figure 10.

Within the scope of this disclosure, it is possible to freely combine the various embodiments, or to modify or omit the various embodiments as appropriate.

Although the present disclosure has been described in detail, the above description is illustrative in all respects and is not limiting. It is understood that countless variations not illustrated can be envisioned.

1 Notification control device, 2 Driver state recognition unit, 3 Surrounding situation recognition unit, 4 Vehicle driving information acquisition unit, 5 Other vehicle driving information acquisition unit, 6 Control unit, 7 Notification unit, 8 Notification control device, 9 Situation memory unit, 10 Driver state detection device, 11 Surrounding situation detection device, 12 Photography device, 13 Automatic driving control device, 14 Communication device, 15 Notification device, 16 Notification control device, 17 Position information acquisition unit, 18 Map information acquisition unit, 19 Positioning device, 20 Map DB storage device, 21 Notification control device, 22 Communication unit, 23 Information distribution server, 24 Processing circuit, 25 Processor, 26 Memory.

Claims (22)

A driver state recognition unit that recognizes a state of a driver of the vehicle;
A surrounding situation recognition unit that recognizes a surrounding situation of the host vehicle;
a host vehicle driving information acquisition unit that acquires host vehicle driving information, which is information regarding the driving of the host vehicle;
a control unit that determines whether or not a danger avoidance operation has been performed by automatic driving of the host vehicle when a dangerous situation occurs while the host vehicle is traveling and the driver of the host vehicle is not aware of the dangerous situation, based on the state of the driver of the host vehicle recognized by the driver state recognition unit, the surrounding situation recognized by the surrounding situation recognition unit, and the host vehicle driving information acquired by the host vehicle driving information acquisition unit;
a notification unit that notifies the driver of the host vehicle of the dangerous situation when the control unit determines that a danger avoidance operation has been performed by the automatic driving of the host vehicle when the driver of the host vehicle does not recognize the dangerous situation ; and
An alarm control device comprising: Further, a vehicle driving information acquisition unit that acquires vehicle driving information, which is information related to the driving of the vehicle, is provided.
The control unit determines, based on the other vehicle driving information acquired by the other vehicle driving information acquisition unit, whether or not a danger avoidance operation has been performed by automatic driving of the other vehicle when a dangerous situation occurs during traveling of the host vehicle and the driver of the host vehicle does not recognize the dangerous situation;
The notification control device described in claim 1, wherein when the control unit determines that a danger avoidance operation is performed by automatic driving of the other vehicle when the driver of the host vehicle is not aware of the dangerous situation , the notification unit notifies the driver of the host vehicle of the dangerous situation when the danger avoidance operation is performed by the other vehicle. Further, a vehicle driving information acquisition unit that acquires vehicle driving information, which is information related to the driving of the vehicle, is provided.
The control unit determines whether or not a risk avoidance operation is performed by a driver of the other vehicle when a dangerous situation occurs during traveling of the host vehicle and the driver of the host vehicle does not recognize the dangerous situation, based on the other vehicle driving information acquired by the other vehicle driving information acquisition unit;
2. The notification control device according to claim 1, wherein when the control unit determines that a danger avoidance action has been taken by the driver of the other vehicle when the driver of the own vehicle is not aware of the dangerous situation , the notification unit notifies the driver of the own vehicle of the dangerous situation at the time the danger avoidance action was taken by the other vehicle. A situation memory unit that stores the dangerous situation when a danger avoidance operation is performed by the autonomous driving of the host vehicle,
The notification control device according to claim 1 , wherein the notification unit notifies the dangerous situation stored in the situation storage unit. a situation storage unit configured to store the dangerous situation when the danger avoidance operation is performed by the other vehicle,
The notification control device according to claim 2 , wherein the notification unit notifies the user of the dangerous situation stored in the situation storage unit. The alarm control device according to claim 4, wherein the dangerous situation includes a video image. The alarm control device according to claim 5, wherein the dangerous situation includes a video image. a position information acquisition unit that acquires current position information of the vehicle;
a map information acquisition unit that acquires map information including a road on which the host vehicle is traveling;
Further equipped with
The notification control device according to claim 1 , wherein the notification unit displays, on a map, a location where a danger avoidance operation is performed by the autonomous driving of the host vehicle. a position information acquisition unit that acquires current position information of the vehicle;
a map information acquisition unit that acquires map information including a road on which the host vehicle is traveling;
Further equipped with
The notification control device according to claim 2 , wherein the notification unit displays, on a map, a position where the other vehicle has performed a danger avoidance operation. The notification control device according to claim 2 or 3, wherein the notification unit notifies the relative position of the other vehicle with respect to the host vehicle, and whether a danger avoidance action has been taken by the host vehicle through autonomous driving or by the other vehicle. The control unit generates a pattern of past dangerous situations not recognized by the driver of the host vehicle based on a plurality of dangerous situations not recognized by the driver of the host vehicle in the past;
The notification control device according to claim 1 , wherein the notification unit notifies a pattern of the past dangerous situations. The notification control device according to claim 11, wherein the notification unit notifies the driver that the current driving conditions of the vehicle are the same as the pattern of the past dangerous conditions. The state of the driver of the vehicle includes at least one of psychology, physiology, facial expression, and gaze of the driver of the vehicle;
The notification control device according to claim 1 , wherein the driver state recognition unit determines whether or not the driver of the host vehicle is aware of the dangerous situation based on a state of the driver of the host vehicle. The state of the driver of the host vehicle includes an action of the driver of the host vehicle,
The notification control device according to claim 1 , wherein the driver state recognition unit determines whether or not the driver of the host vehicle is aware of the dangerous situation based on an action of the driver of the host vehicle when the dangerous situation occurs. The notification control device according to claim 14, wherein the actions of the driver of the vehicle include a danger avoidance action by the driver of the vehicle and an action of the driver of the vehicle attempting to avoid the danger. The vehicle further includes a driver state detection device for detecting a state of a driver of the vehicle.
The notification control device according to claim 1 , wherein the driver's state recognition unit recognizes a state of the driver of the host vehicle based on a state of the driver of the host vehicle detected by the driver's state detection device. The notification control device according to claim 4, further comprising a communication unit that transmits the danger situation stored in the situation storage unit as danger information to an external information distribution server. The notification control device according to claim 5, further comprising a communication unit that transmits the danger situation stored in the situation storage unit as danger information to an external information distribution server. An information distribution server that collects, from the notification control device described in claim 2 or 3 , the danger situation when a danger avoidance operation is performed by the autonomous driving of the vehicle or a danger avoidance operation is performed by the other vehicle as danger information, and distributes the collected danger information to the notification control device. The information distribution server according to claim 19 creates a danger map showing positions where the danger avoidance action by the autonomous driving of the vehicle or the danger avoidance action by the other vehicle was performed on a map based on the collected danger information, and distributes the created danger map to the notification control device. The information distribution server according to claim 19, which stores the vehicle equipped with the notification control device that transmitted the danger information as a vehicle requiring caution. Recognize the driver's status of the vehicle,
Recognizing the surrounding situation of the vehicle;
Acquire vehicle driving information, which is information regarding driving of the vehicle;
determining whether or not a danger avoidance operation has been performed by automatic driving of the host vehicle when a dangerous situation has occurred while the host vehicle is traveling and the driver of the host vehicle is not aware of the dangerous situation, based on the recognized state of the driver of the host vehicle, the recognized surrounding situation, and the acquired host vehicle driving information;
A notification control method, which, when it is determined that a danger avoidance operation is performed by automatic driving of the vehicle when the driver of the vehicle is not aware of the dangerous situation, notifies the driver of the vehicle of the dangerous situation when a danger avoidance operation is performed by automatic driving of the vehicle.

Images