JP7079069B2 - Information presentation control device, self-driving car, and self-driving car driving support system - Google Patents

Description

The present invention relates to an autonomous driving vehicle or the like that performs automatic driving control according to the surrounding conditions detected by an in-vehicle sensor.

In automobile-related technology, driving environment information, which is information on the driving environment of the own vehicle, is acquired using various in-vehicle sensors, and among this driving environment information, information useful for driving according to the driving condition of the own vehicle. (See Patent Document 1).

Japanese Unexamined Patent Publication No. 6-195595

However, in the above-mentioned display device, the driver of a vehicle having an automatic driving function (hereinafter referred to as an automatic driving vehicle) whose automatic control content changes according to the operating status of an in-vehicle sensor, etc., needs to do it by himself / herself. There is a problem that it is not possible to know the details of driving.

Therefore, the present invention provides an information presentation control device or the like that informs the driver of the autonomous driving vehicle of the content of driving to be performed by himself / herself.

The information presentation control device according to one aspect of the present invention is an information presentation control device that causes a presentation device mounted on an autonomous driving vehicle to present information, and is a plurality of information presentation control devices that recognize different objects regarding the surrounding conditions of the autonomous driving vehicle. Autonomous driving of an autonomous vehicle based on at least one of the control that the autonomous vehicle can and cannot control according to the selection information and the selection information acquisition unit that acquires the selection information indicating the selection status of the recognition unit. It is provided with a presentation control unit that causes a presentation device to present driving information regarding at least one of driving by the system and driving by the driver.

It should be noted that these comprehensive or specific embodiments may be realized in a recording medium such as a system, method, integrated circuit, computer program or computer-readable CD-ROM, and the system, method, integrated circuit, computer program. And may be realized by any combination of recording media.

The information presentation control device of the present invention can inform the driver of the autonomous driving vehicle of the content of driving to be performed by himself / herself.

FIG. 1 is a schematic diagram showing a configuration of a network including an autonomous driving vehicle including an information presentation control device according to an embodiment. FIG. 2 is a functional block diagram showing the functional configuration of the above-mentioned autonomous vehicle. FIG. 3 is an example of data referred to by the selection unit included in the information presentation control device according to the embodiment. FIG. 4 is a flow chart showing an example of an operation procedure including selection of a recognition unit and provision of selection information in the automatic driving system included in the above-mentioned automatic driving vehicle. FIG. 5 is an example of data referred to by the information presentation control device according to the embodiment in order to acquire data of information presented to the driver. FIG. 6 is a flow chart showing an example of an operation procedure from acquisition of selection information to presentation of operation information of the information presentation control device according to the embodiment. FIG. 7 is an example of driving information presented to the driver by the presenting device in the embodiment. FIG. 8 is another example of driving information presented by the presenting device to the driver in the embodiment. FIG. 9 is another example of driving information presented by the presenting device to the driver in the embodiment. FIG. 10 is a functional block diagram showing a functional configuration of an autonomous driving vehicle including an information presentation control device in a modified example of the embodiment. FIG. 11A is a flow chart showing an example of the operation of the selection unit in the modified example of the embodiment. FIG. 11B is a flow chart showing an example of the operation of the selection unit in the modified example of the embodiment. FIG. 11C is a flow chart showing an example of the operation of the selection unit in the modified example of the embodiment. FIG. 12 is an example of driving information presented to the driver by the presenting device in the modified example of the embodiment. FIG. 13 is an example of driving information presented to the driver by the presenting device in another embodiment. FIG. 14 is a functional block diagram showing a functional configuration of an autonomous vehicle provided with an autonomous vehicle driving support system according to another embodiment.

(Findings underlying the present invention)
The inventors of the present invention have found that the following problems arise with respect to the display device described in the "Background Techniques" column.

In an automobile, the information acquired by using the in-vehicle sensor as described above is also used for so-called electronic control by an in-vehicle ECU (Electronic Control Unit). However, the driver is not necessarily notified of the content of the electronic control actually performed in the car he is driving, or the change in the operation of the car due to this.

However, in an autonomous vehicle having a more sophisticated electronically controlled autonomous driving function, the content of the autonomous driving changes depending on the operating status of the in-vehicle sensor and the like. For example, in a moving self-driving car, when a sensor that detects the situation in front of the vehicle fails, what kind of target (for example, a person, a road sign, a road display, an obstacle, or other traffic) is the self-driving car. It is necessary to know whether it is possible to recognize the vehicle (vehicle, etc.) and control based on the recognition result, that is, the driver must know the driving behavior (recognition, estimation, judgment, operation) that must be performed by himself / herself.

Further, the self-driving car having a communication function with the outside transmits the surrounding condition detected by the in-vehicle sensor to an external recognition device, for example, a recognition system constructed as a large-scale neural network. Machine learning by deep learning is performed in this neural network, and it is possible to perform multi-class recognition with higher accuracy than in-vehicle lightweight recognition equipment. Then, the self-driving car receives information indicating an object recognized by this recognition system based on the surrounding conditions, and automatically drives using this information. However, depending on the location, the speed of communication between the autonomous vehicle and the outside does not reach the level required for autonomous driving. In this case, since the recognition result by the large-scale neural network cannot be used for the automatic driving, the content of the automatic driving changes. Therefore, in a situation where the communication speed of an autonomous vehicle is slow, the driver needs to know what to drive in that situation.

However, in the conventional display device, such a change of the recognition device and the change of the object of recognition due to the change are not taken into consideration. Therefore, in the self-driving car, there is a problem that the driver is not notified of the content of the driving that the driver himself needs to do, which changes with the change of the object.

In order to solve such a problem, the information presentation control device according to one aspect of the present invention is an information presentation control device that causes a presentation device mounted on the autonomous driving vehicle to present information, and is around the autonomous driving vehicle. At least the selection information acquisition unit that acquires selection information indicating the selection status of multiple recognition units that recognize different targets for the situation, and the control that can be executed and the control that cannot be executed by the autonomous vehicle according to the selection information. It is provided with a presentation control unit that causes the presentation device to present driving information regarding at least one of driving by the automatic driving system of the autonomous driving vehicle and driving by the driver based on one.

This makes it possible to provide the driver with information (driving information) about driving by the automatic driving system that changes according to the situation on the vehicle of the automatic driving vehicle or the driving behavior that he / she should do.

For example, the selection status of the plurality of recognition units may be the status of selection of the recognition unit that executes the recognition of the target or the recognition unit that does not execute the recognition of the target from the plurality of recognition units.

Thereby, the information presentation control device can provide the driver with driving information based on the information indicating the target recognized or not recognized in the automatic driving system.

For example, the selection information acquisition unit may acquire selection information indicating the selection result of the recognition unit from the plurality of recognition units from the automatic driving system.

As a result, the information presentation control device can provide the driver with driving information based on the actual selection status of the recognition unit in the automatic driving system.

Further, for example, the selection information acquisition unit acquires selection judgment material information used for judgment for selection of the recognition unit from a plurality of recognition units, and estimates the selection status based on the selection judgment material information to obtain selection information. May be obtained. Alternatively, for example, the recognition selected based on the information indicating the operating status of the detection unit or the communication unit of the autonomous driving vehicle for acquiring the information used for the recognition of the target is acquired. Selection information may be acquired by determining the unit.

As a result, the information presentation control device can acquire the selection information without waiting for the selection unit to select the recognition unit in the automatic driving system, so that the operation information can be promptly provided to the driver. Further, since the selection information can be acquired without providing an interface with the automatic driving system, the cost can be reduced.

For example, the selection determination material information may include travel location information indicating the characteristics of the location where the autonomous driving vehicle is traveling, which is acquired based on the map information.

As a result, the information presentation control device can provide driving information to the driver when a change such as a change in the content of driving by the automatic driving system occurs in the traveling place of the autonomous driving vehicle.

Further, for example, the selection determination material information may include driver state information indicating the physical state of the driver.

As a result, the information presentation control device can provide driving information to the driver when a change occurs in the physical condition of the driver such that the content of driving by the automatic driving system is changed.

Further, for example, the selection determination material information may include outside environment information indicating the outside environment of the autonomous driving vehicle.

As a result, the information presentation control device can provide driving information to the driver when a change occurs in the environment outside the vehicle such that the content of driving by the automatic driving system is changed.

Further, for example, the information indicating the operating status includes the information indicating the abnormality of the detector possessed by the detector, and the recognition unit that does not execute the recognition of the target uses the detection result output by the detector having the abnormality for the recognition of the target. A recognition unit may be included. Or, for example, the information indicating the operating status includes the communication quality of the communication unit or the information indicating whether or not the communication is executed, and the recognition unit that does not execute the recognition of the target is in a situation where the communication quality of the communication unit is inferior to the predetermined quality. Alternatively, it may include a recognition unit configured to use the information communicated by the communication unit for target recognition in a situation where the communication unit is not executing communication.

As a result, the information presentation control device can provide the driver with driving information that reflects the selection status of the recognition unit that is selected depending on whether or not the information necessary for recognizing the target can be acquired.

Further, for example, the presentation control unit 166 may control the presentation of driving information to the driver using at least one of voice and image.

Thereby, the information presentation control device can present the driving information to the driver who is driving at any time by using a sound, a moving image, a still image, an indicator light, or a combination thereof.

Further, for example, the operation information may be information including the contents of the operation that the automatic operation system does not execute or does not execute.

As a result, the information presentation control device can explicitly present the driver with the driving behavior to be performed by himself / herself.

Further, the autonomous driving vehicle driving support system according to one aspect of the present invention includes an automated driving system and any of the above-mentioned information presentation control devices.

As a result, the driver of the self-driving car can know at any time the driving behavior to be taken by himself / herself, which changes according to the situation on the vehicle.

Further, the autonomous driving vehicle according to one aspect of the present invention includes an automated driving system and any of the above-mentioned information presentation control devices.

In this self-driving car, the driver is notified at any time of his / her own driving behavior, which changes according to the situation of the vehicle or the driver.

It should be noted that these comprehensive or specific embodiments may be realized in a recording medium such as a system, method, integrated circuit, computer program or computer-readable CD-ROM, and the system, method, integrated circuit, computer program. Alternatively, it may be realized by any combination of recording media.

Hereinafter, embodiments will be specifically described with reference to the drawings.

It should be noted that all of the embodiments described below show comprehensive or specific examples. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, steps, the order of steps, etc. shown in the following embodiments are examples, and are not intended to limit the present invention. Further, among the components in the following embodiments, the components not described in the independent claims indicating the highest level concept are described as arbitrary components.

(Embodiment)
FIG. 1 is a schematic diagram showing a configuration of a network including an autonomous driving vehicle including an information presentation control device according to an embodiment.

The self-driving car 100 is one of the nodes of the network 10. The self-driving car 100 can use the recognition system 200 outside the own vehicle to recognize the surrounding situation in the self-driving car in the network 10.

The recognition system 200 is a system that executes machine learning such as deep learning using a neural network, a support vector machine, or Boosting, and can perform multi-class image recognition with high accuracy. Such a recognition system 200 is outside the self-driving car 100 and is a part of a recognition unit used by the self-driving car 100 for automatic driving. Details will be described later.

The self-driving car 100 and the recognition system 200 communicate with each other via a communication line 300 such as the Internet.

From the autonomous driving vehicle 100, information indicating the surrounding conditions of the own vehicle detected by an in-vehicle sensor, which will be described later, is sent to the base station 350 and delivered to the recognition system 200 via the communication line 300. For example, this information is delivered to the recognition system 200 as data of an image showing the surrounding situation of the own vehicle taken by an in-vehicle camera (image sensor).

The recognition system 200 recognizes the existence, direction, distance, movement, other characteristics, and the like of people and objects around the vehicle from the received information, and sends back the information indicating the recognition result to the autonomous driving vehicle 100. For example, the recognition system 200 that receives the data of the above image performs image recognition, recognizes an object such as a person, a road sign and a road display, an obstacle, and another passing vehicle appearing in this image, and indicates the object. The information is returned to the automatic driving vehicle 100 via the communication line 300.

In the automatic driving vehicle 100 that has received this information, the automatic driving system described later included in the automatic driving vehicle 100 controls various systems such as the accelerator, the brake, and the steering based on this information.

FIG. 2 is a functional block diagram showing a functional configuration of the autonomous driving vehicle 100. The power source of the autonomous vehicle 100 is not limited, and may be only a fuel-powered engine, only a power-powered motor, or both an engine and a motor (hybrid).

The autonomous driving vehicle 100 includes a driving support interface 110, a navigation system 140, an automatic driving system 150, an information presentation control device 160, a storage unit 180, and an antenna 190.

The driving support interface 110 is an interface between the autonomous driving vehicle 100 and the driver who is the user thereof, and is a component that accepts input from the driver and a component that presents information to the driver. For example, the driving support interface 110 includes a driving operation unit 120 such as an accelerator, a brake, and a steering wheel operated by the driver of the automatic driving vehicle 100, a display device including a display, instruments, indicator lights, speakers, a microphone, a touch panel, and buttons. include. Of these, a component capable of presenting information to the driver through visual, auditory, or tactile sensations is also referred to as a presenting device. For example, display devices, instruments, indicator lights, speakers, steering wheels, pedals can be presentation devices. The steering wheel and pedals can alert the driver by vibrating, for example, when certain conditions are met.

Further, in the description of the present embodiment, the drive system, the braking system, and the steering system controlled by the operation to these operation units are also included in the operation operation unit 120.

The storage unit 180 is a storage device that stores the map information described later and various other data and programs that the navigation system 140, the automatic operation system 150, and the information presentation control device 160 refer to in order to execute the processing. Yes, for example, it is realized by using a hard disk, a semiconductor memory, a recording medium such as a DVD (Digital Versaille Disc), or a combination thereof.

The antenna 190 is an antenna for information communication with the outside of the autonomous driving vehicle 100, which is performed by the navigation system 140, the automatic driving system 150, and the information presentation control device 160. In addition, the information communication performed by these components includes reception from artificial satellites of satellite positioning systems, VICS (registered trademark) (Vehicle Information and Communication System) and ITS (Intelligent Transport Systems: Intelligent Transport Systems). Multiple types of communication such as reception from (transportation system) and Internet communication may be included.

The driving support interface 110, the storage unit 180, and the antenna 190 in FIG. 2 may be shared by the navigation system 140, the automatic driving system 150, and the information presentation control device 160, and may be accessed from any of them. , Not limited to this. However, in FIG. 2, even if each of the navigation system 140, the automatic driving system 150, and the information presentation control device 160 has a dedicated driving support interface, storage unit, and antenna, the driving support interface 110 included in the automatic driving vehicle 100, It is shown integrally as a storage unit 180 and an antenna 190, respectively.

The navigation system 140 acquires the current position of the autonomous driving vehicle 100, searches for a route to the destination, and guides the route to the driver. The navigation system 140 includes a current position acquisition unit 141, a route search unit 142, a route guidance unit 143, and an autonomous navigation unit 144.

The current position acquisition unit 141 indicates a signal received from an artificial satellite of a satellite positioning system such as a GPS (Global Positioning System) satellite via an antenna 190, and an acceleration, angular speed, and vehicle speed received from an autonomous navigation unit 144 described later. Get the current position based on the signal.

The route search unit 142 calculates one or more routes from the current position to the destination.

The route guidance unit 143 searches for the route search unit 142, and guides the destination along the route selected by the driver with an image, voice, or the like output via the driving support interface 110.

The autonomous navigation unit 144 includes an acceleration sensor, a gyro sensor, and a vehicle speed measuring device, and transmits signals indicating the acceleration, angular velocity, and vehicle speed of the autonomous driving vehicle 100 measured and acquired by these to the above-mentioned current position acquisition unit 141. do.

The current position acquired by the current position acquisition unit 141, the route searched by the route search unit 142, and the guidance provided by the route guidance unit 143 are map information acquired from the outside of the own vehicle via the storage unit 180 or the antenna 190. Together with the traffic information, it is presented to the driver via the driving support interface 110.

In the current position acquisition unit 141, the route search unit 142, the route guidance unit 143, and the autonomous navigation unit 144 as described above, the ECU of the navigation system 140 reads and executes the program, and acquires data from the sensor as necessary. It is a component realized by referring to data such as map information stored in the storage unit 180.

The automatic driving system 150 includes a detection unit 151, a recognition unit 152, a situation determination unit 153, an operation control unit 154, an own vehicle monitoring unit 155, a selection unit 156, and a communication unit 157.

The detection unit 151 includes various sensors (detectors) that detect the surrounding conditions of the autonomous driving vehicle 100. Examples of various sensors include image sensors, radars, and riders. These sensors are output as data such as an image showing the detected ambient condition, a reflected wave, or reflected light.

The recognition unit 152 recognizes the target regarding the surrounding situation based on the data output from the detection unit 151.

In the example of FIG. 2, the automatic driving system 150 includes a plurality of recognition units 152 and recognizes objects different from each other. The "recognition of different objects" here means that, for example, one recognition unit 152 recognizes the road display around the autonomous driving vehicle 100, and the other recognition unit 152 recognizes other traffic objects. Point to that. As another example, one recognition unit 152 recognizes a certain position (direction and distance) of the recognized object, another recognition unit 152 recognizes the movement (speed and direction) of the recognized object, and another recognition unit 152. Refers to predicting the movement of a recognized object. In other words, each recognition unit 152 plays a different role in recognizing the surrounding situation.

Further, in the automatic driving system 150, in addition to the above-mentioned recognition unit 152 on the vehicle, the above-mentioned recognition system 200 (see FIG. 1) outside the vehicle can also be used as the recognition unit. The recognition system 200 recognizes an object about the surrounding situation based on the data from the detection unit 151 transmitted by the automatic operation system 150 via the communication unit 157. The recognition system 200 also recognizes an object different from each recognition unit 152. Further, for example, the recognition system 200 can perform higher recognition even if the object to be recognized is the same as the recognition unit 152. More advanced recognition means, for example, from the same image data, the recognition unit 152 recognizes a person and its position, and the recognition system 200 further specifies the person's age, behavior, belongings, and the like. Alternatively, from the same image data, the recognition unit 152 recognizes the road sign and its position, and the recognition system 200 further specifies the instruction content of the sign. Such a difference in recognition content is also an example of "recognition of different objects" in the present embodiment.

The recognition targets of the recognition unit 152 and the recognition system 200 may be partially overlapped in order to ensure redundancy from the viewpoint of safety.

The situation determination unit 153 makes a risk prediction based on the recognition result by the recognition unit 152 or the recognition system 200, and is appropriate in light of the current or predicted situation in which the driver operates the operation operation unit 120 as necessary. Judge whether or not. Further, based on the result of the danger prediction and the determination of suitability, it is determined whether or not the automatic driving system 150 operates the driving operation unit 120, and if the automatic driving system 150 operates, the content thereof is determined. As an example, the situation determination unit 153 determines the left and right positions of the autonomous driving vehicle 100 in the lane based on the surrounding information indicating the distance between the left and right white lines of the lane, and the autonomous driving vehicle 100 determines the vicinity of the vicinity of the lane. When the situation of being separated from one of the white lines within a certain distance continues for a certain period of time, the operation of the steering system for moving the self-driving car 100 to the center of the lane and its contents are determined.

The operation control unit 154 transmits a control signal for realizing the operation determined by the situation determination unit 153 to the operation operation unit 120. In the driving operation unit 120 that has received this control signal, the drive system, the braking system, and the steering system of the automatic driving vehicle 100 operate according to this control signal, and the steering and each pedal also operate as necessary.

The own vehicle monitoring unit 155 constantly monitors the state of the self-driving vehicle 100 (or in a very short time cycle in milliseconds). The target of this monitoring includes, for example, various detectors or communication units 157 included in the above-mentioned detection unit 151. More specifically, the operating status such as the presence or absence of a failure of the detector or the communication unit 157 is monitored.

In addition, the speed of communication with each detector or the presence or absence of connection, the communication quality of the communication unit 157 (for example, communication speed, presence or absence of connection for communication, communication stability), or the presence or absence of communication execution are also monitored. Can be. For example, it is monitored whether or not the communication speed exceeds the threshold value determined according to the amount of data transmitted and received from the detection of the surrounding situation to the operation control and the processing speed thereof.

In addition, the status of the components of the autonomous driving vehicle 100 that may affect the operation of each detector and the communication unit 157 may also be monitored. For example, the remaining amount of the battery of the autonomous driving vehicle 100 and the resource consumption of the autonomous driving system 150 may be included in the monitoring target.

The own vehicle monitoring unit 155 relates to the state of each device or the self-driving car 100 from the control information (CAN information) of each device in the vehicle transmitted by, for example, CAN (Control Area Network, not shown) included in the self-driving car 100. The above monitoring is executed by extracting information (status information).

The own vehicle monitoring unit 155 provides information indicating such an operating status or the like to the selection unit 156 at any time.

Based on the information provided by the own vehicle monitoring unit 155, the selection unit 156 selects from the recognition unit 152 and the recognition system 200 whether the above-mentioned target recognition is executed or the target recognition is not executed.

For example, while the communication speed with the recognition system 200 exceeds the above threshold value, the selection unit 156 selects the recognition system 200 as a recognizer that executes the recognition of the target. When the information indicating the operating status indicates a failure of any of the detectors, the selection unit 156 is a recognition unit (recognition unit 152 or recognition system 200) that uses the data output by the failed detector for recognition. Is selected as the recognition unit that does not execute the recognition of the target (or other than the recognition unit is selected as the recognition unit that executes the recognition of the target).

The selection unit 156 makes such a selection according to a predetermined rule. This predetermined rule is included in various data stored in, for example, the storage unit 180. When the own vehicle monitoring unit 155 notifies the selection unit 156 of, for example, a code indicating a predetermined operating state of the detector, the selection unit 156 obtains a rule corresponding to this code by referring to the data, and the recognition unit 156 obtains a rule corresponding to this code. Make your selection.

FIG. 3 is a diagram showing an example of this data. In the example of FIG. 3, in the case of communication failure (code: 01-001), the recognition unit outside the vehicle, that is, the recognition system 200 is selected as the recognition unit that does not execute the recognition of the target. Alternatively, with respect to this rule, the selection unit 156 may select only the recognition unit 152 on the vehicle as the recognition unit that executes the recognition of the target.

In addition, there are two entries (codes: 02-020, 02-021) in the rule regarding the failure of the detector 2, and the number of the recognition units 152 selected as the recognition units that do not execute the recognition of the target is an abnormal level. Depends on. For example, the detector 2 is assumed to be a stereo camera consisting of a set of two cameras that project the front of the vehicle. Further, it is assumed that the recognition unit-02, which is one of the recognition units 152, can acquire information on the distance to the object in front of the vehicle based on the image data received from the two cameras. Then, it is assumed that the recognition unit-03 can acquire information on the presence / absence and direction of an object in front of the own vehicle based on the image data received from at least one of the set of cameras. The failure of the abnormality level A of the detector 2 (code: 02-020) indicates the failure of only one of the cameras in one set. In this case, the recognition unit-02, which cannot obtain the data necessary for acquiring the distance, is selected as the recognition unit that does not perform the recognition of the target. Further, the failure of the abnormality level B of the detector 2 (code: 02-021) is a failure of both cameras in one set. In this case, in addition to the recognition unit-02, the recognition unit-03 is also selected as the recognition unit that does not execute the recognition of the target.

Further, when the remaining amount of the battery is less than a predetermined threshold value (code: 03-010), for example, as a recognition unit 152 that uses data output from a detector having a large power consumption, the recognition unit 152 does not execute the recognition of the target. The operating recognition unit is restricted by being selected or by selecting only the recognition unit having a high priority from the viewpoint of safety as the recognition unit that executes the recognition of the target.

Although not illustrated in FIG. 3, even when a sign of performance deterioration such as insufficient resources of the above-mentioned automatic driving system 150 is detected, the operating recognition unit may be restricted.

Further, the selection unit 156 outputs selection information indicating the actual selection status (selection result) based on the information output from the own vehicle monitoring unit 155. This selection information is provided to the information presentation control device 160.

As described above, the communication unit 157 transmits the data output by the detection unit 151 to the recognition system 200 outside the vehicle, receives the recognition result of the recognition system 200, and sends it to the situation determination unit 153. Further, the selection information output by the selection unit 156 is transmitted to the information presentation control device 160.

FIG. 4 is a flow chart showing an example of an operation procedure until the recognition unit is selected according to the state of the autonomous driving vehicle 100 and the selection information is provided in the automatic driving system 150 configured as described above. In this example, the recognition unit is selected according to the speed of communication via the communication unit 157.

In the automatic driving system 150, the own vehicle monitoring unit 155 acquires state information indicating the communication speed of the communication unit 157 (step S40).

Based on this state information, the selection unit 156 selects a recognition unit that executes the recognition of the target or a recognition unit that does not execute the recognition. When the communication speed exceeds a predetermined threshold value (YES in step S41), the selection unit 156 selects the recognition system 200, which is a recognition unit outside the vehicle, as the recognition unit for executing the recognition of the target (step S42). When the communication speed is less than a predetermined threshold value (NO in step S41), the selection unit 156 selects at least a part of the recognition unit on the vehicle as the recognition unit for executing the recognition of the target (step S43).

Next, the selection unit 156 outputs the selection information indicating the status of the selection actually made through the above procedure to be provided to the information presentation control device 160 (step S44). The output selection information may be transmitted to the information presentation control device 160, or may be written in the storage unit 180 and stored as a part of various data.

In step S43, instead of or in addition to the above selection, a selection of selecting the recognition unit outside the vehicle as a recognition unit that does not execute the recognition of the target may be executed.

In each component of the automatic operation system 150 as described above, the ECU of the automatic operation system 150 reads a program from the storage unit 180 and executes it, acquires data from a sensor as necessary, or data via a communication module. Is realized by transmitting and receiving, and referring to various data stored in the storage unit 180.

The information presentation control device 160 is an example of the information presentation control device according to the present embodiment, which includes the selection information acquisition unit 161 and the presentation control unit 166.

The selection information acquisition unit 161 acquires selection information indicating the selection status of a plurality of recognition units that recognize objects different from each other regarding the surrounding conditions of the autonomous driving vehicle 100. In the present embodiment, the selection information acquisition unit 161 is a selection result of the recognition unit from a plurality of recognition units, which is actually performed by the selection unit 156, transmitted from the automatic operation system 150, and more specifically, a target. It is acquired by receiving the selection information which is the selection status of at least one of the recognition unit that executes the recognition of the object and the recognition unit that does not execute the recognition of the target.

The presentation control unit 166 presents driving information, which is information on at least one of driving by the automatic driving system 150 of the autonomous driving vehicle 100 and driving by the driver, to the presenting device according to the selection information acquired by the selection information acquisition unit 161. Let me.

Here, the "driving information" and its presentation will be described in more detail.

The driving information is information for informing the driver of the autonomous driving vehicle of the content of the driving that the driver should do by himself / herself. The content of driving here refers to various driving behaviors of recognition, estimation, judgment, and operation. The driving information is, for example, information indicating recognition of a certain object, estimation and judgment based on this recognition, and operation based on this estimation or judgment when the automatic driving system 150 does not recognize the target, and is automatic driving. It is presented to the driver as a driving action that the system 150 does not perform or does not perform, or that the driver should do.

The presentation of this driving information is voiced through the presenting device, for example, at any time at the request of the driver of the autonomous driving vehicle 100, or when the control that can be executed or the control that cannot be executed by the autonomous driving system 150 changes. Or it is done using at least one of the images. The display mode of this image does not matter as long as it is a display object that the driver can visually recognize. The driving information may include information presented using either an indicator light around the driver's seat or a still image or moving image displayed on the display device.

Such operation information is acquired by the presentation control unit 166 according to the selection information acquired by the selection information acquisition unit 161. For example, as shown in FIG. 5, data indicating information presented according to the selection status of the recognition unit is stored in advance in the storage unit 180, and the presentation control unit 166 refers to this data for selection. Based on the selection information provided by the unit 156, information on the content of the operation to be executed or the content of the operation not to be executed in the automatic operation system 150, or the content of the operation currently being executed or the content of the operation not being executed. And present this information to the driver as driving information.

For example, it is assumed that the presentation control unit 166 newly receives selection information indicating a situation in which the recognition unit -01, which is one of the recognition units 152, is selected as the recognition unit performing the recognition of the target. In this case, the presentation control unit 166 searches for "recognition unit-01" in the column of "recognition unit" and finds it in the first row of the data row. Next, referring to the second column from the left of the bank, the information that the automatic steering for lane keeping is possible is acquired. Then, the presentation control unit 166 displays a message in which this information is substituted into the variable portion of the predetermined template message on the display unit, or outputs the voice from the speaker to present the driving information to the driver.

Another example of the mode of presenting the driving information by the presentation control unit 166 will be given. In the table of FIG. 5, in the column of "presentation information 1 at the time of recognition execution", the image data displayed on the display device when each recognition unit is selected as the recognition unit performing the recognition of the target. The name is shown.

Further, in the column of "presentation information 2 at the time of recognition execution", the name of the character string data displayed on the display device when each recognition unit is selected is shown.

Further, in the column of "Presentation information 1 when recognition is not executed", the name of the voice data output from the speaker when each recognition unit is selected as the recognition unit that does not execute the target recognition is shown. There is.

Each of these image data, character string data, and voice data is data indicating driving information, and is stored in the storage unit 180. The presentation control unit 166 acquires each data based on the selection information acquired by the selection information acquisition unit 161 with reference to the table of FIG. 5, and displays the contents of this data on the display device as an image or a character string. , It may be presented to the driver as driving information by outputting it from a speaker as a voice message. As another example of information presentation, a table in which the indicator lamp to be turned on or its display mode (lighting, blinking, color of the illumination lamp, etc.) is shown in FIG. 5 for each recognition unit selection status. It is specified. The presentation control unit 166 presents operation information by operating the indicator lamp according to the designation.

FIG. 6 is a flow chart showing an example of an operation procedure from acquisition of selection information to presentation of operation information in the information presentation control device 160 configured as described above.

When the selection information acquisition unit 161 acquires the selection information output by the automatic operation system 150 in the information presentation control device 160 (YES in step S61), the presentation control unit 166 acquires the operation information based on this selection information (YES). Step S62).

Next, the presentation control unit 166 causes the driver to present the acquired operation information by an appropriate presentation device (step S63).

In the selection information acquisition unit 161 and the presentation control unit 166 as described above, the ECU of the information presentation control device 160 reads and executes the program, acquires necessary data from other components, or stores the necessary data in the storage unit 180. It is a component realized by referring to the data.

Up to this point, the operations of the automated driving system 150 and the information presentation control device 160 have been described. In the following, a specific example of a series of flows from selection of a recognition unit to presentation of driving information in the autonomous driving vehicle 100 including the automatic driving system 150 and the information presentation control device 160 will be given.

For example, in the automatic driving system 150, the communication speed of the communication unit 157 is less than a predetermined threshold value based on the state information (step S40 in FIG. 4) acquired in the monitoring performed by the own vehicle monitoring unit 155. (NO in step S41 of FIG. 4), the selection unit 156 selects the recognition unit 152 on the vehicle as a recognizer for recognizing the target in response to the result of this finding (step S43 of FIG. 4). ..

At this time, it is assumed that the recognition unit -01 is selected as the on-vehicle recognizer for recognizing the target. The recognition unit-01 recognizes the left and right positions in the lane based on the image data acquired from the detection unit 151, and provides the situation determination unit 153 with information indicating the recognition result. The situation determination unit 153 determines the suitability of the left and right positions in the lane based on this information, and if inappropriate, causes the operation control unit 154 to perform a steering operation to return the own vehicle to the appropriate left and right positions.

Selection information indicating the selection status of such a recognition unit is output from the selection unit 156 and provided to the information presentation control device 160 (step S44 in FIG. 4).

When the selection information acquisition unit 161 acquires the selection information (step S61 in FIG. 6), the presentation control unit 166 acquires operation information data with reference to the table in FIG. 5 based on the selection information (step S62). ). In the above assumption, the data acquired with reference to the table of FIG. 5 is the image data imgLaneKeep and the character string data txtLaneKeep. FIG. 7 shows an example in which the presentation control unit 166 causes the display device to present these data among the presentation devices (step S63).

In this example, the image displayed on the left half of the screen corresponds to the image data imgLaneKeep shown in the table of FIG. 5, and the message displayed on the right half of the screen corresponds to the character string data txtLaneKeep.

The display device uses the image data imgLaneKeep to indicate the lane keep being performed by the automated driving system 150 when recognition unit -01 is selected. The lane keep is an operation including recognition, judgment or guessing, and operation of the steering system.

Further, using the character string data txtLaneKeep, the lane keep executed by the automatic driving system 150 when the recognition unit-01 is selected is shown in the upper right of the screen. Further, in the lower right of the screen, information on driving by the driver is shown. The information about the driving by the driver is also the content of the driving that the automatic driving system 150 does not execute or does not execute. This display allows the driver to know what driving behavior he or she needs to do.

It should be noted that this text may be read aloud automatically or at the request of the driver so that the driver does not have to look at the screen of the display device.

As another example, when it is found that the communication speed of the communication unit 157 is less than a predetermined threshold value (NO in step S41 in FIG. 4), the selection unit 156 is the recognition unit 152 on the vehicle. Among them, it is assumed that the recognition unit-03 is selected as the on-vehicle recognizer for recognizing the target (step S43 in FIG. 4).

The recognition unit-03 recognizes the position of the autonomous driving vehicle 100 on the road surface based on the data acquired from the detection unit 151, and provides the situation determination unit 153 with information indicating the recognition result. Based on this information, the situation judgment unit 153 determines the suitability of the position on the road surface, and if it is inappropriate, the necessary steering operation, accelerator operation, and brake operation for returning the vehicle to the appropriate position are performed. Let the operation control unit 154.

Selection information indicating the selection status of such a recognition unit is output from the selection unit 156 and provided to the information presentation control device 160 (step S44 in FIG. 4).

When the selection information acquisition unit 161 acquires the selection information (step S61 in FIG. 6), the presentation control unit 166 acquires operation information data with reference to the table in FIG. 5 based on the selection information (step S62). ). In the above assumption, the data acquired with reference to the table of FIG. 5 is the image data imgSfcKeep and the character string data txtSfcKeep. FIG. 8 shows an example in which the presentation control unit 166 causes the display device to present these data among the presentation devices (step S63).

In this example, the image displayed on the left half of the screen corresponds to the image data imgSfcKeep shown in the table of FIG. 5, and the message displayed on the right half of the screen corresponds to the character string data txtSfcKeep.

In the display device, the image data imgSfcKeep is used to indicate the road surface keep operation performed by the automatic driving system 150 when the recognition unit-03 is selected. The road surface keeping is an operation including recognition, judgment or estimation, and operation of the drive system, the braking system, and the steering system.

Further, using the character string data txtSfcKeep, the road surface keeping operation performed by the automatic driving system 150 when the recognition unit-03 is selected is shown in the upper right of the screen. Further, in the lower right of the screen, information on driving by the driver is shown. This display allows the driver to know what driving behavior he or she needs to do. The information about the driving by the driver is also the content of the driving that the automatic driving system 150 does not execute or does not execute.

As another example, when it is found that the communication speed of the communication unit 157 is equal to or higher than a predetermined threshold value (YES in step S41 in FIG. 4), the selection unit 156 is a recognition unit (recognition system) outside the vehicle. It is assumed that 200) is selected as a recognizer for recognizing an object (step S42 in FIG. 4).

The recognition system 200 predicts the occurrence of a person or other passing object in front of the own vehicle based on the data acquired from the detection unit 151, and provides the situation determination unit 153 with information indicating the prediction result. Based on this information, the situation determination unit 153 causes the operation control unit 154 to perform braking for deceleration and steering operation for avoiding a collision.

Selection information indicating the selection status of such a recognition unit is output from the selection unit 156 and provided to the information presentation control device 160 (step S44 in FIG. 4).

When the selection information acquisition unit 161 acquires the selection information (step S61 in FIG. 6), the presentation control unit 166 acquires operation information data with reference to the table in FIG. 5 based on the selection information (step S62). ). In the above assumption, the data acquired with reference to the table of FIG. 5 is the image data imgRushPred and the character string data txtRushPred. FIG. 9 shows an example in which the presentation control unit 166 causes the display device to present these data among the presentation devices (step S63).

In this example, the image displayed on the left half of the screen corresponds to the image data imgRushPred shown in the table of FIG. 5, and the message displayed on the right half of the screen corresponds to the character string data pxtRushPred.

The display device uses the image data imgRushPred to indicate the pop-out prediction performed by the automated driving system 150 when the recognition system 200 is selected. The pop-out prediction is an operation including recognition, judgment or estimation, and operation of the braking system and the steering system.

Further, using the character string data pxtRushPred, the pop-out prediction executed by the automatic driving system 150 when the recognition system 200 is selected is shown in the upper right of the screen. Further, in the lower right of the screen, information on driving by the driver is shown. This display allows the driver to know what driving behavior he or she needs to do. The information about driving by the driver is also the content of the driving that the automatic driving system 150 does not execute.

As described above, the information presentation control device 160 in the present embodiment includes the selection information acquisition unit 161 and the presentation control unit 166.

The selection information acquisition unit 161 acquires selection information indicating the selection status of a plurality of recognition units (recognition unit 152 and recognition system 200) that recognize different targets for the surrounding conditions of the autonomous driving vehicle 100.

The presentation control unit 166 is driven by the automatic driving system 150 of the automatic driving vehicle 100 and driven by the driver based on at least one of the control that can be executed and the control that cannot be executed by the automatic driving vehicle 100 according to the selection information. The driving information regarding at least one of the above is presented to the presenting device mounted on the autonomous driving vehicle 100.

This makes it possible to provide the driver with information (driving information) about driving by the automatic driving system that changes according to the situation on the vehicle of the automatic driving vehicle or the driving behavior that he / she should do.

For example, the selection status of the plurality of recognition units is the status of selection of the recognition unit that executes the recognition of the target or the recognition unit that does not execute the recognition of the target from the plurality of recognition units.

As a result, the information presentation control device 160 can provide the driver with driving information based on the information indicating the target recognized or not recognized by the automatic driving system 150.

For example, the selection information acquisition unit 161 acquires selection information indicating selection results of the recognition units from the plurality of recognition units from the automatic operation system 150.

As a result, the information presentation control device 160 can provide the driver with driving information based on the actual selection status of the recognition unit in the automatic driving system 150.

The mode of providing and acquiring selection information is not limited to the above.

For example, the selection information acquisition unit 161 acquires the selection judgment material information used for the judgment for selection in the automatic operation system 150, and acquires the selection information by estimating the selection status based on the selection judgment material information. You may.

The selection determination material information referred to here is information used for determination for selection of the recognition unit, and is, for example, information provided by the own vehicle monitoring unit 155. The selection information acquisition unit 161 processes this selection determination material information according to an algorithm common to that used by the selection unit 156. As a result, the selection information acquisition unit 161 acquires the selection information indicating the selection status by estimating the recognition unit selected by the selection unit 156.

Further, for example, the selection information acquisition unit 161 acquires information indicating the operation status of the detection unit 151 or the communication unit 157 of the autonomous driving vehicle 100 for acquiring the information used for recognizing the target, and obtains the operation status. The selection information may be acquired by determining the recognition unit to be selected based on the indicated information.

The information indicating the operating status here is, for example, the control information on the CAN acquired by the own vehicle monitoring unit 155. Based on this information, the selection information acquisition unit 161 determines a recognition unit that cannot execute the recognition of the target.

By doing so, the information presentation control device 160 can acquire the selection information without waiting for the selection of the recognition unit by the selection unit 156 in the automatic operation system 150, so that the operation information is promptly provided to the driver. be able to.

Further, the above-mentioned information indicating the operating status includes information indicating an abnormality of the detector possessed by the detection unit 151, and the recognition unit that does not execute the recognition of the target recognizes the detection result output by the detector having the abnormality. The recognition unit used for the above may be included.

Further, the above-mentioned information indicating the operating status includes information indicating the communication quality of the communication unit 157 or the presence / absence of communication execution, and the recognition configured to use the information communicated by the communication unit 157 for the recognition of the target. May include parts.

As a result, the information presentation control device 160 operates to reflect the selection status of the recognition unit, which is selected according to whether or not the information necessary for the recognition of the target can be acquired or whether or not the data of the recognition result can be transmitted. Information can be provided to the driver.

The presenting device also presents driving information to the driver using at least one of a sound or an image.

Thereby, the information presentation control device 160 can present the driving information to the driver who is driving at any time by using the visual information by sound, moving image, still image, indicator light, or a combination thereof.

The driving information is information including the content of the driving that the automatic driving system 150 does not execute.

Such information indicates driving behavior that the autonomous driving system 150 is not performing and that the driver must do on his own. In other words, by presenting this information, the driver can grasp the driving behavior to be taken by himself / herself.

(Modified example of the embodiment)
The information presentation control device in the modified example of the embodiment is mounted on an autonomous driving vehicle provided with an automatic driving system, like the information presentation control device 160. Hereinafter, differences from the embodiments will be mainly described, common components will be shown using common reference numerals, and description thereof will be omitted.

FIG. 10 is a functional block diagram showing a functional configuration of the autonomous driving vehicle 1100.

In the automatic driving system 1150 included in the automatic driving vehicle 1100 in this modification, further information can be used in addition to the embodiment for the selection of the recognition unit.

The automatic driving system 1150 includes a traveling place information acquisition unit 1151, a driver status information acquisition unit 1152, and an outside environment information acquisition unit 1153, in addition to the components of the embodiment.

The traveling location information acquisition unit 1151 acquires traveling location information indicating the characteristics of the location where the autonomous driving vehicle 1100 is traveling, based on the map information stored in the storage unit 180.

The "characteristics of the place where the vehicle is traveling" here means, for example, the classification of the road such as whether it is a general road or an expressway, the shape of the road such as whether a straight line continues or meanders, an urban area, an urban area, or It is a special situation such as the type of area such as a mountainous area, whether it is inside or outside a tunnel. Information indicating the location where the autonomous vehicle 1100 is traveling, that is, the current position, is provided by, for example, the current position acquisition unit 141 of the navigation system 140. The travel location information acquired by the travel location information acquisition unit 1151 is output to the selection information acquisition unit 161.

The driver status information acquisition unit 1152 monitors the driver in the passenger compartment of the autonomous driving vehicle 1100 and acquires the driver status information indicating the physical condition of the driver.

The "physical condition of the driver" as used herein refers to a condition that may affect the performance of safe driving, such as the degree of fatigue, the degree of concentration, the degree of arousal, and the change in physical condition of the driver. Such driver status information is data output by various sensors (not shown) that photograph the driver or measure the driver's biological data, which is included in the driver status information acquisition unit 1152, or the autonomous driving vehicle 1100. The driver state information acquisition unit 1152 may analyze and acquire the data of the CAN information indicating the operation status (for example, the continuous operation time, the operation status of the driver to the operation operation unit 120). The driver status information acquired by the driver status information acquisition unit 1152 is output to the selection information acquisition unit 161.

The vehicle outside environment information acquisition unit 1153 acquires the vehicle outside environment information indicating the vehicle outside environment of the autonomous driving vehicle 1100.

The "outside environment" here means, for example, the outside conditions of the own vehicle such as the weather and brightness in the place where the autonomous driving vehicle 1100 is traveling, or those indirectly indicating these (eg, temperature, time, etc.). Visibility, road surface conditions). Such outside environment information may be detected by a sensor (not shown) included in the outside environment information acquisition unit 1153, or outside the vehicle that has acquired data output by each detector included in the detection unit 151 of the automatic driving system 150. The environmental information acquisition unit 1153 may analyze and acquire this data. Alternatively, information that can be acquired from the outside of the automatic driving system via the communication unit 157 (eg, weather information or traffic information of each place distributed on the Internet) may be acquired as outside environment information. The vehicle outside environment information acquired by the vehicle outside environment information acquisition unit 1153 is output to the selection unit 156.

In the driving location information acquisition unit 1151, the driver status information acquisition unit 1152, and the vehicle exterior environment information acquisition unit 1153 as described above, the ECU of the information presentation control device 160 reads and executes the program, and if necessary, from the sensor. It is a component realized by acquiring data and referring to the data stored in the storage unit 180.

On the other hand, the selection unit 156 may select the recognition unit according to the characteristics indicated by the travel location information received from the travel location information acquisition unit 1151 in addition to the abnormality detected by the own vehicle monitoring unit 155. Alternatively, the selection unit 156 may further select the recognition unit according to the physical condition of the driver indicated by the driver state information received from the driver state information acquisition unit 1152. Alternatively, the selection unit 156 may select the recognition unit according to the vehicle exterior environment indicated by the vehicle exterior environment information received from the vehicle exterior environment information acquisition unit 1153. Hereinafter, the selection of the recognition unit made according to this information will be described later using a specific example.

FIG. 11A is a flow chart showing an example of the operation of the selection unit 156 in the automatic operation system 1150 configured as described above. In this example, the selection unit 156 uses the travel location information output by the travel location information acquisition unit 1151 in addition to the information regarding the communication state monitored by the own vehicle monitoring unit 155.

The selection unit 156 evaluates the communication state based on the information provided by the own vehicle monitoring unit 155 (step S80A). Specifically, for example, if the communication speed is equal to or higher than a predetermined threshold value, it is evaluated that the communication state is good.

When the communication state is good (YES in step S80A), the selection unit 156 selects the recognition system 200, which is a recognition unit outside the vehicle, as the recognition unit that executes the recognition of the target (step S84A).

When the communication state is not good (NO in step S80A), the selection unit 156 then acquires the travel location information from the travel location information acquisition unit 1151.

When the traveling place is an expressway (“expressway” in step S81A), the selection unit 156 selects the recognition unit -01, which is one of the recognition units 152 on the vehicle, as the recognition unit for executing the target recognition (“highway”). Step S82A). The selected recognition unit-01 is a recognition unit that recognizes a display such as a white line indicating a traffic zone of a road.

When the driving place is a general road in an urban area (“mountainous general road” in step S81A), the selection unit 156 targets the recognition unit -05, which is the recognition unit 152 on the vehicle different from the recognition unit -01. It is selected as a recognition unit for executing recognition (step S83A). The selected recognition unit-05 is a recognition unit that recognizes obstacles such as animals and stones that obstruct passage, for example.

As described above, when the recognition unit is selected by the selection unit 156 according to a predetermined algorithm, the selection information is provided to the information presentation control device 160 (step S85A). In the information presentation control device 160, the presentation control unit 166 acquires the operation information corresponding to the selection information and presents it to the driver. For example, when the recognition unit-01 is selected, the operation information as shown in FIG. 7 is presented to the display device.

FIG. 11B is a flow chart showing another example of the operation of the information presentation control device 160. In this example, the driver status information output by the driver status information acquisition unit 1152 is further used.

The selection unit 156 evaluates the communication state based on the information provided by the own vehicle monitoring unit 155 (step S80B).

When the communication state is good (YES in step S80B), the selection unit 156 selects the recognition system 200, which is a recognition unit outside the vehicle, as the recognition unit that executes the recognition of the target (step S85B).

When the communication state is not good (NO in step S80B), the selection unit 156 then acquires the travel location information from the travel location information acquisition unit 1151. The case where the driving place is a highway is omitted in this example.

When the traveling place is a general road in an urban area (“urban general road” in step S81B), the selection unit 156 then acquires the driver status information from the driver status information acquisition unit 1152.

When the driver status information indicates that the concentration of the driver is low (“low” in step S82B), the selection unit 156 selects the recognition unit-06, which is one of the recognition units 152 on the vehicle (step). S83B). The selected recognition unit-06 is a recognition unit that recognizes a person such as a passerby.

When the driver status information indicates that the concentration of the driver is sufficiently high (“high enough” in step S82B), the selection unit 156 recognizes that the recognition unit 152 is a recognition unit 152 on the vehicle different from the recognition unit-06. Part-02 is selected (step S84B). The recognition unit-02 selected at this time is, for example, a recognition unit that recognizes the distance (inter-vehicle distance) to a vehicle traveling in front of the same lane.

When the recognition unit is selected by the selection unit 156 according to a predetermined algorithm, the selection information is provided to the information presentation control device 160 (step S86B). In the information presentation control device 160, the presentation control unit 166 acquires the operation information corresponding to the selection information and presents it to the driver. For example, when the recognition unit-06 is selected, the operation information as illustrated in FIG. 12 is presented to the display device. In this example, the image displayed on the left half of the screen is shown as image data imgPsnAvd in the table of FIG. 5 referenced by the presentation control unit 166, and the text message displayed on the right half of the screen is shown as character string data txtPsnAvd. Has been done. The image data imgPsnAvd indicates human avoidance performed by the automated driving system 1150 when recognition unit-06 is selected. Further, the character string data txtPsnAvd describes the content of the operation (executable operation) currently being executed by the automatic operation system 1150 when the recognition unit-06 is selected, in the frame at the upper part of the screen, at this time, the driver. Shows the driving actions that must be performed by oneself (contents of driving that the automatic driving system 1150 does not perform) in the frame at the bottom of the screen.

FIG. 11C is a flow chart showing another example of the operation of the information presentation control device 160. In this example, the outside environment information output by the outside environment information acquisition unit 1153 is further used.

The selection unit 156 evaluates the communication state based on the information provided by the own vehicle monitoring unit 155 (step S80C).

When the communication state is good (YES in step S80C), the selection unit 156 selects the recognition system 200, which is a recognition unit outside the vehicle, as the recognition unit that executes the recognition of the target (step S86C).

When the communication state is not good (NO in step S80C), the selection unit 156 then acquires the travel location information from the travel location information acquisition unit 1151. The case where the driving place is a highway is omitted in this example.

When the traveling place is a general road in an urban area (“urban general road” in step S81C), the selection unit 156 then acquires the driver status information from the driver status information acquisition unit 1152. The case where the driver status information indicates that the concentration of the driver is low is omitted in this example.

When the driver status information indicates that the concentration of the driver is sufficiently high (“high enough” in step S82C), the selection unit 156 then acquires the vehicle outside environment information from the vehicle outside environment information acquisition unit 1153.

When the outside environment information indicates that the outside of the vehicle is sufficiently bright (“sufficiently bright” in step S83C), the selection unit 156 selects the recognition unit-02, which is one of the recognition units 152 on the vehicle (step S84C). The recognition unit-02 selected at this time is, for example, a recognition unit that recognizes the distance (inter-vehicle distance) to a vehicle traveling in front of the same lane.

When the outside environment information indicates that the outside of the vehicle is dark due to rain (“dark due to rain” in step S83C), the selection unit 156 is one of the recognition units 152 on the vehicle different from the recognition unit-02. -04 is selected (step S85C). The recognition unit-04 selected here is, for example, a recognition unit that recognizes the display content of a road sign.

When the recognition unit is selected by the selection unit 156 according to a predetermined algorithm, the selection information is provided to the information presentation control device 160 (step S87C). In the information presentation control device 160, the presentation control unit 166 acquires the operation information corresponding to the selection information and presents it to the driver. For example, when the recognition unit-04 is selected, a voice message from the speaker indicates that the automatic driving system 1150 is currently performing (executable driving) that "operates the steering wheel and brake according to the sign". The contents and the driving actions that the driver needs to do by himself, "Please operate the steering wheel and brakes paying close attention to pedestrians and bicycles" (contents of driving that the automatic driving system 1150 does not execute) The driving information to be shown is presented to the driver.

The selection of the recognition unit as shown in FIGS. 11A to 11C and the presentation of the driving information corresponding thereto described above are repeated, for example, while the automatic driving system 150 is executing the automatic driving in the automatic driving vehicle. The driver can be notified at any time about the driving behavior that he / she needs to do.

The mode of providing and acquiring selection information is not limited to the above.

For example, the selection determination material information acquired by the selection information acquisition unit 161 may include travel location information indicating the characteristics of the location in which the autonomous driving vehicle is traveling, which is acquired based on the map information. Further, the selection determination material information may include driver status information. In addition, the selection determination material information may include information on the environment outside the vehicle.

The selection information acquisition unit 161 processes this selection determination material information according to an algorithm common to that used by the selection unit 156. That is, by executing the process shown in FIGS. 11A to 11C by the selection information acquisition unit 161 using the selection determination material information used by the selection unit 156, the recognition unit selected by the selection unit 156 is estimated and the selection information is selected. May be obtained.

As a result, the information presentation control device 160 provides the driver with driving information indicating the change when, for example, a change occurs in the driving place of the automatic driving vehicle 1100 such that the driving content of the automatic driving system 1150 is changed. can do. Alternatively, when a change occurs in the physical condition of the driver such that the content of the driving by the automatic driving system 1150 is changed, the driving information indicating the change can be provided to the driver. Alternatively, when a change occurs in the environment outside the vehicle such that the content of driving by the automatic driving system 1150 is changed, driving information indicating this change can be provided to the driver.

Since the resources of the automatic driving system 1150 are limited, it is not always possible for the automatic driving system 1150 to always operate all detectors and recognition units to assist or substitute for driving control. Therefore, the content of the automatic operation by the automatic operation system 1150 is limited to what is possible based on the information acquired by the recognition unit selected by the selection unit 156. And as mentioned above, the selection of the recognition unit depends not only on the condition of the equipment on the own vehicle but also on the content of automatic driving that should be prioritized and changes depending on the driving place, the physical condition of the driver, or the environment outside the vehicle. It will be changed accordingly. The selection of the recognition unit by the selection unit 156 described above with reference to FIGS. 11A to 11C is an example of a selection algorithm for this modification, and the present invention is not limited to these examples. For example, the combination of the driving place information, the driver status information, and the outside environment information, and the order in which these information are used do not have to be as described above.

(Other embodiments)
As described above, embodiments and modifications have been described as examples of the techniques disclosed in the present application. However, the technique in the present disclosure is not limited to these, and can be applied to embodiments in which changes, replacements, additions, omissions, etc. are made as appropriate. Further, it is also possible to combine the constituent elements described in the above-described embodiments and modifications to form a new embodiment. Therefore, other embodiments will be exemplified below.

The objects recognized by the recognition unit 152 are not limited to the lanes, inter-vehicle distances, people, and obstacles mentioned as examples in the above description. For example, the object of recognition may be a more subdivided category such as a young person or an elderly person among human beings. Alternatively, the presence or absence of the passerby, the position or distance to the passerby, and the motion prediction are not limited, and a specific operating state may be recognized. For example, an operating state such as a wobbling of a passing object or a pedestrian's gaze on the screen of a portable device may be recognized. In addition, the degree of danger of the situation around the own vehicle is determined according to the recognition of the person and the operating state that are subdivided in this way, and the division of driving behavior with the driver is determined according to this degree of danger. The decision content may be presented to the presenting device as operation information.

FIG. 13 is an example of driving information displayed on the display device when a pedestrian with a high degree of danger is recognized. In this example, the selection unit 156 selects the recognition unit-07. In response to the selection information indicating the selection status, the presentation control unit 166 of the information presentation control device 160 obtains the image data imgSpdLmt and the character string data pxtSpdLmt with reference to the table shown in FIG. 5, and obtains these data. The image of FIG. 13 is displayed on the display device using the display device. In this image, both the information about driving by the automatic driving system and the information about driving by the driver are presented as driving information.

Further, in the above example, only the combination of the image, the character, and the voice is shown in the presentation of the driving information, but only one of these may be used for the presentation. Further, not only the voice for reading out the character string but also the sound for calling attention to the change in the driving information may be used. Further, it may be combined with a tactile notification such as vibration of the steering wheel or push-back of the pedal.

Further, in the above example, the driving information is mainly presented to the driver when the selection is changed between the recognition units on the vehicle, but the timing of presenting the driving information is not limited to this. It may also be presented when the selection is changed from the recognition system 200, which is a recognition unit outside the vehicle, to the recognition unit on the vehicle. In this case, information that the recognition result by the recognition system 200 cannot be used due to poor communication status may be presented. Since the automatic driving control based on the recognition by the lightweight recognition unit on the car has more restrictions on the function, in this case, there is less assistance or agency for driving by the automatic driving system, that is, there are many driving actions to be taken by oneself. Can call the driver's attention.

Further, in the above description, the driving information is described to be presented to the driver when the selection of the recognition unit is changed, but the timing of presenting the driving information is not limited to this. For example, the driver may be presented at any time as a response to a request for presenting driving information by a predetermined operation or voice through the driving support interface 110.

Further, in the above description, there is a description that the information presentation control device selects the recognition unit in response to the detection of the abnormality, but this selection may be performed, for example, according to the change in the presence or absence of the abnormality. That is, for example, when there is some abnormality, the selection unit 156 may execute the selection of the recognition unit even when the selection unit 156 is notified of the information indicating that the abnormality is not present. As a result, for example, if it can be determined from the information from the own vehicle monitoring unit 155 that the communication state is restored or that the obstacles adhering to the surface of the in-vehicle sensor have disappeared, the selection unit 156 selects the recognition unit. Run. The trigger for changing the selection status of the recognition unit is not limited to abnormalities such as breakdowns, but as described above, it is triggered by changes in the current position of the autonomous vehicle, the environment outside the vehicle, or the physical condition of the driver. You may.

Further, according to the present invention, the information presentation control device 160 may be realized and provided as an autonomous driving vehicle driving support system by integrating the information presentation control device 160 with the navigation system 140, the automatic driving system 150 (1150), or both of them. FIG. 14 is a diagram showing, as an example, the configuration of an autonomous driving vehicle driving support system 1000 in which an automatic driving system 1150 and an information presentation control device 160 are integrated, and an automatic driving vehicle 2100 including the automatic driving vehicle driving support system 1000.

Further, the present invention may be realized and provided as an autonomous driving vehicle 100, 1100, or 2100.

In each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution device such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the program that realizes the information presentation control device of each of the above embodiments is as follows.

That is, this program is an information presentation control method in which a computer mounted on an autonomous driving vehicle presents information to a presentation device mounted on the autonomous driving vehicle, and recognizes different objects regarding the surrounding conditions of the autonomous driving vehicle. Autonomous driving of an autonomous vehicle based on at least one of a control that the autonomous vehicle can and cannot control, depending on the step of acquiring selection information indicating the selection status of a plurality of recognition units. A driving information presentation method including a step of causing a presentation device to present driving information regarding at least one of driving by a system or driving by a driver is executed.

Further, among the information presentation control device 160, the automatic driving system 150 or 1500, or the automatic driving vehicle driving support system 1000 realized by the program execution device executing the program, and the functional components of each of these. The above division of functions is an example and may be changed. The scope of the present invention also includes a form realized by arbitrarily dividing or combining these components and functions.

Although the information presentation control device according to one or more embodiments has been described above based on the embodiments, the present invention is not limited to these embodiments. As long as it does not deviate from the gist of the present invention, a form in which various modifications conceived by those skilled in the art are applied to the present embodiment or a form constructed by combining components in different embodiments is also within the scope of one or a plurality of embodiments. May be included within.

The present invention can be used for self-driving cars and the like.

10 Network 100, 1100, 2100 Self-driving car 1000 Self-driving car Driving support system 110 Driving support interface 1151 Driving location information acquisition unit 1152 Driver status information acquisition unit 1153 External environment information acquisition unit 120 Driving operation unit 140 Navigation system 150, 1150 Autonomous driving system 151 Detection unit 152 Recognition unit (recognition unit on the vehicle)
153 Situation judgment unit 154 Operation control unit 155 Own vehicle monitoring unit 156 Selection unit 157 Communication unit 160 Information presentation control device 166 Presentation control unit 180 Storage unit 190 Antenna 200 Recognition system (recognition unit outside the vehicle)
300 communication line 350 base station

Claims (16)

It is an information presentation control device that causes a presentation device mounted on an autonomous vehicle to present information.
A selection information acquisition unit that acquires selection information indicating the selection status of a plurality of recognition units that recognize different targets for the surrounding conditions of the autonomous driving vehicle based on the output of the detection unit.
Image data showing the content of driving by the automatic driving system of the autonomous driving vehicle executed according to the selected recognition unit indicated by the selection information, and execution according to the selected recognition unit. It is provided with a presentation control unit that causes the presentation device to present driving information corresponding to character string data indicating the driving content by the driver, which is necessary for the driving content of the automatic driving system .
The image data is image data including an image related to an object to be recognized by the selected recognition unit.
The character string data is data corresponding to the image data.
Information presentation control device. The information according to claim 1, wherein the selection status of the plurality of recognition units is a status of selection of a recognition unit that executes the recognition of the target or a recognition unit that does not execute the recognition of the target from the plurality of recognition units. Presentation control device. The information presentation control device according to claim 2, wherein the selection information acquisition unit acquires the selection information indicating the selection result of the recognition unit from the plurality of recognition units from the automated driving system. The selection information acquisition unit acquires selection determination material information used for determination for selection of the recognition unit from the plurality of recognition units, and estimates the selection status based on the selection determination material information. The information presentation control device according to claim 2 or 3, wherein the selection information is acquired. The selection information acquisition unit acquires information indicating the operating status of the detection unit or the communication unit of the autonomous driving vehicle for acquiring information used for recognizing the target, and uses the information indicating the operating status as the information. The information presentation control device according to any one of claims 2 to 4, wherein the selection information is acquired by determining a recognition unit to be selected based on the method. The information presentation control device according to claim 4, wherein the selection determination material information includes travel location information indicating the characteristics of the location where the autonomous driving vehicle is traveling, which is acquired based on map information. The information presentation control device according to claim 4, wherein the selection determination material information includes driver state information indicating the physical state of the driver. The information presentation control device according to claim 4, wherein the selection determination material information includes vehicle outside environment information indicating the vehicle outside environment of the autonomous driving vehicle. The information indicating the operating status includes information indicating an abnormality of the detector included in the detection unit.
The information presentation control device according to claim 5, wherein the recognition unit that does not perform recognition of the target includes a recognition unit that uses the detection result output by the detector having the abnormality to recognize the target. The information indicating the operating status includes information indicating the communication quality of the communication unit or the presence / absence of communication execution.
The recognition unit that does not execute the recognition of the target refers to the information communicated by the communication unit in a situation where the communication quality of the communication unit is inferior to a predetermined quality or the communication unit does not execute communication. Including a recognition unit configured to be used for recognition,
The information presentation control device according to claim 5. The information presentation control device according to any one of claims 1 to 10, wherein the presentation control unit controls the presentation of the driving information to the driver using at least one of a sound or an image. The information presentation control device according to any one of claims 1 to 11, wherein the operation information is information including the content of operation that the automatic operation system does not execute or is not executed. The automatic driving system according to claim 1 and
An automated driving vehicle driving support system including the information presentation control device according to any one of claims 1 to 12. The automatic driving system according to claim 1 and
An autonomous vehicle including the information presentation control device according to any one of claims 1 to 12. It is an information presentation control method that causes a presentation device mounted on an autonomous vehicle to present information.
Based on the output of the detection unit, the selection information indicating the selection status of a plurality of recognition units that recognize different targets for the surrounding conditions of the autonomous driving vehicle is acquired.
Image data showing the content of driving by the automatic driving system of the autonomous driving vehicle executed according to the selected recognition unit indicated by the selection information, and execution according to the selected recognition unit. The presenting device is made to present the driving information corresponding to the character string data indicating the driving content by the driver, which is necessary for the driving content of the automatic driving system .
The image data is image data including an image related to an object to be recognized by the selected recognition unit.
The character string data is data associated with the image data.
Information presentation control method. An information presentation program for causing a computer mounted on the autonomous driving vehicle to execute the information presentation control method according to claim 15.

Images