JP2019135150A - Automatic drive control apparatus, and automatic drive control method - Google Patents

Abstract

To provide an automatic drive vehicle which allows any driver to move comfortably.SOLUTION: It is determined whether a driver monitors drive operation or not under automatic drive mode. In addition, in a case where it is determined that the driver monitors the drive operation, contents of the drive operation is previously notified in advance of the drive operation. Thereby, even in a case where the driver is anxious about automatic determination of circumstances and relying on automatic drive operation, a driver may previously recognize contents of the drive operation executed by the automatic drive. As a result, anxiety to the automatic drive may be reduced and comfortable movement under automatic drive mode may be realized.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technique for controlling an autonomous driving vehicle.

An automatic driving vehicle has been developed in which a computer grasps a surrounding situation and performs a driving operation without the driver performing the driving operation (Patent Document 1).
In this self-driving vehicle, the computer grasps the situation performed by the driver during manual driving (for example, grasping the presence or absence of obstacles or vehicles in the vicinity, the state of the traffic light, the state of the own vehicle, etc.) The computer also performs driving operations such as steering wheel operation, braking operation, and accelerator operation according to the situation. Therefore, if an autonomous driving vehicle is realized, the driver is not only freed from the physical burden associated with the driving operation, but also freed from the mental burden of checking the situation around the vehicle and judging the situation. Therefore, it is expected to be able to move to the destination more comfortably than the current situation.

JP2015-111386A

However, even if an autonomous driving vehicle is realized, there is a problem that not all drivers can move comfortably.
This is because in autonomous driving vehicles, the driving operation and surrounding situation judgment are left to the computer, so some drivers feel uneasy about this, and as a result, they constantly monitor the movement of the vehicle. This is because a situation in which the mental burden is hardly alleviated can occur.

  The present invention has been made in view of the above-described problems of the prior art, and is a technique for realizing an autonomous driving vehicle capable of comfortably moving to a destination for any driver. For the purpose of provision.

In order to solve the above-described problem, the automatic driving control device and the automatic driving control method of the present invention determine whether or not the driver is monitoring the driving operation in the automatic driving state. When it is determined that the driver is monitoring the driving operation, the driver is notified of the content of the driving operation prior to the driving operation.
In this way, even if the driver feels uneasy about entrusting automatic driving to the surrounding situation judgment and driving operation accompanying driving, the driver recognizes in advance the content of the driving operation that the automatic driving is going to be performed. Can do. As a result, since anxiety about automatic driving can be reduced, it is possible to move comfortably in the automatic driving state.

It is explanatory drawing which shows the automatic driving vehicle 1 carrying the automatic driving control apparatus 100 of a present Example. It is a block diagram which shows the internal structure of the automatic driving | operation control apparatus 100 of a present Example. It is a flowchart of the first half part of the automatic driving | running control process which the automatic driving | operation control apparatus 100 performs. It is a flowchart of the second half part of an automatic operation control process. It is explanatory drawing which showed a mode that the content of driving operation is notified in advance by moving the headrest 8h of the driver's seat 8. FIG. FIG. 6 is an explanatory diagram showing a relationship between a backward inclination angle of a backrest portion 8b of a driver seat 8 and a driver's line of sight. It is explanatory drawing which illustrated the motion correction coefficient determined according to the back inclination angle of the backrest part 8b. It is a flowchart of the monitoring determination process which determines whether the driver | operator is monitoring the driving operation in an automatic driving | running state. It is explanatory drawing which shows a mode that the traffic signal displayed with respect to the own vehicle was detected in the front image. It is explanatory drawing about the 1st modification which moves the headrest 8h and notifies the content of driving operation. It is explanatory drawing about the 2nd modification which moves the backrest part 8b and notifies the content of driving operation. It is explanatory drawing about the other aspect of the 2nd modification which moves the driver's seat 8 whole, and notifies the content of driving operation. It is explanatory drawing about the 2nd modification which notifies the content of driving operation by moving the support parts 8R and 8L of the driver's seat 8. FIG. It is explanatory drawing about the 3rd modification which notifies the content of driving operation by moving the steering wheel.

Hereinafter, examples will be described in order to clarify the contents of the present invention described above.
A. Device configuration :
FIG. 1 shows a rough configuration of an automatic driving vehicle 1 equipped with an automatic driving control device 100 of the present embodiment. As shown in the figure, the autonomous driving vehicle 1 is equipped with a steering handle 2, a steering shaft 3, an accelerator pedal 4, a brake pedal 5, and the like. The steering shaft 3 is provided with a steering wheel drive portion 2m for steering the steering handle 2. The accelerator pedal 4 is provided with an accelerator driving portion 4m for driving the accelerator pedal 4. Further, the brake pedal 5 is provided with a brake. A brake drive unit 5m for driving the pedal 5 is mounted.

The automatic driving vehicle 1 includes an in-vehicle camera 6 that captures an image in front of the traveling direction (hereinafter, a front image), a driver seat camera 7 that is mounted in front of the driver's seat and captures a driver's face image, A current position detection unit 9 that detects the current position of the host vehicle based on information such as a positioning signal from a positioning satellite, a map information storage unit 10 that stores map information, and the like are mounted.
The automatic driving control device 100 detects the travel position of the host vehicle and the travel route to the destination based on information from the current position detection unit 9 and the map information storage unit 10. Then, automatic driving is performed by driving the handle driving unit 2m, the accelerator driving unit 4m, and the brake driving unit 5m while analyzing the front image acquired from the in-vehicle camera 6 and grasping the surrounding situation.

However, in the automatic driving state, surrounding situation grasping and driving operation are left to the automatic driving control device 100. For this reason, for a driver who feels uneasy about this, the automatic driving state is not necessarily a comfortable state. For example, when the traffic light in front changed to “red” and was worried that “Is it really stopped? If it does n’t stop, it ’s an accident.” It is difficult to become a state. Or if you saw that the traffic light changed to “blue” and were worried about “Is it going to start properly?”, It is difficult to feel the automatic driving state as a comfortable state.
Therefore, in the autonomous driving vehicle 1 of the present embodiment, as shown in FIG. 1, headrest driving units 8m and 8t for driving the headrest 8h are mounted on the driver's seat 8 where the driver sits. Then, before the automatic driving control device 100 drives the handle driving unit 2m, the accelerator driving unit 4m, and the brake driving unit 5m to perform the driving operation, the head driving unit 8m, 8t is driven to move the headrest 8h. The details of the driving operation will be announced. In this way, since the content of the driving operation that the automatic driving control apparatus 100 is trying to recognize can be recognized, a sense of security can be given to the driver who cannot fully believe the driving operation of the automatic driving control apparatus 100. Can do. As a result, the driver's anxiety can be reduced, so that it is possible to move to the destination while comfortably feeling the automatic driving state.

FIG. 2 shows a rough internal structure of the automatic operation control device 100 of this embodiment. As shown in the figure, the automatic driving control apparatus 100 includes an automatic driving operation unit 101, a face image acquisition unit 102, a direction detection unit 103, a front image acquisition unit 104, a traffic light detection unit 105, and a driving monitoring determination unit 106. And a driving operation notifying unit 107.
These “parts” are classified into the interior of the automatic driving control device 100 for the sake of convenience, focusing on the function that the automatic driving control device 100 has for notifying the driver of the driving operation in the automatic driving state. Abstract concept. Therefore, it does not represent that the automatic operation control apparatus 100 is physically divided into these “parts”. These “units” can be realized as a computer program executed by the CPU, can be realized as an electronic circuit including an LSI or a memory, or can be realized by combining them. it can.

  The automatic driving operation unit 101 is connected to the current position detection unit 9 and the map information storage unit 10, acquires the current position of the host vehicle from the current position detection unit 9, and the host vehicle from the map information storage unit 10. Get map information around. The current position detection unit 9 is based on information on a positioning signal received from a positioning satellite, information on a moving distance acquired from a vehicle speed sensor (not shown), and information on a moving direction acquired from a direction sensor (not shown). The position can be detected. The map information storage unit 10 stores map information including road shapes.

The automatic driving operation unit 101 is also connected to the front image acquisition unit 104.
The front image acquisition unit 104 acquires a front image captured by the in-vehicle camera 6 at a predetermined imaging cycle and outputs the acquired front image to the automatic driving operation unit 101. The automatic driving operation unit 101 grasps the situation around the automatic driving vehicle 1 by analyzing the front image received from the front image acquisition unit 104. Then, based on the current position of the host vehicle received from the current position detection unit 9, map information, and surrounding conditions, the steering wheel drive unit 2m, the accelerator drive unit 4m, the brake drive unit 5m, and the like are driven automatically. Realize driving conditions.

The face image acquisition unit 102 is connected to the driver's seat camera 7 and acquires the face image of the driver sitting in the driver's seat 8 (see FIG. 1). The driver's seat camera 7 captures a clear face image at a constant cycle using infrared rays that are invisible to humans, and outputs the image to the face image acquisition unit 102.
The direction detection unit 103 receives the face image taken by the driver's seat camera 7 from the face image acquisition unit 102 and analyzes the face image to detect the direction in which the driver is looking. Here, various methods can be used as the direction in which the driver is looking, but the direction detection unit 103 of the present embodiment detects the direction of the face and the direction of the line of sight 2 One way is to detect the direction the driver is looking.

The traffic signal detection unit 105 detects a traffic signal to be displayed on the host vehicle from the front image by acquiring the front image captured by the driver's seat camera 7 from the front image acquisition unit 104 and analyzing the front image. .
The driving monitoring determination unit 106 determines whether or not the driver is monitoring the content of the driving operation by the automatic driving operation unit 101 in the automatic driving state. Although the detailed method of determination will be described later, the information acquired from the direction detection unit 103 regarding the direction the driver is looking at, the information on the traffic signal position acquired from the traffic signal detection unit 105, and acquired from the automatic driving operation unit 101 Judgment is made using information on the driving operation.
When it is determined that the driver is monitoring the driving operation in the automatic driving state, the driving operation notifying unit 107 automatically drives the content of the driving operation before the automatic driving operation unit 101 executes the driving operation. Obtained from the operation unit 101. Then, by driving the headrest driving units 8m and 8t mounted on the driver's seat 8 to move the headrest 8h, the driver is notified of the details of the driving operation.
The automatic driving control apparatus 100 according to the present embodiment including these “parts” notifies the driver of the content of the driving operation in the automatic driving state by executing the following processing. For this reason, even a driver who feels uneasy about entrusting driving to the automatic driving control apparatus 100 can move to the destination while feeling comfortable in the automatic driving state.

B. Automatic operation control processing:
3 and 4 show a flowchart of the automatic driving control process executed by the automatic driving control device 100 of this embodiment. As shown in the figure, when the automatic operation control process is started, first, it is determined whether or not the current operation state is an automatic operation state (S100). Since the automatic driving state is realized by the driving operation of the automatic driving control device 100, the automatic driving control device 100 can easily determine whether or not the current driving state is the automatic driving state.
As a result, when it is determined that the current operation state is not the automatic operation state (S100: no), the subsequent processing is unnecessary, and it is determined whether or not to end the operation (S111 in FIG. 4). And when not complete | finishing driving | operation (S111: no), it returns to the head of a process again and it is judged whether the present driving | running state is an automatic driving | running state (S100 of FIG. 3).
If the driver finishes driving while repeating such operations, it is determined as “yes” in S111 of FIG. 4 and the automatic driving control process is ended.
On the other hand, if the driving state is switched to the automatic driving state according to the driver's setting, “yes” is determined in S100 of FIG.

In this case (S100: yes), the current position of the host vehicle and the movement route are grasped based on the information acquired from the current position detection unit 9 and the map information storage unit 10 (S101). When detecting the current position of the host vehicle, it can be detected using a positioning signal from a positioning satellite. It is also possible to improve the detection accuracy of the current position by reading the road shape from the map information stored in the map information storage unit 10 and performing so-called map matching.
Subsequently, by acquiring a front image captured by the in-vehicle camera 6 and analyzing the image, the situation of the surroundings of the vehicle including the position of the lane in which the vehicle is running, the presence of obstacles, the presence of pedestrians, etc. Grasping (S102).
Then, the content of the driving operation is determined based on information such as the current position of the host vehicle, map information, and surrounding conditions (S103). Here, the contents of the driving operation include a steering wheel driving unit 2m, an accelerator driving unit 4m, and a brake driving unit 5m for realizing stop, start, acceleration and deceleration of the own vehicle, and further right steering and left steering. It is a driving amount.

When the content of the driving operation is determined, the monitoring determination process is subsequently started (S200). As will be described in detail later, in the monitoring determination process, it is determined whether or not the driver is monitoring the driving operation during the automatic driving state. That is, in the first place, the automatic driving control device 100 executes the driving operation in the automatic driving state, so that the driver can leave the driving operation to the automatic driving control device 100. However, some drivers feel uneasy about leaving the driving operation to the automatic driving control apparatus 100. And such a driver | operator monitors whether the automatic driving | operation control apparatus 100 performs suitable driving | operation operation according to the surrounding condition.
Therefore, if it is known whether or not the driver is monitoring the content of the driving operation during the automatic driving state, it can be determined whether or not the driver trusts the automatic driving. And it is thought that it is possible to relieve a driver | operator by notifying the content of driving operation with respect to the driver who does not trust automatic driving. On the other hand, for the driver who trusts the automatic driving, since the content of the driving operation is unnecessary information, there is a risk that it will be annoying even if the driver is notified of the driving operation every time.
Therefore, in the automatic driving control process of this embodiment, when the content of the driving operation is determined in the automatic driving state (S103), it is determined whether or not the driver is monitoring the driving operation executed by the automatic driving control device 100. . A method for determining whether or not the driver is monitoring the driving operation will be described in detail later. The obtained determination result is stored in a built-in memory (not shown) of the automatic driving control apparatus 100.

Subsequently, it is determined whether or not the driver is monitoring the driving operation (S104). As described above, since the determination result obtained by the monitoring determination process is stored in the built-in memory of the automatic driving control apparatus 100, the driver can perform the driving operation by referring to the determination result stored in the memory. It is possible to immediately determine whether or not monitoring is performed.
As a result, when it is determined that the driver is monitoring the driving operation (S104: yes), the notice mode and the amount of notice operation are determined according to the content of the driving operation previously determined in S103 (S105). .

FIG. 5 shows a method for determining a notice mode and a notice action amount in accordance with a driving operation. In the present embodiment, the driving operation is notified in advance by moving the headrest 8h of the driver's seat 8, and the mode of moving the headrest 8h (that is, the notification mode) is predetermined according to the content of the driving operation. .
For example, when the content of the driving operation is starting or accelerating, a notice mode in which the headrest 8h is moved backward is associated with the driving operation as shown by a white arrow in FIG. Is set. Of these, a notice mode for moving the headrest 8h small and suddenly is set for starting, and a notice mode for slowly moving the headrest 8h with a relatively large amount of movement is set for acceleration. Yes. Further, the amount of movement of the headrest 8h in the case of acceleration increases as the degree of acceleration (that is, acceleration) that the autonomous driving vehicle 1 is going to perform from now on increases.

On the contrary, when the content of the driving operation is to stop or decelerate, as shown by the black arrow in FIG. 5A, the notice mode for moving the headrest 8h forward corresponds to the driving operation. Is set. Of these, a notice mode for moving the headrest 8h small and suddenly is set for the stop, and a notice mode for slowly moving the headrest 8h with a relatively large amount of movement is set for the deceleration. Further, the amount of movement of the headrest 8h during deceleration increases as the deceleration that the autonomous driving vehicle 1 is going to perform increases.
Note that the amount of movement of the headrest 8h is the amount of advance notice for the start, stop, acceleration and deceleration of the autonomous driving vehicle 1. Further, the movement of moving the headrest 8h in the front-rear direction can be realized by driving the headrest driving unit 8m mounted on the driver's seat 8.

The above describes the case where the content of the driving operation is start, acceleration, stop, and deceleration. However, when the content of the driving operation is left and right steering, the following notice mode is set. ing. First, for the right steering, as shown by a white arrow in FIG. 5B, a notice mode is set such that the headrest 8h is rotated clockwise. Further, the rotation amount of the headrest 8h at this time increases as the steering amount of the right steering that the autonomous driving vehicle 1 is going to perform increases.
On the contrary, for the left steering, as shown by a black arrow in FIG. 5B, a notice mode is set such that the headrest 8h is rotated counterclockwise. Further, the amount of rotation of the headrest 8h increases as the steering amount of the left steering that the autonomous driving vehicle 1 is about to perform increases.
Note that the amount of rotation of the headrest 8h is the amount of advance notice for the right steering and the left steering of the autonomous driving vehicle 1. The movement of rotating the headrest 8h clockwise or counterclockwise can be realized by driving the headrest driving unit 8t mounted on the headrest 8h.
In S105 shown in FIG. 3, the notice mode and the notice operation amount using the headrest 8h are determined as described above.

Subsequently, the automatic driving control device 100 detects the backward tilt angle of the backrest portion 8b of the driver's seat 8 (S106). Here, the backward inclination angle of the backrest 8b is an angle at which the backrest 8b is tilted backward from 0 degrees when the backrest 8b is vertically set to 0 degrees.
As shown in FIG. 6, an angle sensor 8 s for detecting a backward inclination angle of the backrest 8 b is mounted on a portion where the backrest 8 b of the driver seat 8 is attached to the seat 8 a. Therefore, the automatic operation control device 100 can detect the backward inclination angle of the backrest portion 8b based on the output of the angle sensor 8s.

  When the backward inclination angle of the backrest 8b is detected, an operation correction coefficient corresponding to the backward inclination angle is determined (S107 in FIG. 3). Here, the motion correction coefficient is a correction coefficient used to correct the amount of advance motion determined in S105. The reason why the motion correction coefficient is determined according to the back tilt angle of the backrest 8b is as follows.

FIG. 6A shows a state in which the driver is seated on the driver's seat 8 whose rearward tilt angle is set to 15 degrees. As shown in the drawing, when the rearward tilt angle is 15 degrees, the driver sits on the driver's seat 8 in a slightly cramped posture. In addition, the driver at this time is in a state of looking straight ahead as indicated by the dashed arrow in the figure.
On the other hand, FIG. 6B shows a state in which the driver is seated in the driver's seat 8 with the backward tilt angle set to 45 degrees. As shown in the drawing, when the rearward tilt angle is 45 degrees, the driver can sit in the driver's seat 8 in an easy posture. In addition, as shown by the broken arrow in the figure, the driver at this time faces upward and it is difficult to check the situation ahead.

In the automatic driving state, the driver does not need to perform a driving operation. Nevertheless, as shown in FIG. 6 (a), the fact that the driver is sitting in such a posture that the front can be seen straight even if he / she endures a cramped posture is due to the automatic driving control device 100. You can think of it as an indication that you do not trust the driving operation (that is, automatic driving).
On the other hand, as shown in FIG. 6 (b), a driver sitting in an easy posture in which the front cannot be seen is unlikely to grasp the situation in front, so such a driver is You can think that you trust driving.

From the above, it can be considered that the reliability of the driver with respect to the automatic driving is reflected in the backward inclination angle of the backrest 8b of the driver's seat 8 in the automatic driving state. In other words, the smaller the rearward tilt angle, the lower the confidence level for automatic driving. In particular, when the rearward tilting angle is smaller than 15 degrees as shown in FIG. Good. And it is considered important for such a driver to give a notice so that the contents of the driving operation can be clearly understood.
Conversely, as the rearward tilt angle increases, the confidence level for automatic driving increases. In particular, when the rearward tilt angle exceeds 45 degrees as shown in FIG. You can think of it. For such a driver, it is not necessary to give notice of the content of the driving operation. Rather, the driver may feel annoying by giving a notice, so it is better not to give a notice.
Therefore, in the automatic driving control apparatus 100 of the present embodiment, an operation correction coefficient is set according to the back tilt angle of the backrest 8b, and the advance operation amount is corrected using the operation correction coefficient.

  FIG. 7 illustrates a state in which an operation correction coefficient for the backward tilt angle is stored in a built-in memory (not shown) of the automatic operation control apparatus 100. As shown in the figure, the motion correction coefficient “1.0” is stored in association with the backward tilt angle of less than 15 degrees. In addition, the motion correction coefficient “0.8” is stored in association with the backward inclination angle of 15 degrees or more and less than 25 degrees, and the backward inclination angle of 25 degrees or more and less than 35 degrees is stored. The motion correction coefficient “0.3” is stored for the backward tilt angle of the motion correction coefficient “0.6” of 35 degrees or more and less than 45 degrees. Furthermore, an operation correction coefficient “0.0” is stored for a backward tilt angle of 45 degrees or more.

  Here, the motion correction coefficient for the backward tilt angle of 45 degrees or more is 0.0 because the driver who is considered to fully trust the automatic driving as shown in FIG. 6B. On the other hand, this is to prevent the advance notice.

Further, the amount of decrease in the motion correction coefficient with respect to the increase in the rearward tilt angle is larger in the angle range of 35 degrees or more than the angle range in which the rearward tilt angle is less than 35 degrees. This is because when the rearward tilt angle is increased, the driver's head firmly contacts the headrest 8h, so that the movement of the headrest 8h is easily transmitted to the driver.
That is, for example, if the rearward tilt angle is set to 40 degrees, the driver is considered to trust the automatic driving to a considerable extent if not completely, and for such a driver, The need to notify the details of driving operations is considered to be small. Therefore, in order to avoid the driver feeling annoying the notice, it is desirable to correct the notice action amount to be small.
However, since the backrest 8b is tilted backward, the driver's head is firmly in contact with the headrest 8h, and the movement of the headrest 8h is easily transmitted to the driver. However, the effect may not be sufficiently obtained. Therefore, in the automatic operation control apparatus 100 of the present embodiment, in the angle range where the rearward tilt angle is 35 degrees or more, the reduction amount of the operation correction coefficient with respect to the increase of the rearward tilt angle is made larger than the angle range of less than 35 degrees. Yes.

In S107 of FIG. 3, the motion correction coefficient corresponding to the back tilt angle of the backrest 8b is determined as described above.
Subsequently, the amount of advance notice is corrected by multiplying the amount of advance notice determined in S105 by the action correction coefficient (S108).

Then, the content of the driving operation is notified in advance by driving the headrest driving unit 8m or the headrest driving unit 8t and moving the headrest 8h with the corrected amount of notification operation (S109 in FIG. 4).
For example, when the automatic driving vehicle 1 is going to decelerate, the headrest 8h is moved forward, and when it is going to be accelerated, the headrest 8h is moved backward (see FIG. 5A). Or when the autonomous driving vehicle 1 is going to steer right or left, the headrest 8h is rotated clockwise or counterclockwise (see FIG. 5B). Therefore, the driver can recognize in advance the content of the driving operation that the autonomous driving vehicle 1 is about to perform by the movement of the headrest 8h.
Further, in the automatic driving state, the driver has released his hand from the steering handle 2, so that the driver naturally takes a posture with his head on the headrest 8h. Therefore, by moving the headrest 8h, the driver can be surely recognized the contents of the driving operation. Nevertheless, the advance notice due to the movement of the headrest 8h is as if the driver is receiving a light massage, and may cause a burden on the driver as if using a sound or an image. Nor.

  If the driver is notified of the details of the driving operation in this way, the driving operation is performed by driving the handle driving unit 2m, the accelerator driving unit 4m, and the brake driving unit 5m in accordance with the driving operation content determined in S103 of FIG. (S110 in FIG. 4).

The process for the case where it is determined that the driver is monitoring the driving operation during the automatic driving (S104: yes in FIG. 3) has been described.
On the other hand, when the driver does not monitor the driving operation (S104: no), it is not necessary to notify the driver of the content of the driving operation. Therefore, a series of processing for notifying the driving operation (ie, S105 to S109) is omitted, and the driving operation is immediately performed (S110).

  When the driving operation is carried out in this manner (S110), it is determined whether or not the driving is to be ended (S111). If the driving is not to be ended (S111: no), the driving operation is continued again. After determining whether or not (S100 in FIG. 3), the above-described series of processing is repeated.

  In order to appropriately notify the driver of the content of the driving operation in the automatic driving control process described above, it is important to appropriately determine whether or not the driver is monitoring the content of the driving operation. In the automatic driving control apparatus 100 of the present embodiment, it is determined whether or not the driver is monitoring the content of the driving operation by performing the following monitoring determination process in the automatic driving control process.

C. Monitoring judgment processing:
FIG. 8 shows a flowchart of the monitoring determination process executed by the automatic driving control apparatus 100 of this embodiment. This process is a process executed by the automatic driving control device 100 when it is determined that the driving state is the automatic driving state in the automatic driving control process described above.

As shown in the figure, when the monitoring determination process (S200) is started, a driver's face image is acquired (S201). The driver's seat camera 7 captures the driver's face image at a constant cycle, and acquires the latest face image captured by the driver's seat camera 7 in S201.
Subsequently, the orientation of the driver's face is detected by analyzing the face image (S202). The orientation of the driver's face can be detected by various methods. For example, the positions of eyes and nose are detected from the face image, and the orientation of the face is detected based on those positions in the face. If the face is facing directly in front, the position where the nose is detected is the center of the face, and eyes are detected on both sides at the same distance from the nose. In contrast, as the face turns to the left, the position of the nose and eyes shifts to the right in the face image. If the face is tilted to some extent, the driver's left eye (that is, the right eye on the face image) Will not be detected. Therefore, the orientation of the face can be detected based on the position where the eyes and nose are detected in the face image.

  Next, the direction of the driver's line of sight is detected (S203). The direction of the driver's line of sight can also be detected by various methods. For example, an image in which eyes are shown in the driver's face image is detected, and a portion in which black eyes are shown in the eye image is further detected. If the line of sight is looking straight ahead, the black eye position should be the center of the eye. On the other hand, as the line of sight faces the left side, the position of the black eye shifts from the center of the eye to the right side on the image. Conversely, as the line of sight faces the right side, the position of the black eye shifts from the center of the eye to the left side on the image. Therefore, the direction of the line of sight can be detected based on the position of the black eye in the image in which the eyes are shown.

  When the direction of the driver's face and the direction of the line of sight are thus detected, the steering angle of the autonomous driving vehicle 1 is acquired (S204). As described above, the monitoring determination process is a process executed in the automatic driving state, and the automatic driving control device 100 steers the steering handle 2 during the automatic driving state. Therefore, the automatic driving control apparatus 100 can immediately acquire the steering angle of the steering handle 2 at the present time.

Subsequently, it is determined whether or not the direction of the acquired steering angle is a straight traveling direction (S205). If the acquired steering angle is within a predetermined angle range with 0 degree as the center, it can be determined that the vehicle is traveling straight.
As a result, when the direction of the steering angle is not the straight direction (S205: no), it is determined whether or not the driver is looking at the direction of the steering angle (S206). For this determination, the driver's face orientation acquired in S202 and the line-of-sight orientation acquired in S203 are used. That is, when at least one of the driver's face direction or the line-of-sight direction is directed to the steering angle direction, it is determined that the driver is looking at the steering angle direction (S206: yes).
Here, the direction of the driver's face or line of sight can be obtained in rough directions such as “front direction”, “right diagonal direction”, “right direction”, “left diagonal direction”, “left direction”. If so, it is determined whether or not the direction of the steering angle corresponds to a rough direction. On the other hand, when the orientation of the face or the direction of the line of sight is obtained as an angle, the deviation between the angle indicating the direction of the face or the line of sight and the steering angle is obtained, and if the deviation is within a predetermined angle, It is determined that the direction of the face or line of sight is in the direction of the steering angle.

  If the driver is looking at the direction of the steering angle (S206: yes), it is determined that the driver is monitoring the driving operation, and the result is stored in the built-in memory of the automatic driving control device 100. (S210). In other words, since the driver does not need to drive in the automatic driving state, he / she looks at the surrounding scenery and streets, just like a taxi, or looks at the books and mobile terminals at hand. Seem. Nevertheless, for example, if you are steering to the right in an automatic driving state, check the right direction, and if you are steering to the left, check the left direction. You can think of it as a sign of being. Therefore, when the driver is looking at the direction of the steering angle (S206: yes), it is determined that the driver is monitoring the driving operation, and that fact is stored in the memory (S210).

  On the other hand, when the direction of the steering angle acquired in S204 is a straight direction (S205: yes), or when it is determined that the driver is not looking at the direction of the steering angle (S206: no), the in-vehicle camera 6 A photographed front image is acquired (S207). Then, by analyzing the front image, it is determined whether or not the traffic signal to be displayed on the host vehicle is reflected in the front image (S208).

FIG. 9 illustrates a state in which a traffic signal to be displayed on the host vehicle is detected from the front image. Of course, the traffic light to be displayed on the vehicle is shown with the front facing this side. And since the traffic signal seen from the front has a characteristic shape, if such a traffic signal is reflected in the front image, it can be easily detected. In the example shown in FIG. 9, corresponding traffic signals are detected in the areas A and B in the front image.
As described above, when a traffic signal to be displayed for the host vehicle is detected in the front image, it is determined as “yes” in S208, and the direction of the driver's line of sight detected in S203 is subsequently It is determined whether or not it is facing the direction (S209). In the example illustrated in FIG. 9, it is determined whether or not the driver's line of sight is facing either region A or region B.

As a result, when the driver's line of sight is directed to the traffic light (S209: yes), it is determined that the driver is monitoring the driving operation, and the fact is built in the automatic driving control device 100. Store in the memory (S210).
On the other hand, when the direction of the driver's line of sight does not face the direction of the traffic light (S209: no), the driver does not see the operation direction or the direction of the traffic light. Therefore, it is determined that the driving operation is not monitored, and the fact is stored in the built-in memory of the automatic driving control apparatus 100 (S211).
Further, when the traffic signal to be displayed for the host vehicle is not shown in the front image (S208: no), the driver determines that the driver does not monitor the driving operation, and the automatic driving control device 100 indicates that. (S211).

  When the determination result as to whether or not the driver is monitoring the driving operation is stored as described above (S210, S211), the monitoring determination process shown in FIG. 8 is terminated, and FIGS. Return to the automatic operation control process shown. As described above, in the automatic driving control process, it is determined whether or not the driver is monitoring the driving operation by reading the stored determination result (S104 in FIG. 3), and the driving operation is monitored. (S104: yes), the driver is notified of the details of the driving operation (S110 in FIG. 4). On the other hand, when it is determined that the driving operation is not monitored (S104: no), the driving operation is not notified.

  In this way, the driver who feels uneasy about entrusting driving to the automatic driving control device 100 can be made to recognize in advance the content of the driving operation performed by the automatic driving control device 100. For this reason, since driver | operator's anxiety can be reduced, it becomes possible to move to a destination comfortably. In addition, for the driver who does not feel uneasy about entrusting driving to the automatic driving control device 100, the content of the driving operation is not notified in advance. For this reason, the driver does not feel bothered by the notice.

  Further, even if it is determined that the driver is monitoring the driving operation by looking at the direction of the operation angle, etc., if the driver defeats the backrest 8b of the driver's seat 8, the driving operation is performed. The content of is not notified in advance, or the headrest 8h moves slightly and is only notified in advance. For this reason, it is possible to avoid that the driver feels annoying the notice.

In the above-described monitoring determination process, it has been described that it is determined that the driving operation is being monitored if the driver looks at the direction of the steering angle (S206: yes in FIG. 8).
However, when the driver looks at the direction of the steering angle over a certain period of time, or when the ratio of the time during which the direction of the steering angle is viewed within a certain period of time is equal to or greater than a predetermined ratio, It may be determined that the driving operation is being monitored.
In this way, when the driver moves the face direction or the direction of the line of sight, the face image is taken, and the face direction or the direction of the line of sight reflected in the face image happens to be in the direction of the steering angle. Therefore, it can be avoided that the driver judges that the driving operation is being monitored.

Alternatively, the final determination result may be determined based on a plurality of determination results obtained so far, instead of adopting the determination result every time as it is. For example, there are many judgment results that the judgment results of each time are stored once, the judgment results for the past five times including the judgment results stored at this time are read out, and the driving operation is monitored in them. In such a case, the final determination result may indicate that the driving operation is being monitored.
Alternatively, if the determination results for the past three times including the determination result of this time are all the determination results indicating that the driving operation is being monitored, it is finally determined that the driving operation is being monitored. If it is not so, the final determination result may be that the driving operation is not monitored.
Even in this case, the face image is taken while the driver is moving the face direction or the direction of the line of sight, and the face direction or the direction of the line of sight reflected in the face image happens to change the direction of the steering angle. Therefore, it can be avoided that the driver judges that the driving operation is being monitored.

D. Modified example:
Various modifications exist in the above-described embodiment. Hereinafter, these modified examples will be briefly described with a focus on differences from the present embodiment.
(1) First modification:
In the present embodiment described above, the headrest 8h built in the backrest 8b is moved forward and backward by tilting the headrest 8h, and the headrest 8h is rotated by the headrest drive 8t built in the headrest 8h. explained.
On the other hand, by mounting the two headrest drive units 8c and 8e that can be expanded and contracted inside the headrest 8h, a surface on which the driver of the headrest 8h places the head (hereinafter referred to as a head placement surface) is provided. It may be moved.

FIG. 10 shows a rough structure of the headrest 8h of the first modified example. As shown in the figure, inside the headrest 8h of the first modified example, two headrest driving portions 8c and 8e are mounted side by side on the left and right. These headrest drive units 8c and 8e can be expanded and contracted under the control of the automatic operation control device 100.
Then, when the headrest driving unit 8c is extended and the headrest driving unit 8e is contracted, the right side (left side in the drawing) of the head resting surface of the headrest 8h is deformed. Conversely, when the headrest driving unit 8c is contracted and the headrest driving unit 8e is extended, the left side (right side in the drawing) of the headrest mounting surface of the headrest 8h is deformed so as to rise.
Further, when both the headrest driving unit 8c and the headrest driving unit 8e are extended, the head placement surface of the headrest 8h moves forward, and when both the headrest driving unit 8c and the headrest driving unit 8e are contracted, the headrest 8h The head mounting surface moves backward.

  Therefore, by appropriately driving the headrest driving units 8c and 8e, it is possible to realize a movement similar to the movement of the headrest 8h described above with reference to FIGS. 5A and 5B. For this reason, even if the headrest 8h of the first modification is used, it is possible to notify the driver of the details of the driving operation by the movement of the headrest 8h as in the above-described embodiment.

(2) Second modification:
In the above-described embodiment and the first modification, the description has been made on the assumption that the driver is notified of the content of the driving operation by the movement of the headrest 8h. However, not only the headrest 8h but also a part of the driver's seat 8 can be driven to notify the content of the driving operation.

For example, as illustrated in FIG. 11, a driving unit 8 d that drives the backrest unit 8 b may be provided in the driver seat 8, and the driving unit 8 d may be driven by the control of the automatic driving control device 100.
For example, before starting or accelerating, as shown by the white arrow in FIG. 11, the backrest 8b is moved to tilt backward, and before stopping or before deceleration, it is indicated by the black arrow. As described above, the backrest 8b is moved so as to be tilted forward.
Alternatively, as illustrated in FIG. 12, a drive unit 8 g that moves the entire driver seat 8 is provided in the seat 8 a of the driver seat 8, and the drive unit 8 g is driven by the control of the automatic operation control device 100. good.
For example, as shown by the white arrow in FIG. 12 before starting or accelerating, the entire driver's seat 8 is moved backward, as indicated by the black arrow before stopping or before deceleration. Next, the driver's seat 8 is moved forward.

Further, as shown in FIG. 13, the drive units 8Rd and 8Ld are provided in the support units 8R and 8L provided on the left and right sides of the driver's seat 8, and the drive units 8Rd and 8Ld are controlled by the control of the automatic operation control device 100. 8Ld may be driven.
For example, before left steering, the drive units 8Rd and 8Ld are driven such that the right support unit 8R and the left support unit 8L are inclined in the direction indicated by the black arrows in the drawing.
Further, before the right steering, the drive units 8Rd and 8Ld are driven so that the right support unit 8R and the left support unit 8L are inclined in the directions indicated by white arrows in the drawing.

  In this way, it is possible to notify the driver of the details of the driving operation by the movement of the backrest part 8b, the movement of the entire driver seat 8, the movements of the support parts 8R and 8L, and the like.

(3) Third modification:
Further, since the driver who should give notice of the content of the driving operation is looking forward in the traveling direction of the vehicle even in the automatic driving state, the steering handle 2 is considered to be in view.
Therefore, as shown in FIG. 14, a handle moving unit 2 d is provided in the steering handle 2, and the position of the steering handle 2 is moved by driving the handle moving unit 2 d under the control of the automatic driving control device 100. It is also possible to make it.
For example, before starting or accelerating, the steering handle 2 is moved upward as indicated by a black arrow in FIG. 14A, and indicated by a white arrow before stopping or before deceleration. As described above, the steering handle 2 is moved downward.
Further, before the left steering, as shown by the black arrow in FIG. 14B, the steering handle 2 is moved to the left, and before the right steering, as indicated by the white arrow. Then, the steering handle 2 is moved rightward.
In this way, it is possible to notify the driver of the details of the driving operation by the movement of the steering wheel 2.

  While various embodiments and modifications have been described above, the present invention is not limited to the above-described various embodiments and modifications, and can be implemented in various modes without departing from the scope of the invention.

DESCRIPTION OF SYMBOLS 1 ... Auto-driving vehicle, 2 ... Steering handle, 2d ... Handle moving part,
6 ... In-vehicle camera, 7 ... Driver seat camera, 8 ... Driver seat,
8b ... backrest part, 8c, 8e, 8m, 8t ... headrest drive part,
8h ... headrest, 8s ... angle sensor, 9 ... current position detector,
DESCRIPTION OF SYMBOLS 10 ... Map information storage part, 100 ... Automatic driving | operation control apparatus, 101 ... Automatic driving | operation operation part,
102 ... face image acquisition unit, 103 ... direction detection unit, 104 ... front image acquisition unit,
105 ... Traffic light detection unit, 106 ... Driving monitoring determination unit, 107 ... Driving operation notice unit.

Claims (13)

It is mounted on an automatic driving vehicle (1) that can be switched between a manual driving state in which a driving operation is performed by a driver and an automatic driving state in which the driving operation is performed without depending on the driver, and controls the automatic driving vehicle. An automatic operation control device (100) that
An automatic driving operation unit (101) for performing the driving operation on the automatic driving vehicle in the automatic driving state;
A face image acquisition unit (102) for acquiring the driver's face image;
A direction detection unit (103) for detecting a direction in which the driver is looking based on the face image;
When the steering direction of the automatic driving vehicle is acquired from the automatic driving operation unit and the driver is looking at the steering direction of the automatic driving vehicle, the driving operation in the automatic driving state is performed by the driver. An operation monitoring determination unit (106) that determines that the
When it is determined that the driver is monitoring the driving operation, prior to the driving operation by the automatic driving operation unit, a driving operation notifying unit that notifies the driver of the content of the driving operation ( 107) and an automatic operation control device. It is mounted on an automatic driving vehicle (1) that can be switched between a manual driving state in which a driving operation is performed by a driver and an automatic driving state in which the driving operation is performed without depending on the driver, and controls the automatic driving vehicle. An automatic operation control device (100) that
An automatic driving operation unit (101) for performing the driving operation on the automatic driving vehicle in the automatic driving state;
A face image acquisition unit (102) for acquiring the driver's face image;
A direction detection unit (103) for detecting a direction in which the driver is looking based on the face image;
A front image acquisition unit (104) for acquiring a front image obtained by photographing the front in the traveling direction from the autonomous driving vehicle;
A traffic signal detector (105) for detecting a traffic signal to be displayed on the vehicle from the front image;
A driving monitoring determination unit (106) for determining that the driver is monitoring the driving operation in the automatic driving state when the driver is looking at the direction of the traffic light;
When it is determined that the driver is monitoring the driving operation, prior to the driving operation by the automatic driving operation unit, a driving operation notifying unit that notifies the driver of the content of the driving operation ( 107) and an automatic operation control device. The automatic operation control device according to claim 1 or 2,
The automatic driving control device, wherein the direction detecting unit detects a direction in which the driver is looking by detecting a direction of the driver's line of sight from the face image. It is mounted on an automatic driving vehicle (1) that can be switched between a manual driving state in which a driving operation is performed by a driver and an automatic driving state in which the driving operation is performed without depending on the driver, and controls the automatic driving vehicle. An automatic operation control device (100) that
An automatic driving operation unit (101) for performing the driving operation on the automatic driving vehicle in the automatic driving state;
A driving monitoring determination unit (106) for determining whether or not the driver is monitoring the driving operation in the automatic driving state;
A driver seat drive unit (8m, 8t, 8c, 8e) for moving the headrest of the driver seat;
When it is determined that the driver is monitoring the driving operation, the driver's seat driving unit is driven prior to the driving operation by the automatic driving operation unit, thereby to determine the content of the driving operation. An automatic driving control device comprising: a driving operation notifying unit (107) that notifies the driver. It is mounted on an automatic driving vehicle (1) that can be switched between a manual driving state in which a driving operation is performed by a driver and an automatic driving state in which the driving operation is performed without depending on the driver, and controls the automatic driving vehicle. An automatic operation control device (100) that
An automatic driving operation unit (101) for performing the driving operation on the automatic driving vehicle in the automatic driving state;
A driving monitoring determination unit (106) for determining whether or not the driver is monitoring the driving operation in the automatic driving state;
A driver seat drive unit (8m, 8t, 8c, 8e, 8d, 8g, 8Rd, 8Ld) for moving at least a part of the driver seat;
A back tilt angle detector (8s) for detecting a back tilt angle of the backrest of the driver seat;
If it is determined that the driver is monitoring the driving operation, the driver's seat driving unit is driven with a driving amount corresponding to the rearward tilt angle prior to the driving operation by the automatic driving operation unit. An automatic driving control device comprising: a driving operation notifying unit (107) that notifies the driver of the contents of the driving operation. It is mounted on an automatic driving vehicle (1) that can be switched between a manual driving state in which a driving operation is performed by a driver and an automatic driving state in which the driving operation is performed without depending on the driver, and controls the automatic driving vehicle. An automatic operation control device (100) that
An automatic driving operation unit (101) for performing the driving operation on the automatic driving vehicle in the automatic driving state;
A driving monitoring determination unit (106) for determining whether or not the driver is monitoring the driving operation in the automatic driving state;
A handle moving part (2d) for moving the position of the handle steered by the driver;
When it is determined that the driver is monitoring the driving operation, prior to the driving operation by the automatic driving operation unit, by driving the handle moving unit, the content of the driving operation is An automatic driving control device comprising: a driving operation notifying unit (107) for notifying the driver. The automatic operation control device according to claim 6,
A rear tilt angle detector (8s) for detecting a rear tilt angle of the backrest of the driver seat;
The automatic operation control device, wherein the driving operation notifying unit drives the handle driving unit with a movement amount corresponding to the rearward tilt angle. The automatic operation control device according to claim 5 or 7,
The automatic operation control device, wherein the driving operation notifying unit does not perform the warning when the backward tilt angle is larger than a predetermined threshold angle. Automatic driving applied to an automatic driving vehicle that can be switched between a manual driving state in which a driving operation is performed by a driver and an automatic driving state in which the driving operation is performed without depending on the driver, and controls the automatic driving vehicle A control method,
Determining the content of the driving operation for the autonomous driving vehicle in the automatic driving state (S103);
Acquiring the driver's face image (S201);
Detecting the direction in which the driver is looking based on the face image (S202);
Obtaining a steering direction of the autonomous driving vehicle (S204);
A step of determining that the driver is monitoring the driving operation in the automatic driving state when the driver is looking at a steering direction of the autonomous driving vehicle (S200);
When it is determined that the driver is monitoring the driving operation, prior to the driving operation, the step of notifying the driver of the content of the driving operation (S109);
An automatic operation control method comprising the step of performing the driving operation (S110). Automatic driving applied to an automatic driving vehicle that can be switched between a manual driving state in which a driving operation is performed by a driver and an automatic driving state in which the driving operation is performed without depending on the driver, and controls the automatic driving vehicle A control method,
Determining the content of the driving operation for the autonomous driving vehicle in the automatic driving state (S103);
Acquiring the driver's face image (S201);
Detecting the direction in which the driver is looking based on the face image (S202);
Obtaining a front image obtained by photographing the front in the traveling direction from the autonomous driving vehicle (S207);
Detecting a traffic signal to be displayed on the host vehicle from the front image (S208);
A step of determining that the driver is monitoring the driving operation in the automatic driving state when the driver is looking in the direction of the traffic light (S200);
When it is determined that the driver is monitoring the driving operation, prior to the driving operation, the step of notifying the driver of the content of the driving operation (S109);
An automatic operation control method comprising the step of performing the driving operation (S110). Automatic driving applied to an automatic driving vehicle that can be switched between a manual driving state in which a driving operation is performed by a driver and an automatic driving state in which the driving operation is performed without depending on the driver, and controls the automatic driving vehicle A control method,
Determining the content of the driving operation for the autonomous driving vehicle in the automatic driving state (S103);
Determining whether the driver is monitoring the driving operation in the automatic driving state (S200);
When it is determined that the driver is monitoring the driving operation, a step of notifying the driver of the content of the driving operation by moving a headrest of a driver's seat prior to the driving operation. (S109),
An automatic operation control method comprising the step of performing the driving operation (S110). Automatic driving applied to an automatic driving vehicle that can be switched between a manual driving state in which a driving operation is performed by a driver and an automatic driving state in which the driving operation is performed without depending on the driver, and controls the automatic driving vehicle A control method,
Determining the content of the driving operation for the autonomous driving vehicle in the automatic driving state (S103);
Determining whether the driver is monitoring the driving operation in the automatic driving state (S200);
Detecting the back tilt angle of the backrest of the driver's seat (S106);
When it is determined that the driver is monitoring the driving operation, by moving at least a part of the driver's seat by a moving amount according to the backward tilt angle, prior to the driving operation, A step of notifying the driver of the content of the driving operation (S109);
An automatic operation control method comprising the step of performing the driving operation (S110). Automatic driving applied to an automatic driving vehicle that can be switched between a manual driving state in which a driving operation is performed by a driver and an automatic driving state in which the driving operation is performed without depending on the driver, and controls the automatic driving vehicle A control method,
Determining the content of the driving operation for the autonomous driving vehicle in the automatic driving state (S103);
Determining whether the driver is monitoring the driving operation in the automatic driving state (S200);
If it is determined that the driver is monitoring the driving operation, prior to the driving operation, the driver moves the position of the steering wheel to change the content of the driving operation. A step of notifying (S109),
An automatic operation control method comprising the step of performing the driving operation (S110).

Images