JP5013175B2 - TRAVEL CONTROL DEVICE AND METHOD, PROGRAM, AND RECORDING MEDIUM - Google Patents

Description

  The present invention relates to a travel control device and method, a program, and a recording medium, and in particular, a travel control device and method, a program, and a program that can detect a dangerous situation during driving more accurately and travel safely. And a recording medium.

  Conventionally, various safety devices for automobiles have been proposed and put into practical use. For example, a driver's face image is taken by a camera, the face image is analyzed, and the driver's blink is recognized. It has been proposed to prevent snoozing driving by warning.

  In addition, the driver's line-of-sight direction is detected from the face direction or the position of the eyes or pupils based on the image of the driver, and the driver's current driving intention is recognized based on the line-of-sight direction. When it is detected that a driving operation contrary to the driver's intention is performed based on the driver's intention and the current operation state, it is also proposed to control the vehicle operation according to the driving operation (for example, a patent Reference 1).

Japanese Patent Laying-Open No. 2005-14752

  However, the driver's intention may not be recognized correctly even if the driver's line-of-sight direction is detected from the face direction or the position of the eyes or eyes.

  For example, at a point where it joins the main line on a highway, it is common for the driver to perform a driving operation to start the vehicle forward while temporarily stopping the vehicle and checking other vehicles coming from behind. is there. In such a case, as shown in FIG. 1, the traveling direction of the vehicle and the line-of-sight direction are different, but that does not mean that the driving operation contrary to the driver's intention was performed.

  In addition, when parking in a narrow place, the driver may watch the front while paying attention to the distance between the front obstacle and the front end of the vehicle while reversing the vehicle. In such a case, as shown in FIG. 2, the traveling direction of the automobile and the line-of-sight direction are still different, but that does not mean that the driving operation contrary to the driver's intention was performed.

  Furthermore, for example, as shown in FIG. 3, when one vehicle is approaching from the right at a good-looking intersection with no traffic lights, the driver must confirm that there are no other vehicles, pedestrians or obstacles. However, if the driver goes straight while looking to the right, this may not collide with a car approaching at high speed from the right when looking straight ahead.

  That is, if the driver's intention is recognized based only on the driver's line of sight and intervention by a safety device is performed, safe driving may be hindered.

  Also, when driving at high speeds, even if the driver's line of sight accurately captures the direction of travel or obstacles, take into account the danger of accidentally operating the brakes or accelerator, etc. There is a need.

  The present invention has been made in view of such a situation, and is capable of more accurately detecting a dangerous situation during driving and allowing the vehicle to travel safely.

A travel control device according to the present invention is a travel control device that controls the travel of an automobile, a travel state information acquisition unit that acquires travel state information that represents a traveling direction and a traveling speed of the automobile, and an object that exists around the automobile. Ambient state information acquisition means for acquiring ambient state information representing the direction and distance up to, a driving operation information acquisition means for acquiring driving operation information representing the operation state of each part operated to drive the automobile, and the automobile Driver state information acquisition means for acquiring driver state information representing the state of the driver driving the vehicle, travel state information acquired by the travel state information acquisition means, ambient state information acquired by the ambient state information acquisition means, On the basis of the driving operation information acquired by the driving operation information acquisition means and the driver state information acquired by the driver state information acquisition means, a car is determined in advance. Safety determination means for determining whether or not the vehicle is traveling safely based on the criteria, and the safety determination means, if it is determined that the vehicle is not traveling safely, regardless of the driver's operation, Driving control means for controlling the driving of the automobile so as to avoid danger, and the safety judging means is an object existing in the surroundings when the automobile travels as it is based on the driving state information and the surrounding state information. If it is determined that there is a danger of a collision, and if it is determined that there is a danger of a collision, an operation that avoids the danger of a collision is performed based on the driving operation information. Appropriate state for the driver to drive the car safely based on the driver state information when it is determined that the operation to avoid the risk of collision is not performed. Whether or not If it is determined that the person is not in an appropriate state for driving the vehicle safely, the vehicle is determined not to be traveling safely, and the vehicle is determined based on the driving state information and the ambient state information. If it is determined that there is no risk of collision when traveling as it is, or if it is determined that an operation that avoids the risk of collision is performed based on the driving operation information, without determining the driver's state based, on the driver state information is determined based image sensor for imaging the driver, the sensor information output from the voice sensor for detecting the voice of the driver information representing a viewing direction of the driver, the information the driver indicates whether or not the arousal state, and information indicating the sound volume of the driver, as well as movement or legs of the arm of the driver Sen to detect motion The amount of movement of the driver specified based on information output from the driver is included, and in determining the state of the driver based on the driver state information, the driver's line-of-sight direction is the progress of the vehicle It is determined whether or not the direction matches the direction of the driver, and if it is determined that the driver's line-of-sight direction matches the traveling direction of the vehicle, it is determined whether or not the driver is awake and the driver is awakened. If it is determined that the driver is in a panic state, it is determined whether or not the driver is in a panic state based on information indicating the driver's movement amount and the driver's voice amount. When it is determined that the vehicle is not in a state, the travel control device determines that the driver is in an appropriate state to drive the vehicle safely.

In the traveling control device of the present invention, traveling state information representing the traveling direction and traveling speed of the vehicle is acquired, ambient state information representing the direction and distance to an object existing around the vehicle is acquired, and the vehicle is driven. Driving operation information representing the operation state of each part operated in order to obtain, driver state information representing the state of the driver driving the vehicle is obtained, the obtained traveling state information, the obtained ambient state information Based on the acquired driving operation information and the acquired driver state information, it is determined whether or not the vehicle is traveling safely based on a predetermined criterion, and the vehicle is not traveling safely If it is determined, the driving of the automobile is controlled so as to avoid danger regardless of the operation of the driver. Also, based on the driving state information and the surrounding state information, it is determined whether or not there is a risk of a collision with an object present in the surroundings when the vehicle is traveling as it is. When it is determined that there is a risk of collision, it is determined whether or not an operation that avoids the risk of collision is performed based on the driving operation information, and an operation that avoids the risk of collision is not performed Is determined based on the driver state information whether the driver is in an appropriate state for safely driving the vehicle, and the driver is in an appropriate state for safely driving the vehicle. If it is determined that the vehicle is not traveling safely, and if the vehicle travels as it is based on the driving state information and the surrounding state information, it is determined that there is no risk of collision. Or Based on the rolling operation information, when the operation to avoid the risk of collision is determined to have been performed, not performed the determination of the driver of the state based on the driver state information. The driver status information includes the driver's line-of-sight direction specified based on sensor information output from an image sensor that captures the driver and a voice sensor that detects the driver's voice. Information indicating whether the driver is in an awake state, information indicating the driver's voice volume, and a sensor that detects movement of the arm or leg of the driver. The amount of movement of the driver specified based on the information is included, and in determining the state of the driver based on the driver state information, it is determined whether the driver's line-of-sight direction matches the traveling direction of the vehicle. When it is determined that the driver's line-of-sight direction coincides with the traveling direction of the vehicle, it is determined whether the driver is awake, and when it is determined that the driver is awake, Quantity, and driver's sound Based on the information representing the quantity, it is determined whether or not the driver is in a panic state, and if it is determined that the driver is not in a panic state, the driver is in an appropriate state to drive the car safely. It is determined that there is.

  Therefore, the dangerous situation can be detected more accurately and the danger can be avoided.

  Accordingly, it is possible to prevent a dangerous situation from being erroneously detected when driving safely.

  Therefore, it is more accurately determined whether or not safe driving is being performed.

A travel control method according to the present invention is a travel control method of a travel control device that controls travel of an automobile, acquires travel state information representing the travel direction and travel speed of the car, and travels to an object existing around the car. Ambient state information representing the direction and distance is obtained, driving operation information representing the operation state of each part operated to drive the vehicle is obtained, and driver state information representing the state of the driver driving the vehicle is obtained. The vehicle travels safely based on predetermined criteria based on the acquired driving state information, the acquired ambient state information, the acquired driving operation information, and the acquired driver state information. Including the step of controlling the driving of the vehicle so as to avoid danger regardless of the operation of the driver when it is determined that the vehicle is not traveling safely. Based on the surrounding state information, when the vehicle travels as it is, it is determined whether there is a risk of collision with an object present in the surroundings, and when the vehicle travels as it is, there is a risk of collision When it is determined, based on the driving operation information, it is determined whether or not an operation for avoiding the risk of collision is performed, and when it is determined that an operation for avoiding the risk of collision is not performed, Based on the driver status information, it is determined whether or not the driver is in an appropriate state for safely driving the vehicle, and the driver is determined not to be in an appropriate state for driving the vehicle safely. If it is determined that the vehicle is not traveling safely, and it is determined that there is no risk of a collision, if the vehicle travels as it is based on the traveling state information and the ambient state information, or Based on driving operation information When it is determined that an operation for avoiding the danger of a collision is being performed, it is determined whether the driver is in an appropriate state for safely driving the vehicle based on the driver state information. The driver's line-of-sight direction specified based on sensor information output from the image sensor that images the driver and the voice sensor that detects the driver's voice is not included in the driver state information Information indicating whether or not the driver is awake, information indicating the driver's voice volume, and a sensor that detects the driver's arm movement or leg movement. that information includes the operation amount of the driver that has been specified based on, in the determination of the driver of the state based on driver status information, whether the line-of-sight direction of the driver coincides with the traveling direction of the vehicle Judgment and direction of driver's line of sight Is determined to match the traveling direction of the car, it is determined whether or not the driver is awake. If it is determined that the driver is awake, the amount of movement of the driver and the voice of the driver are determined. Based on the information representing the quantity, it is determined whether or not the driver is in a panic state, and if it is determined that the driver is not in a panic state, the driver is in an appropriate state to drive the car safely. This is a travel control method determined to be present.

A program according to the present invention is a program that causes a travel control device that controls travel of an automobile to execute travel control processing, controls the acquisition of travel state information that represents the travel direction and travel speed of the automobile, and exists around the automobile. Control of acquisition of ambient state information indicating the direction and distance to the object to be controlled, control of acquisition of driving operation information indicating the operation state of each part operated to drive the vehicle, and control of the driver driving the vehicle The vehicle is controlled in advance based on the acquired driving state information, the acquired ambient state information, the acquired driving operation information, and the acquired driver state information. To control whether or not the vehicle is traveling safely based on the specified criteria, and to avoid danger when it is determined that the vehicle is not traveling safely, regardless of the driver's operation Whether the operation of avoiding the risk of collision is performed based on the driving operation information when it is determined that there is a risk of collision when the vehicle travels as it is, including the step of controlling driving of the moving vehicle If it is determined that the operation to avoid the danger of a collision is not performed, whether or not the driver is in an appropriate state for safely driving the vehicle based on the driver state information Is determined, and it is determined that the driver is not in an appropriate state for safely driving the vehicle, it is determined that the vehicle is not traveling safely, and the driving state information and the surrounding state information are On the basis of when the car travels as it is, when it is determined that there is no risk of collision, or when it is determined that an operation to avoid the risk of collision is performed based on the driving operation information, Driver status information Not performed the determination of the driver of the state of brute, the driver state information is determined based image sensor for imaging the driver, the sensor information output from the voice sensor for detecting the voice of the driver information representing a viewing direction of the driver, the information the driver indicates whether or not the arousal state, and information indicating the sound volume of the driver, as well as movement or legs of the arm of the driver The amount of movement of the driver specified based on information output from a sensor that detects movement is included, and in determining the state of the driver based on the driver state information, the driver's line-of-sight direction is the progress of the vehicle. It is determined whether or not the direction matches the direction, and if it is determined that the driver's line-of-sight direction matches the traveling direction of the car, it is determined whether or not the driver is awake and the driver is awake If judged, drive It is determined whether or not the driver is in a panic state based on the information indicating the driver's movement amount and the driver's voice amount. If the driver is determined not to be in a panic state, the driver It is a computer readable program that is determined to be in an appropriate state for safe driving.

A recording medium according to the present invention is a recording medium in which a program for causing a traveling control device that controls traveling of a vehicle to execute a traveling control process is recorded, and obtains traveling state information indicating a traveling direction and a traveling speed of the vehicle. To control the acquisition of ambient state information that represents the direction and distance to the object that exists around the vehicle, and to control the acquisition of driving operation information that represents the operational state of each part operated to drive the vehicle , Controlling the acquisition of the driver state information representing the state of the driver driving the car, the acquired traveling state information, the acquired ambient state information, the acquired driving operation information, and the acquired driver state information Based on the control of whether or not the vehicle is traveling safely based on predetermined criteria, and if it is determined that the vehicle is not traveling safely, regardless of the driver's operation An operation that includes the step of controlling the driving of the vehicle so as to avoid danger, and when the vehicle is running as it is, when it is determined that there is a risk of collision, an operation that avoids the risk of collision based on the driving operation information If it is determined whether or not an operation to avoid the danger of a collision is performed, it is appropriate for the driver to drive the car safely based on the driver state information. It is determined whether or not the vehicle is in an appropriate state for safely driving the vehicle, it is determined that the vehicle is not traveling safely, and the driving state information and When the vehicle travels as it is based on the surrounding state information, when it is determined that there is no risk of collision, or on the basis of the driving operation information, an operation for avoiding the risk of collision is performed. Determined In this case, the determination of the driver state based on the driver state information is not performed, and the driver state information is output from an image sensor that images the driver and a voice sensor that detects the driver's voice. Information indicating the driver's line-of-sight direction identified based on the sensor information, information indicating whether or not the driver is awake, information indicating the driver's voice volume, and the driver The amount of movement of the driver specified based on the information output from the sensor that detects the movement of the arm or the movement of the leg is included, and in determining the state of the driver based on the driver state information, the driver If it is determined whether the driver's line-of-sight coincides with the direction of travel of the vehicle, and if it is determined that the driver's line-of-sight coincides with the direction of travel of the vehicle, it is determined whether the driver is awake and Person is awake Is determined based on information representing the amount of movement of the driver and the amount of voice of the driver, it is determined whether or not the driver is in a panic state, and the driver is determined not to be in a panic state. In this case, the recording medium stores a computer-readable program that is determined to be in an appropriate state for the driver to drive the car safely.

In the traveling control method and program of the present invention, traveling state information representing the traveling direction and traveling speed of the automobile is acquired, and ambient state information representing the direction and distance to an object existing around the automobile is acquired. Driving operation information representing the operation state of each part operated to drive the vehicle is acquired, driver state information indicating the state of the driver driving the vehicle is acquired, acquired driving state information, acquired Based on the surrounding state information, the acquired driving operation information, and the acquired driver state information, it is determined whether or not the vehicle is traveling safely based on predetermined criteria, and the vehicle is traveling safely. If it is determined that the vehicle has not been operated, the driving of the automobile is controlled so as to avoid danger regardless of the operation of the driver. Also, based on the driving state information and the surrounding state information, it is determined whether or not there is a risk of a collision with an object present in the surroundings when the vehicle is traveling as it is. When it is determined that there is a risk of collision, it is determined whether or not an operation that avoids the risk of collision is performed based on the driving operation information, and an operation that avoids the risk of collision is not performed Is determined based on the driver state information whether the driver is in an appropriate state for safely driving the vehicle, and the driver is in an appropriate state for safely driving the vehicle. If it is determined that the vehicle is not traveling safely, and if the vehicle travels as it is based on the driving state information and the surrounding state information, it is determined that there is no risk of collision. Or Based on the rolling operation information, when the operation to avoid the risk of collision is determined to have been performed, not performed the determination of the driver of the state based on the driver state information. The driver status information includes the driver's line-of-sight direction specified based on sensor information output from an image sensor that captures the driver and a voice sensor that detects the driver's voice. Information indicating whether the driver is in an awake state, information indicating the driver's voice volume, and a sensor that detects movement of the arm or leg of the driver. The amount of movement of the driver specified based on the information is included, and in determining the state of the driver based on the driver state information, it is determined whether the driver's line-of-sight direction matches the traveling direction of the vehicle. When it is determined that the driver's line-of-sight direction coincides with the traveling direction of the vehicle, it is determined whether the driver is awake, and when it is determined that the driver is awake, Quantity, and driver's sound Based on the information representing the quantity, it is determined whether or not the driver is in a panic state, and if it is determined that the driver is not in a panic state, the driver is in an appropriate state to drive the car safely. It is determined that there is.

  According to the present invention, it is possible to detect a dangerous situation during driving more accurately and to travel safely.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 4 is a block diagram showing a configuration example according to an embodiment of the travel control device of the present invention. The traveling control device 100 shown in the figure is attached to a vehicle, for example, detects information representing the traveling state, ambient state, driving operation, driver's state, etc. of the vehicle, and detects the vehicle based on the detected information. It is configured to run safely.

  The driver state sensor 121 includes, for example, an image sensor that captures an image of the driver's face and body, an audio sensor that detects the driver's voice, and the like. The sensor information output from the driver state sensor 121 is analyzed by the driver state analysis unit 124, and driver state information described later is generated.

  The driving operation unit 122 receives, for example, operations such as a steering hole, an accelerator pedal, a brake pedal, and a shift lever by the driver, and supplies information related to the received driving operation to the driving operation analysis unit 125. The driving operation analysis unit 125 generates driving operation information described later based on the information supplied from the driving operation unit 122.

  The travel sensor 123 is configured by, for example, a radar or an image sensor, acquires information related to objects around the automobile, and generates ambient state information described later.

  For example, the danger determination unit 126 is based on the information on the driving state of the automobile supplied from the driving state detection unit 127 and the ambient state information supplied from the driving sensor 123. Judgment of sex is performed and information on the determination result is output.

  The safety determination unit 128 is based on the driver state information supplied from the driver state analysis unit 124, the driving operation information supplied from the driving operation analysis unit 125, and the information supplied from the danger determination unit 126. If it is determined that it is not possible to safely drive the vehicle, the driving restriction unit 129 is controlled to forcibly drive to avoid danger. As a result, a signal for controlling the operation control unit 130 is output.

  The driving control unit 130 controls the driving operation unit 122 based on the signal supplied from the travel restriction unit 129, stops the reception of the operation by the driver as necessary, and forcibly drives to avoid danger. Let me do it.

  FIG. 5 is a block diagram illustrating a detailed configuration example of the driver state sensor 121. In the figure, the line-of-sight direction detection unit 201 includes, for example, an image sensor that uses a CCD image sensor or the like as an image sensor, a computer that analyzes an image captured by the image sensor, and the like (driver's face (head)). The direction of the driver's line of sight is detected based on the direction of the vehicle and the position of the pupil.

  The face image detection unit 202 includes, for example, a computer that analyzes an image captured by the above-described image sensor, and detects facial expressions, blinks, and the like. Note that the gaze direction detection unit 201 and the face image detection unit 202 may be configured as the same functional block.

  The arm operation detection unit 203 includes, for example, a computer that analyzes an image of the upper body of the driver imaged by a CCD image sensor or the like. Detects driver's operations and movements such as operation or shaking hands.

  The pedal operation detection unit 204 includes, for example, proximity sensors attached to various pedals, detects movements of the driver's legs and toes, and sensors attached to the pedals such as an accelerator pedal and a brake pedal. The operation amount and operation force are detected.

  The voice detection unit 205 is constituted by a microphone or the like, detects a voice, and detects a volume, a feature amount of a voice signal, and the like.

  FIG. 6 is a block diagram illustrating a detailed configuration example of the driving operation unit 122. The steering operation unit 221 to the brake operation unit 224 shown in the figure are connected to a steering wheel, a transmission device (shift lever, etc.), an accelerator pedal, and a brake pedal of an automobile, respectively, and the steering wheel, transmission device (shift lever, etc.), respectively. ), Detecting the driver's operation performed on the accelerator pedal and the brake pedal.

  For example, the steering operation unit 221 detects the rotation direction and rotation angle of the steering wheel, and the shift operation unit 222 selects any of the shift lever selection positions such as D (drive), R (reverse), and N (neutral). It is detected whether it is set to the position. In addition, the accelerator operation unit 223 and the brake operation unit 224 detect the amount by which the accelerator pedal and the brake pedal are depressed (for example, the amount of movement with respect to the reference position), respectively.

  FIG. 7 is a block diagram illustrating a detailed configuration example of the travel sensor 123. The travel sensor 123 includes a front radar 241, a rear radar 242, a front image monitoring unit 243, a surrounding image monitoring unit 244, a night sensor 245, and an analysis unit 246.

  The front radar 241 and the rear radar 242 are laser radars, radio wave radars, and the like, and detect the direction and distance of objects constituting the road such as front and rear cars, pedestrians, obstacles and guardrails. Or you may make it comprise using the object sensor etc. which detect the presence or absence of the object of a predetermined position by electromagnetic waves, visible light, infrared rays, a sound wave, an electrostatic capacitance, etc. The forward image monitoring unit 243 detects a road shape, a line, a car, a pedestrian, an obstacle, a road sign, and the like using an image obtained by photographing the front.

  The surrounding image monitoring unit 244 uses the image captured by an image sensor or the like that captures an image of the surroundings of the vehicle, the surroundings of the vehicle, particularly the side and rear road conditions, the position of other vehicles, pedestrians, and obstacles. Detect size and size.

  The night sensor 245 is composed of a high-sensitivity image sensor, an image sensor that captures images with far infrared rays generated by a human body, and detects pedestrians and obstacles that are difficult to see due to being far away or dark in the traveling direction of the car at night.

  The analysis unit 246 analyzes information output from the front radar 241 to the night sensor 245, and generates information on objects existing around the automobile.

  5 to 7 described above are examples of a driver state sensor, a driving operation unit, and a travel sensor. For example, other functions may be added according to the price and usage of the device, or Each of the configurations in FIGS. 5 to 7 may be changed as appropriate so that the functions of the units are deleted.

  Next, the details of the travel control processing by the travel control device 100 will be described with reference to the flowchart of FIG.

  In step S101, the traveling state detection unit 127 acquires information regarding the traveling state of the automobile. At this time, for example, as shown in FIG. 9, the traveling direction (traveling direction) of the automobile and the traveling speed are acquired and recorded as information representing them. In the example of FIG. 9, it is recorded that the traveling direction of the automobile is “front (front)” and the traveling speed is 40 km / h.

  After the process of step S101, in step S102, the traveling sensor 123 executes an ambient state information acquisition process which will be described later with reference to FIG. Thereby, ambient state information which is information related to objects around the automobile is generated.

  Here, the details of the surrounding state information acquisition processing in step S102 of FIG. 8 will be described with reference to the flowchart of FIG.

  In step S201, the analysis unit 246 analyzes information output from each unit (sensor) of the front radar 241 to the nighttime sensor 245. In step S202, the analysis unit 246 detects the distance and direction to an object that is an obstacle to the progress of the automobile based on the analysis result in the process of step S201.

  At this time, for example, as shown in FIG. 11, information representing the direction of the object and the distance to the object is generated. In the example of FIG. 11, an obstacle object exists at a position 50 m ahead of the automobile, an obstacle object exists at the rear 50 m of the automobile, an obstacle object exists on the right side 15 m of the automobile, and the left side of the automobile. It is shown that there is an object that becomes an obstacle at 20 m. The information shown in FIG. 11 is supplied to the danger determination unit 126 as ambient state information.

  Returning to FIG. 8, after the process of step S <b> 102, in step S <b> 103, the driving operation analysis unit 125 executes a driving operation information acquisition process described later with reference to FIG. 12. Thus, driving operation information that is information relating to the current driving operation of the automobile is generated.

  Here, the details of the driver operation information acquisition process in step S103 in FIG. 8 will be described with reference to the flowchart in FIG.

  In step S221, the driving operation analysis unit 125 acquires shift lever operation information. At this time, for example, information indicating the current selection position of the shift lever is acquired.

  In step S222, the driving operation analysis unit 125 acquires pedal operation information. At this time, for example, information indicating the amount of depression of the accelerator pedal and the brake pedal at the present time is acquired.

  In step S223, the driving operation analysis unit 125 acquires steering wheel operation information. At this time, for example, information representing the rotation direction and rotation angle of the steering wheel is acquired.

  And the information acquired by the process of step S221 thru | or S223 is produced | generated as driving operation information as shown, for example in FIG. In the example of FIG. 11, the driving operation information is configured by each operation part and the operation situation corresponding to the operation part, the selection position of the shift lever is “D (drive)”, and the operation situation of the accelerator pedal is , “ON (represents a depressed state) −3 (represents a depressed amount)”, and the operation state of the brake pedal is “OFF (represents an undepressed state)”, and steering The investigation status of the wheel is “left (represents the direction of rotation) 5 ° (represents the rotation angle)”.

  The driving operation information shown in FIG. 13 is supplied to the safety determination unit 128.

  Returning to FIG. 8, after step S <b> 103, in step S <b> 104, the driver state analysis unit 124 executes driving state information acquisition processing described later with reference to FIG. 14. Thereby, driver state information which is information about the current state of the driver is generated.

  Here, with reference to the flowchart of FIG. 14, the detail of the driver state information acquisition process of step S104 of FIG. 8 is demonstrated.

  In step S241, the driver state analysis unit 124 detects the direction of the driver's line of sight based on the information output from the line-of-sight direction detection unit 201.

  In step S242, the driver state analyzing unit 124 detects the state of the driver's face based on the information output from the face image detecting unit 202. At this time, for example, it is detected whether the driver's face is awake or is not awake, such as being asleep.

  In step S243, the driver state analysis unit 124 detects the driver's action based on information output from the arm operation detection unit 203 and the pedal operation detection unit 204. At this time, for example, how the driver's arm moves or how the pedal is depressed is normalized as a value when compared with a preset reference value, and is recognized as a numerical value. "," Small ", etc. are detected as information representing a predetermined level.

  In step S <b> 244, the driver state analysis unit 124 detects the voice based on the information output from the voice detection unit 205. At this time, for example, a voice that is estimated to be emitted by the driver based on a feature amount of the voice signal is extracted, and the level (amplitude) of the voice is compared with a preset reference value. It is normalized and recognized as a numerical value, and the driver's voice is detected as information representing a predetermined level such as “high”, “medium”, and “small”.

  And the information acquired by the process of step S241 thru | or S244 is produced | generated as driver state information as shown, for example in FIG. In the example of FIG. 15, the driver state information includes detection items and detection results corresponding to the detection items, the line-of-sight direction is “front”, the face state is “wakefulness”, and the action is “ “Small” and the sound is “Small”.

  The driver state information shown in FIG. 15 is supplied to the safety determination unit 128.

  Returning to FIG. 8, after the process of step S <b> 104, the process proceeds to step S <b> 105, and a traveling safety determination process is executed.

  Here, the details of the traveling safety determination process in step S105 of FIG. 8 will be described with reference to the flowchart of FIG.

  In step S261, the danger determination unit 126 detects a collision risk based on the traveling state information acquired in step S101 and the ambient state information acquired in step S102. Execute the process.

  Here, the details of the collision risk check process in step S261 of FIG. 16 will be described with reference to the flowchart of FIG.

  In step S301, the danger determination unit 126 determines the traveling direction of the vehicle based on the traveling direction information included in the traveling state information acquired in the process of step S101.

  If it is determined in step S301 that the traveling direction is forward, the process proceeds to step S302, and the risk determining unit 126 adds the traveling speed information included in the traveling state information acquired in the process of step S101. Based on this, the speed of the automobile is detected, and a threshold value corresponding to a preset speed is acquired. The threshold is a distance determined to be a risk of collision when traveling (running) at that speed, and is a distance between the object and the automobile (for example, an inter-vehicle distance determined according to the speed). Here, it is assumed that l (m) is acquired as the threshold value.

  In step S303, the danger determination unit 126 determines whether there is an obstacle within l (m) ahead of the automobile. At this time, the distance to the object ahead is acquired based on the ambient state information acquired in the process of step S102, and it is determined whether or not the distance exceeds lm.

  If it is determined in step S303 that there is an obstacle within l (m) ahead of the vehicle, there is a danger of a collision. Therefore, the process proceeds to step S306, and a collision danger flag indicating that there is a danger of a collision. Is set to ON. On the other hand, if it is determined in step S303 that there is no obstacle within l (m) ahead of the vehicle, there is no danger of collision, so the collision risk check process ends with the collision risk flag set to OFF.

  If it is determined in step S301 that the traveling direction is backward, the process proceeds to steps S304 and S305, and the risk of collision with a rear object is checked in the same manner as in steps S302 and S303.

  In steps S303 and S305, a threshold value may be set according to the time until the collision. For example, it may be determined whether or not the distance is 1.5 seconds or less until the collision.

  Here, although the traveling direction determined in the process of step S301 is set to the front or the rear, for example, the information indicating the traveling direction in the information on the traveling state includes information such as “right front” and “left rear”. In this way, the traveling direction determined in the process of step S301 may be determined in more detail.

  In this way, the risk of collision is checked.

  Returning to FIG. 16, after the process of step S <b> 261, in step S <b> 262, the safety determination unit 128 determines whether there is a risk of collision. The determination in the process of step S262 is performed based on the above-described collision danger flag.

  If it is determined in step S262 that there is a risk of a collision, the process proceeds to step S263, and the safety determination unit 128 passes the driving operation information acquired in the process of step S103 and the process of step S261. Based on the information supplied from the risk determination unit 126, an avoidance operation check process is executed.

  Here, the details of the avoidance operation check process in step S263 of FIG. 16 will be described with reference to the flowchart of FIG.

  In step S321, the safety determination unit 128 determines whether or not the steering wheel is rotated in a direction away from the object. For example, if it is determined in step S261 that there is a risk of a collision with an object ahead, the object is determined based on information indicating the steering wheel operation status included in the driving operation information acquired in step S103. It is determined whether the steering wheel is rotated to the right or left at an angle sufficient to avoid.

  If it is determined in step S321 that the steering wheel is not rotated in the direction away from the object, the process proceeds to step S322, and the safety determination unit 128 is included in the driving operation information acquired in the process of step S103. It is determined whether or not the brake pedal has been depressed by an amount sufficient to stop the vehicle based on the information indicating the brake operation status.

  If it is determined in step S321 that the steering wheel is rotating in a direction away from the object, or it is determined in step S322 that the brake pedal is depressed by an amount sufficient to stop the vehicle. In this case, since the operation for avoiding the collision is performed, the process proceeds to step S323, and the avoidance operation flag indicating that the operation for avoiding the collision is performed is set to ON.

  On the other hand, if it is determined in step S322 that the brake pedal is not depressed by an amount sufficient to stop the vehicle, an operation for avoiding the collision is not performed, and the avoidance operation flag remains OFF. The operation check process ends.

  Note that the determination of whether or not the avoidance operation is performed is not limited to the above-described example, and it is also possible to determine based on more detailed criteria using information included in the driving operation information. is there.

  Returning to FIG. 16, after the process of step S <b> 263, the process proceeds to step S <b> 264, and it is determined whether or not a driving operation for avoiding a collision is performed based on the avoidance operation flag described above.

  If it is determined in step S264 that the driving operation for avoiding the collision is not performed, the process proceeds to step S265, and the safety determination unit 128 includes the information on the traveling state acquired in the process of step S101, and the step A driver state check process is executed based on the driver state information acquired in the process of S104.

  Here, the details of the driver state check process in step S264 in FIG. 16 will be described with reference to the flowchart in FIG.

  In step S341, the safety determination unit 128 determines whether or not the current speed is equal to or less than a preset threshold value based on the information on the traveling speed included in the traveling state information acquired in the process of step S101. . This threshold value is the upper limit of the speed at which the automobile can be operated sufficiently safely and sufficiently reliably, and here the speed a (km) is set as the threshold value.

  If it is determined in step S341 that the traveling speed is not equal to or higher than a (km) per hour, the process proceeds to step S342, and the safety determination unit 128 is included in the driver state information acquired in the process of step S104. It is determined whether or not the line-of-sight direction matches the traveling direction based on the information on the line-of-sight direction.

  When it is determined in step S341 that the traveling speed exceeds the speed a (Km), the process in step S342 is skipped, and the process proceeds to step S343.

  If it is determined in step S342 that the line-of-sight direction does not match the traveling direction, the driver state check process is terminated. That is, there is a possibility that the operation cannot be performed safely and reliably in a state where it is determined that there is a risk of collision in the process of step S262 and that an operation for avoiding the collision is not performed in the process of step S264. If the driver's line-of-sight direction of a car traveling at speed does not match the direction of travel, the driver must immediately stop accepting operations and force driving to avoid danger Therefore, there is no need to continue the driver status check process. In this case, it is determined that the current operation is not appropriate.

  If it is determined in step S342 that the line-of-sight direction matches the traveling direction, the process proceeds to step S343, and the safety determination unit 128 is included in the driver state information acquired in the process of step S104. Based on the face state information, it is determined whether the driver is awake.

  If it is determined in step S343 that the driver is not awake, the driver state check process is terminated. That is, in the state in which it is determined in step S262 that there is a risk of collision and in step S264 it is determined that the operation for avoiding the collision has not been performed, the speed of the vehicle is sufficiently slow and progresses. If it is determined that the driver is not awake, even if the driver is gazing at the direction, the driver will immediately stop accepting operations and force the driver to avoid danger. There is no need to continue the driver status check process any further. In this case, it is determined that the current operation is not appropriate.

  If it is determined in step S343 that the driver is awake, the process proceeds to step S344, and the safety determination unit 128 includes information on the operation included in the driver state information acquired in the process of step S104. Based on the voice information or the like, it is determined whether or not the driver is in a panic state. Whether or not the camera is in the panic state is determined based on a preset condition. For example, when the operation is sufficiently large and a loud sound is detected, the panic state is determined.

  If it is determined in step S344 that the driver is in a panic state, the driver state check process is terminated. In other words, in a state where it is determined that there is a risk of a collision and it is determined that an operation for avoiding the collision has not been performed, the speed of the car is sufficiently slow and the driver is awakened by watching the direction of travel. However, if it is determined that the vehicle is in a panic state, it is necessary to immediately stop the acceptance of the operation by the driver and force the operation to avoid danger. It is not determined to be.

  On the other hand, if it is determined in step S344 that the driver is not in a panic state, the process proceeds to step S345, where an appropriate flag indicating that an appropriate operation is being performed is set to ON. In other words, even if it is determined that there is a risk of collision and it is determined that an operation to avoid the collision has not been performed, the speed of the car is sufficiently slow and the direction of travel is closely watched, and further panic If it can be confirmed that the vehicle is not in a state, it is not necessary to force the driver to accept the operation and to force the driver to avoid danger. Therefore, the appropriate flag is set to ON.

  In this way, the state of the driver is checked to determine whether or not proper driving is being performed.

  Returning to FIG. 16, after the process of step S <b> 265, the process proceeds to step S <b> 266, and the safety determination unit 128 determines whether or not the driver's state is appropriate, that is, whether the appropriate driving is being performed. Determine whether or not. The determination as to whether or not the driver is in an appropriate state is made based on the above-described appropriate flag.

  If it is determined in step S266 that the driver's state is appropriate, the process proceeds to step S267, and a safety flag indicating that the driving state of the vehicle is safe is set to ON.

  In this way, traveling safety is determined.

  Returning to FIG. 8, after the process of step S <b> 105, the process proceeds to step S <b> 106, and the safety determination unit 128 determines whether the vehicle is currently traveling safely. Whether or not the vehicle is traveling safely is determined based on the safety flag described above.

  If it is determined in step S106 that the vehicle is not traveling safely, the process proceeds to step S107, and automatic control is performed so as to avoid danger. Thereby, the driving | running | working restrictions part 129 outputs the signal which controls the driving | operation control part 130 so that driving | running | working which avoids danger may be forced. For example, an operation such as applying a brake is performed regardless of the operation of the driver.

  On the other hand, if it is determined in step S106 that the automobile is traveling safely, the process returns to step S101, and the subsequent processes are repeatedly executed.

  In this way, traveling control of the automobile is performed. In the present invention, since the driving safety is determined based on the information on the driving state, the surrounding state, the driving operation, and the driver state of the automobile, a dangerous situation during driving can be detected more accurately.

  As a technique for driving a car safely, for example, a driver's intention is recognized by detecting the driver's gaze direction from the face direction or the position of eyes or pupils, and driving different from the driver's intention is performed. In such a case, there is a technique for performing control so that a driving operation that avoids danger by intervening a safety device is forcibly (automatically) performed.

  However, for example, at a point where it joins the main line on a highway, it is common for a driver to perform a driving operation to start the vehicle forward while temporarily stopping the vehicle and checking other vehicles coming from behind. In such a case, the traveling direction of the vehicle and the line-of-sight direction are different, but that does not mean that a driving operation contrary to the driver's intention was performed.

  Also, when parking in a confined place, the driver may look back ahead while taking the distance between the front obstacle and the front end of the car while reversing the car. In this case, the traveling direction of the car and the line-of-sight direction are different, but that does not mean that the driving operation contrary to the driver's intention was performed.

  In addition, when a car is approaching from the right at a good-looking intersection with no traffic lights, if the driver confirms that there are no other cars, pedestrians or obstacles, the driver must turn right. Although it goes straight while looking at it, this is not a side look, but conversely, when looking straight ahead, there is a possibility of colliding with a car approaching at high speed from the right.

  That is, if the driver's intention is recognized based only on the driver's line of sight and intervention by a safety device is performed, safe driving may be hindered.

  On the other hand, in the present invention, the driver's intention is recognized in more detail in the driver state check process described above with reference to FIG. For example, in the present invention, even when it is determined that there is a risk of a collision and it is determined that an operation for avoiding the collision is not performed, the speed of the automobile is sufficiently slow and the direction of travel is closely watched. If it is confirmed that the vehicle is not in a panic state, it is not necessary to stop the driver from accepting the operation and force the vehicle to avoid danger. It is judged appropriate according to the intention of the person.

  Further, as described above with reference to FIG. 16, the driver state check process of FIG. 19 is determined to have a risk of collision in step S262, and the driving operation for avoiding the collision is not performed in step S264. It is executed when it is determined.

  The information obtained from the driver's voice, driver's facial expression, etc., including the driver's line of sight mentioned above, does not uniformly indicate whether the driver's state is appropriate, but changes according to the situation. I think that. For example, when there is no danger of a collision, it is unlikely that a driver's loud voice or sleepy expression will cause an accident. Therefore, even though there is no risk of collision, driving that avoids danger by intervening safety devices based on information obtained from the driver's line of sight, driver's voice, driver's facial expression, etc. If forced to do so, it may interfere with safe driving.

  On the other hand, when there is a danger of a collision and a driving operation for avoiding the collision is not performed, whether or not the driver is in an appropriate state is an important factor for avoiding the danger.

  In the present invention, the driver state check process described above is performed when it is determined that there is a risk of a collision and the driving operation for avoiding the collision is not performed. On the basis of this, intervention by the safety device is performed, and on the other hand, it is possible to prevent the safe operation from being disturbed while the safety device is prevented from being hindered.

  Thus, in the present invention, it is more accurately determined whether or not the traveling is different from the driver's intention, and as a result, the traveling of the automobile can be controlled more safely.

  Also, for example, when driving at high speeds, even if the driver's line of sight accurately captures the direction of travel or obstacles, there is a risk that the brake or accelerator will be operated incorrectly, causing spin or rollover. Need to be taken into account, and it is not enough to intervene with safety devices based on the driver's line of sight.

  On the other hand, in the present invention, in the driver state check process described above with reference to FIG. 19, it is determined whether the driver is awake or panic. If the driver is not awake or is in a panic, it is possible to immediately stop accepting the operation by the driver and force the driving to avoid danger. .

  It does not matter whether the above-described series of processing is realized by hardware or software. When the above-described series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, a general-purpose personal computer 500 as shown in FIG. 20 is installed from a network or a recording medium.

  20, a CPU (Central Processing Unit) 501 executes various processes according to a program stored in a ROM (Read Only Memory) 502 or a program loaded from a storage unit 508 to a RAM (Random Access Memory) 503. To do. The RAM 503 also appropriately stores data necessary for the CPU 501 to execute various processes.

  The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input / output interface 505 is also connected to the bus 504.

  The input / output interface 505 includes an input unit 506 including a keyboard and a mouse, a display including a CRT (Cathode Ray Tube) and an LCD (Liquid Crystal display), an output unit 507 including a speaker, a hard disk, and the like. A communication unit 509 including a storage unit 508, a network interface card such as a modem and a LAN card, and the like is connected. A communication unit 509 performs communication processing via a network including the Internet.

  A drive 510 is connected to the input / output interface 505 as necessary, and a removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately attached, and a computer program read from them is loaded. It is installed in the storage unit 508 as necessary.

  When the above-described series of processing is executed by software, a program constituting the software is installed from a network such as the Internet or a recording medium such as the removable medium 511.

  The recording medium shown in FIG. 20 is a magnetic disk (including a floppy disk (registered trademark)) on which a program is recorded, which is distributed to distribute the program to the user separately from the apparatus main body, Removable media consisting of optical disks (including CD-ROM (compact disk-read only memory), DVD (digital versatile disk)), magneto-optical disks (including MD (mini-disk) (registered trademark)), or semiconductor memory It includes not only those configured by 511 but also those configured by a ROM 502 on which a program is recorded, a hard disk included in the storage unit 508, and the like distributed to the user in a state of being incorporated in the apparatus main body in advance.

  In addition, the steps of executing the series of processes described above in this specification are performed in parallel or individually even if they are not necessarily processed in time series, as well as processes performed in time series in the order described. The processing to be performed is also included.

It is a figure explaining the example of the advancing direction of a motor vehicle, and a driver | operator's gaze direction. It is a figure explaining another example of the advancing direction of a car, and a driver's gaze direction. It is a figure explaining the further another example of the advancing direction of a motor vehicle, and a driver | operator's gaze direction. It is a block diagram which shows the structural example which concerns on one embodiment of the traveling control apparatus of this invention. It is a block diagram which shows the detailed structural example of the driver | operator state sensor of FIG. It is a block diagram which shows the detailed structural example of the driving | operation operation part of FIG. It is a block diagram which shows the detailed structural example of the traveling sensor of FIG. It is a flowchart explaining the example of a traveling control process. It is a figure which shows the example of the information of a driving | running | working state. It is a flowchart explaining the example of ambient condition information acquisition processing. It is a figure which shows the example of surrounding state information. It is a flowchart explaining the example of a driving operation information acquisition process. It is a figure which shows the example of driving operation information. It is a flowchart explaining the example of a driver state information acquisition process. It is a figure which shows the example of driver | operator state information. It is a flowchart explaining the example of a driving | running | working safety determination process. It is a flowchart explaining the example of a collision risk check process. It is a flowchart explaining the example of an avoidance operation check process. It is a flowchart explaining the example of a driver | operator state check process. And FIG. 16 is a block diagram illustrating a configuration example of a personal computer.

Explanation of symbols

100 travel control device,
121 driver state sensor,
122 driving operation part,
123 travel sensor,
124 Driver condition analysis unit,
125 driving operation analysis unit,
126 danger judgment part,
127 running state detection unit,
128 safety judgment part,
129 travel restriction part,
130 operation control unit,
501 CPU,
502 ROM,
508 storage unit,
511 Removable media

Claims (4)

A travel control device for controlling travel of an automobile,
Traveling state information acquisition means for acquiring traveling state information representing the traveling direction and traveling speed of the automobile,
Ambient state information acquisition means for acquiring ambient state information representing the direction and distance to an object existing around the automobile;
Driving operation information acquisition means for acquiring driving operation information representing an operation state of each part operated to drive the automobile;
Driver state information acquisition means for acquiring driver state information representing the state of the driver driving the automobile;
Driving state information acquired by the driving state information acquisition unit, ambient state information acquired by the surrounding state information acquisition unit, driving operation information acquired by the driving operation information acquisition unit, and driver state information acquisition unit Safety judging means for judging whether or not the vehicle is traveling safely based on a predetermined criterion based on the driver state information acquired by
And a driving control means for controlling the driving of the automobile so as to avoid danger when the safety judging means judges that the automobile is not traveling safely, regardless of the operation of the driver. ,
The safety determination means includes
Based on the traveling state information and the ambient state information, when the automobile travels as it is, it is determined whether there is a risk of collision with the object existing in the surroundings,
When it is determined that there is a risk of collision when the vehicle travels as it is, based on the driving operation information, it is determined whether or not an operation to avoid the risk of collision is being performed,
If it is determined that an operation for avoiding the risk of collision is not performed, whether or not the driver is in an appropriate state for safely driving the vehicle based on the driver state information. Judgment,
If it is determined that the driver is not in an appropriate state to drive the vehicle safely, it is determined that the vehicle is not traveling safely; and
Based on the traveling state information and the ambient state information, when the automobile has traveled as it is, when it is determined that there is no risk of collision, or based on the driving operation information, the risk of collision When it is determined that an operation for avoiding the vehicle is performed, the determination of the state of the driver based on the driver state information is not performed.
The driver status information includes information indicating the driver's line-of-sight direction specified based on sensor information output from an image sensor that captures the driver and a voice sensor that detects the driver's voice. , Information indicating whether or not the driver is awake, information indicating the amount of voice of the driver , and information output from a sensor that detects movement of the arm or leg of the driver It includes operation amount of the driver, which is identified based,
In determining the state of the driver based on the driver state information,
It is determined whether or not the driver's line-of-sight direction matches the traveling direction of the automobile,
When it is determined that the driver's line-of-sight direction matches the traveling direction of the vehicle, it is determined whether or not the driver is awake.
When it is determined that the driver is awake, it is determined whether or not the driver is in a panic state based on information indicating the amount of movement of the driver and the amount of voice of the driver.
When it is determined that the driver is not in a panic state, it is determined that the driver is in an appropriate state for driving the automobile safely. A travel control method for a travel control device for controlling travel of an automobile,
Obtaining driving state information representing the traveling direction and traveling speed of the automobile,
Obtaining ambient state information representing the direction and distance to an object existing around the automobile;
Obtain driving operation information representing the operation state of each part operated to drive the automobile,
Obtaining driver status information representing the status of the driver driving the vehicle;
Based on the acquired traveling state information, the acquired ambient state information, the acquired driving operation information, and the acquired driver state information, the vehicle is safely based on a predetermined standard. Determine if you are running,
If it is determined that the vehicle is not traveling safely, regardless of the operation of the driver, including the step of controlling the driving of the vehicle to avoid danger,
Based on the traveling state information and the ambient state information, when the automobile travels as it is, it is determined whether there is a risk of collision with the object existing in the surroundings,
When it is determined that there is a risk of collision when the vehicle travels as it is, it is determined whether or not an operation to avoid the risk of collision is performed based on the driving operation information,
If it is determined that an operation for avoiding the danger of the collision is not performed, whether or not the driver is in an appropriate state for safely driving the vehicle based on the driver state information. Judged,
If it is determined that the driver is not in an appropriate state to drive the vehicle safely, it is determined that the vehicle is not driving safely, and
Based on the traveling state information and the ambient state information, when the automobile has traveled as it is, when it is determined that there is no risk of collision, or based on the driving operation information, the risk of collision When it is determined that an operation to avoid is performed, the determination of the state of the driver based on the driver state information is not performed,
The driver status information includes information indicating the driver's line-of-sight direction specified based on sensor information output from an image sensor that captures the driver and a voice sensor that detects the driver's voice. , Information indicating whether or not the driver is awake, information indicating the amount of voice of the driver , and information output from a sensor that detects movement of the arm or leg of the driver It includes operation amount of the driver, which is identified based,
In determining the state of the driver based on the driver state information,
It is determined whether or not the driver's line-of-sight direction matches the traveling direction of the automobile,
When it is determined that the driver's line-of-sight direction matches the traveling direction of the vehicle, it is determined whether or not the driver is awake.
When it is determined that the driver is awake, it is determined whether or not the driver is in a panic state based on information indicating the amount of movement of the driver and the amount of voice of the driver.
When it is determined that the driver is not in a panic state, it is determined that the driver is in an appropriate state for driving the vehicle safely. A program that causes a traveling control device that controls traveling of an automobile to execute a traveling control process,
Controlling the acquisition of traveling state information representing the traveling direction and traveling speed of the automobile,
Controlling the acquisition of ambient state information representing the direction and distance to objects present around the automobile;
Controlling the acquisition of driving operation information representing the operation state of each part operated to drive the automobile,
Controlling the acquisition of driver status information representing the status of the driver driving the vehicle;
Based on the acquired traveling state information, the acquired ambient state information, the acquired driving operation information, and the acquired driver state information, the vehicle is safely based on a predetermined standard. Control whether or not you are driving,
If it is determined that the vehicle is not traveling safely, regardless of the operation of the driver, including the step of controlling the driving of the vehicle to avoid danger,
When it is determined that there is a risk of collision when the vehicle travels as it is, it is determined whether or not an operation to avoid the risk of collision is performed based on the driving operation information,
If it is determined that an operation for avoiding the danger of the collision is not performed, whether or not the driver is in an appropriate state for safely driving the vehicle based on the driver state information. Judged,
If it is determined that the driver is not in an appropriate state to drive the vehicle safely, it is determined that the vehicle is not driving safely, and
Based on the traveling state information and the ambient state information, when the automobile has traveled as it is, when it is determined that there is no risk of collision, or based on the driving operation information, the risk of collision When it is determined that an operation to avoid is performed, the determination of the state of the driver based on the driver state information is not performed,
The driver status information includes information indicating the driver's line-of-sight direction specified based on sensor information output from an image sensor that captures the driver and a voice sensor that detects the driver's voice. , Information indicating whether or not the driver is awake, information indicating the amount of voice of the driver , and information output from a sensor that detects movement of the arm or leg of the driver It includes operation amount of the driver, which is identified based,
In determining the state of the driver based on the driver state information,
It is determined whether or not the driver's line-of-sight direction matches the traveling direction of the automobile,
When it is determined that the driver's line-of-sight direction matches the traveling direction of the vehicle, it is determined whether or not the driver is awake.
When it is determined that the driver is awake, it is determined whether or not the driver is in a panic state based on information indicating the amount of movement of the driver and the amount of voice of the driver.
A computer readable program that determines that the driver is in an appropriate state to drive the car safely when it is determined that the driver is not in a panic state. A recording medium on which a program for causing a travel control device for controlling travel of an automobile to execute travel control processing is recorded,
Controlling the acquisition of traveling state information representing the traveling direction and traveling speed of the automobile,
Controlling the acquisition of ambient state information representing the direction and distance to objects present around the automobile;
Controlling the acquisition of driving operation information representing the operation state of each part operated to drive the automobile,
Controlling the acquisition of driver status information representing the status of the driver driving the vehicle;
Based on the acquired traveling state information, the acquired ambient state information, the acquired driving operation information, and the acquired driver state information, the vehicle is safely based on a predetermined standard. Control whether or not you are driving,
If it is determined that the vehicle is not traveling safely, regardless of the operation of the driver, including the step of controlling the driving of the vehicle to avoid danger,
When it is determined that there is a risk of collision when the vehicle travels as it is, it is determined whether or not an operation to avoid the risk of collision is performed based on the driving operation information,
If it is determined that an operation for avoiding the danger of the collision is not performed, whether or not the driver is in an appropriate state for safely driving the vehicle based on the driver state information. Judged,
If it is determined that the driver is not in an appropriate state to drive the vehicle safely, it is determined that the vehicle is not driving safely, and
Based on the traveling state information and the ambient state information, when the automobile has traveled as it is, when it is determined that there is no risk of collision, or based on the driving operation information, the risk of collision When it is determined that an operation to avoid is performed, the determination of the state of the driver based on the driver state information is not performed,
The driver status information includes information indicating the driver's line-of-sight direction specified based on sensor information output from an image sensor that captures the driver and a voice sensor that detects the driver's voice. , Information indicating whether or not the driver is awake, information indicating the amount of voice of the driver , and information output from a sensor that detects movement of the arm or leg of the driver It includes operation amount of the driver, which is identified based,
In determining the state of the driver based on the driver state information,
It is determined whether or not the driver's line-of-sight direction matches the traveling direction of the automobile,
When it is determined that the driver's line-of-sight direction matches the traveling direction of the vehicle, it is determined whether or not the driver is awake.
When it is determined that the driver is awake, it is determined whether or not the driver is in a panic state based on information indicating the amount of movement of the driver and the amount of voice of the driver.
When it is determined that the driver is not in a panic state, it is determined that the driver is in an appropriate state for safely driving the vehicle. A recording medium on which a computer-readable program is recorded.

Images