JP2002367100A - Device for detecting driver condition - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a driver condition detecting device which can surely detect a driver condition and is excellent in detection performance. SOLUTION: The sight line direction E of the driver is detected, and a sight line direction frequency calculating part 37 calculates a scrutiny frequency distribution F. A pupil diameter dr of the driver is detected, and a pupil diameter rate of change calculating part 49 calculates the rate drv of change to a reference pupil diameter d0 for a prescribed period of time from the detected pupil diameter dr. A driver condition detected in accordance with the degree of approaching the preceding vehicle is set, and the driver condition is detected on the basis of a scrutiny frequency Ff to the front, its prescribed decision value Ff0, the rate drv of change of a pupil diameter and its prescribed decision value drv0.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a driver state detecting device for detecting a driver state using a driver's gaze behavior and a pupil diameter change rate.

[0002]

2. Description of the Related Art As a device for recognizing a driving action intention using a gaze action of a driver who drives a vehicle, Japanese Patent Application Laid-Open No. 9-25 / 1991 has been proposed.
No. 4742 is disclosed (conventional device 1). This driving action intention detection device detects a decrease in the driver's arousal level by creating a frequency distribution pattern in the line-of-sight direction by dividing a projection plane in front of the vehicle into a plurality of regions, and comparing this with a reference pattern. Things. Also,
In the driving action intention detection device (conventional device 2) disclosed in Japanese Patent Application Laid-Open No. 8-178712, the line-of-sight detection range is divided into a plurality of regions, and the line-of-sight detection frequencies in the divided regions are counted and compared. In this way, it is determined whether or not the driver is driving casually. Further, JP-A-2000-35133
Japanese Patent Application Laid-Open No. 7-107421 discloses a device that detects a driver's state based on a changing speed of a driver's pupil diameter and provides various alarms and a driving support function based on the detection result (conventional device 3).

[0003]

However, in the above-described conventional apparatus, the driver's state is determined from a single detection value such as the line-of-sight behavior and the rate of change of the pupil diameter. There is a problem that it is difficult to separate and detect a plurality of driver states. For example, in the case where the driver is approaching the driver ahead of the driver's prediction when approaching, the driver is considered to be in an excessively nervous state and staring at the driver's vehicle. However, in the determination based on only the movement frequency distribution in the line of sight as in the conventional device 1 and the conventional device 2, there are cases where the degree of approach to the preceding vehicle is determined in a normal and calm mental state,
In the case of an excessively tensed state, the ratio of gazing forward is high in both cases, so that it is not possible to sufficiently determine in what state the driver is actually driving. Conventional device 3
In the judgment based only on the driver's pupil diameter as in above, since the relationship with the object to be watched is not explicit, it is necessary to detect whether the user is watching the object or is in an excessively tense state Is difficult.

[0004] It is an object of the present invention to provide a driver condition detecting device capable of reliably detecting the condition of a driver and having excellent detection performance.

FIG. 1 shows an embodiment of the present invention.
The present invention will be described with reference to FIG. (1) The invention of claim 1 is a gaze direction detecting means 1 for detecting a gaze direction of a driver on a vehicle fixed projection plane, a pupil diameter detecting means 5 for detecting a pupil diameter of a driver, and a pupil diameter detecting means 5 A pupil diameter change rate calculating means 7 for calculating a change rate of the pupil diameter with respect to an absolute value in a predetermined time from the pupil diameter detected in step 2, data relating to the gaze direction detected by the gaze direction detection means 1, and a pupil diameter change rate The above-described object is achieved by providing a driver state determination unit that determines a driver's state based on the pupil diameter change rate calculated by the calculation unit. (2) According to a second aspect of the present invention, in the driver state detecting apparatus according to the first aspect, a gaze direction frequency distribution in a plurality of divided regions is calculated using a plurality of gaze direction data detected during a predetermined time. The apparatus further includes a gaze direction frequency distribution calculation unit 3, and the driver state determination unit 11 determines the driver based on the gaze direction frequency and the pupil diameter change rate in a specific area calculated by the gaze direction frequency distribution calculation unit 3. Is determined. (3) The invention according to claim 3 is the driver condition detecting device according to claim 1, further comprising a preceding vehicle approaching degree detecting means 49 for detecting an approaching degree to the preceding vehicle.
1 to determine the driver state based on the pupil diameter change rate,
The driver's state determination means 11 is also performed by taking into account the degree of approach of the preceding vehicle detected by the degree of preceding vehicle detection means 49.
Is characterized in that, when the approaching degree of the preceding vehicle is larger than a predetermined value, and when the approaching degree of the preceding vehicle is smaller than the predetermined value, different driver states are determined. (4) The invention according to claim 4 is the driver condition detecting device according to claim 2, further comprising a preceding vehicle approaching degree detecting means 49 for detecting an approaching degree to the preceding vehicle.
The determination of the driver state based on the line-of-sight frequency and the pupil diameter change rate according to No. 1 is also performed in consideration of the degree of approach of the preceding vehicle detected by the degree of approach of the preceding vehicle, and the driver state determination means 11 It is characterized in that different driver states are determined when the approaching degree of the preceding vehicle is larger than a predetermined value and when the approaching degree of the preceding vehicle is smaller than the predetermined value. (5) According to a fifth aspect of the present invention, in the driver state detecting device of the first aspect, there is further provided a preceding vehicle approaching degree detecting means 49 for detecting an approaching degree to the preceding vehicle. Means 49 for calculating the preceding vehicle approach degree change rate during a predetermined time of the detected preceding vehicle approach degree, and detecting the driver state based on the pupil diameter change rate by the driver state determining means 11 The determination is performed based on the diameter change rate and a predetermined value for the determination.
Is characterized in that a predetermined judgment value relating to the pupil diameter change rate is changed based on the calculated preceding vehicle approach degree change rate. (6) According to a sixth aspect of the present invention, in the driver state detecting device of the second aspect, there is further provided a preceding vehicle approaching degree detecting means 49 for detecting an approaching degree to the preceding vehicle. Means for calculating the rate of change of the degree of approach of the preceding vehicle at a predetermined time of the degree of approach of the detected preceding vehicle. The driver state determination means 11 detects the driver state based on the gaze direction frequency and the pupil diameter change rate. Is performed on the basis of the gaze direction frequency and the determination predetermined value thereof, and the pupil diameter change rate and the determination predetermined value thereof. The driver state determination means 11 determines the gaze direction frequency determination predetermined value and / or the pupil. It is characterized in that a predetermined value for determination regarding the diameter change rate is changed based on the calculated preceding vehicle approach degree change rate. (7) According to a seventh aspect of the present invention, in the driver state detecting apparatus according to the third aspect, the preceding vehicle approaching degree detecting means 49 calculates a preceding vehicle approaching degree change rate in a predetermined time of the detected preceding vehicle approaching degree. Means 49 for determining the driver state
1, the detection of the driver state based on the pupil diameter change rate,
The determination is performed based on the pupil diameter change rate and the determination predetermined value, and the driver state determination unit 11 changes the determination predetermined value regarding the pupil diameter change rate based on the calculated preceding vehicle approach degree change rate. It is characterized by. (8) According to an eighth aspect of the present invention, in the driver state detecting apparatus according to the fourth aspect, the preceding vehicle approaching degree detecting means 49 calculates a preceding vehicle approaching degree change rate in a predetermined time of the detected preceding vehicle approaching degree. Means 49 for determining the driver state
1, the detection of the driver state based on the gaze direction frequency and the pupil diameter change rate is performed based on the gaze direction frequency and the judgment predetermined value, and the pupil diameter change rate and the judgment predetermined value. The judging means 11 is characterized in that the judgment predetermined value regarding the gaze direction frequency and / or the judgment predetermined value regarding the pupil diameter change rate is changed based on the calculated preceding vehicle approach degree change rate. (9) According to the ninth aspect of the present invention, in the driver state detecting apparatus of the first aspect, the driver state determining means 1 further includes a preceding vehicle approaching degree detecting means 49 for detecting an approaching degree to the preceding vehicle.
1 is to determine one of two or more set driver states based on the pupil diameter change rate when the detected preceding vehicle approach level is equal to or more than a predetermined value. . (10) According to a tenth aspect of the present invention, in the driver state detecting apparatus of the second aspect, the driver state determining means 11 further includes a preceding vehicle approaching degree detecting means 49 for detecting an approaching degree to the preceding vehicle. When the detected preceding vehicle approaching degree is equal to or more than a predetermined value, based on the gaze direction frequency and the pupil diameter change rate, it is characterized by determining any one of two or more set driver states. I do. (11) According to the eleventh aspect of the present invention, in the driver state detecting device according to any one of the third, fifth, and seventh aspects, the driver state determining means 11 determines whether the detected preceding vehicle approaching degree is equal to or smaller than the detected degree. When the value is equal to or more than a predetermined value, one of two or more set driver states is determined based on the pupil diameter change rate. (12) According to a twelfth aspect of the present invention, in the driver status detecting device according to any one of the fourth, sixth and eighth aspects, the driver status determining means 11 determines whether the detected preceding vehicle approaching degree is equal to or smaller than the detected level. When the value is equal to or more than the predetermined value, one of the two or more set driver states is determined based on the gaze direction frequency and the pupil diameter change rate. (13) The invention of claim 13 is the invention of claims 1 to 12
In the driver condition detecting device according to any one of the above, further comprising: an alarm presenting means 15 for presenting an alarm by voice or the like; and a braking force control means 17 for controlling a braking force so as to generate a target deceleration. The alarm issuance by the alarm presenting means 15 and the braking force control by the braking force control means 17 are performed in accordance with the driver's state determined by the driver state determining means. And the braking force control means 17 changes the braking force control start timing in accordance with the determined state of the driver. (14) The invention of claim 14 is the invention of claims 1 to 13
The steering condition detection device according to any one of the above, further comprising a steering gear ratio variable means 19 for changing a steering gear ratio which is a ratio of a front wheel steering angle output amount to a driver steering angle input amount. The ratio varying means 19 changes the steering gear ratio according to the driver's state determined by the driver state determining means 11.

[0005] In the section of the means for solving the above-mentioned problems, which explains the configuration of the present invention, the drawings of the embodiments are used to explain the present invention in an easy-to-understand manner. However, the present invention is not limited to this.

[0006]

According to the present invention, the following effects can be obtained. (1) According to the first aspect of the invention, a gaze direction detecting means for detecting a gaze direction of a driver, a pupil diameter detecting means for detecting a pupil diameter of a driver, and a pupil diameter detected by the pupil diameter detecting means. And a pupil diameter change rate calculating means for calculating a change rate of the absolute value of the pupil diameter at a predetermined time with respect to the absolute value of the pupil diameter. The state of the person can be reliably detected. (2) According to the second aspect of the present invention, the gaze direction frequency is distributed from a plurality of gaze direction data detected at a predetermined time, and the driver's gaze direction frequency and the pupil diameter change rate in a specific area are determined. Since the state is determined, the state of the driver can be reliably detected. (3) According to the third or fourth aspect of the present invention, the vehicle further includes a preceding vehicle approaching degree detecting means for detecting the approaching level to the preceding vehicle. When the degree of approach is smaller than a predetermined value, a different driver state is determined. Therefore, it is difficult to make a determination based on the detection content using the driver's action as an index. It is possible to determine the driver state that has been converted. (4) According to the fifth to eighth aspects of the present invention, the predetermined judgment value relating to the gaze direction frequency and / or the predetermined judgment value relating to the pupil diameter change rate is changed based on the preceding vehicle approach degree change rate. In addition, a driver state corresponding to an external environment can be detected. (5) According to the ninth to twelfth aspects of the present invention, when the detected degree of approach of the preceding vehicle is equal to or greater than the predetermined value, two or more are set on the basis of the data regarding the gaze direction and the pupil diameter change rate. Since any one of the driver states is determined, it is possible to detect an excessively nervous state of the driver in a situation where the vehicle is approaching the preceding vehicle rapidly. (6) According to the thirteenth aspect, the alarm presenting timing and the braking force control start timing are respectively changed according to the determined driver's state. An appropriate driving support function according to the state can be provided. (7) According to the fourteenth aspect of the present invention, the steering gear ratio is changed according to the determined driver's state, so that in addition to the effects described above, appropriate driving support according to the driver's state is provided. Function can be provided.

[0007]

DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment FIG. 1 shows a schematic configuration of a driver state detecting device according to the present invention. This driver state detecting device includes a gaze direction detecting means 1, a gaze direction frequency distribution calculating means 3, a pupil diameter detecting means 5, a pupil diameter change rate calculating means 7, a vehicle surrounding environment measuring means 9, a driver state determining means 11, Warning / vehicle control determining means 13, warning presenting means 1
5, vehicle braking control means (braking force control means) 17 and steering gear ratio variable control means (steering gear ratio variable means) 19.

FIG. 2 is a block diagram showing a functional configuration of the driver state detecting device according to the present invention. A CCD camera 21 that captures a face image including the driver's eyeball and an infrared radiator 23 that radiates infrared light to the driver's face are fixedly installed on a meter cluster or the like in front of the driver in the vehicle cabin (FIG. 16). CC
The image data captured by the D camera 21 and the data from the infrared irradiator 23 are input to the image processing unit 25, and the pupil center / corneal reflection point detection unit 27 detects the driver's pupil center and the corneal reflection point. Using these detection results, the line-of-sight direction detection unit 29 uses the line-of-sight direction E (θu,
θv). The line-of-sight direction E (θu, θv) is represented by an angle-based coordinate value with the reference eye point P0 as the origin. The calculation method will be described later.

The detected line-of-sight direction data E (θu, θ
v) is transmitted to the electronic control unit 31. The gaze direction area determining unit 33 receives the gaze direction data E (θ
u, θv) is determined in which of the plurality of divided areas D, that is, where in the area D is being watched.
To set the area D, there are a method of dividing the vehicle fixed projection plane into a plurality of rectangular areas as shown in FIG. 3, and a method of defining a specific area such as a mirror or a meter as shown in FIG. .

When the area D is set as shown in FIG. 4, the area to be extracted may not be adjacent to the area D. However, when the gaze direction E (θu, θv) exists outside the set area, the gaze frequency calculation is not performed. Excluded from the subject. Gaze direction data E (θ
u, θv) is held in a buffer (not shown) for a certain period of time, and the gaze behavior of the driver is detected from the data existing in the latest extraction area and the data existing in the latest extraction area. The range of the n areas di on the vehicle fixed projection plane, which are the elements of the setting area D, is stored in the line-of-sight direction area map storage unit 35 and read out when the electronic control unit 31 is started.
The gaze frequency in a predetermined time interval between the plurality of divided gaze direction regions is determined by the gaze direction frequency distribution calculation unit 37 based on the gaze direction data e (di) for which the corresponding region di has been determined by the gaze direction region determination unit 33. The distribution F is calculated.

Further, in the image processing unit 25, a pupil diameter detecting section 39 detects a driver's pupil diameter dr from a result detected by the pupil center / corneal reflection point detecting section 27. The detected pupil diameter dr is sent to the electronic control unit 31, and the pupil diameter change rate calculation unit 41 calculates the pupil diameter change rate drv of the detected pupil diameter dr in a predetermined time. The pupil diameter change rate drv indicates the rate of change with respect to the reference value d0, where d0 is the reference pupil diameter.

The radar 43 measures an inter-vehicle distance to a preceding vehicle using, for example, a laser radar or a millimeter-wave radar.
The vehicle speed sensor 45 measures the vehicle speed of the own vehicle. These detection results are input to the preceding vehicle approaching degree calculating section (preceding vehicle approaching degree detecting means) 49 on the radar processing unit 47, and the approaching degree Lf to the preceding vehicle is calculated. The proximity degree data Lf to the preceding vehicle includes general environmental information such as inter-vehicle distance, inter-vehicle time (inter-vehicle distance / own vehicle speed), and TT.
Either C (inter-vehicle time / relative vehicle speed) or the reciprocal of TTC can be used.

The electronic control unit 31 is based on the gaze frequency distribution (gaze direction frequency distribution) F detected by the gaze direction frequency distribution 37 and the pupil diameter change rate drv calculated by the pupil diameter change rate calculation unit 41. The driver status detection unit 51 detects the status of the driver. Driver state detection unit 51
Can store the detected results for a long time. The detection result is sent to the alarm / control determination unit 53, and based on the detection result, an alarm is presented and the vehicle control content is determined.
These results are sent to each controller (FIG. 15) via the alarm / control command output unit 55. The alarm is presented by voice from the speaker 57. The braking force control is performed via the brake hydraulic controller 59 via the wheel cylinder 6.
The control is performed by controlling the hydraulic pressure applied to 1. The steering gear ratio is controlled by the steering gear ratio variable controller 63, and the steering gear ratio of the steering gear box 65 is changed. Driver state detector 5
Normally, the driver state detected in 1 is set to be inattentive, check surroundings, change lanes, look aside, excessively nervous, and the like.

Here, the pupil center / corneal reflection point detection unit 27
Now, a method of calculating the driver's line-of-sight direction E (θu, θv) will be described. 2 of detected pupil center and corneal reflection point
From the coordinate positional relationship of the point on the image, the horizontal direction angle θu and the vertical direction angle θ in the two-dimensional vehicle fixed projection plane angle coordinate system when facing the vehicle forward direction as the line of sight.
Calculate v. Here, as shown in FIG. 3, the vehicle fixed projection plane angle coordinate system has the reference eye point P0 as the origin,
The x-axis is a horizontal angle θu, and the y-axis is a vertical angle θv.

In the eye gaze measuring method based on the corneal reflection method and the image processing, the head movement can be corrected by using a reflection point generated on the cornea. This technique is described in “Behavior Research Meth.
ods & Instrumentation 1975,
397-429, Survey of eye mov
For example, a technique disclosed in “element recording methods” or the like is used. From the horizontal distance LPu and the vertical distance LPv in consideration of the reference eye point P0 and the eye point P1 after the head movement correction in the vehicle fixed projection plane distance coordinate system, the driver's gaze direction E (θu,
θv) is represented by the following (Equation 1).

[Formula 1] θu = Ku × LPu θv = Kv × LPv (Equation 1) where Ku and Kv are a horizontal distance LPu and a vertical distance for converting a vehicle fixed projection plane distance coordinate system into an angular coordinate system. LPv is a coefficient applied to each.

The size of an adult pupil is about 4 mm on average, but the pupil diameter changes depending on the amount of light entering the eye. Also,
The pupil is governed by the parasympathetic and sympathetic nerves, so when you get tired or sleepy, the pupil diameter decreases,
Conversely, the pupil diameter increases when excited. The driver state detecting device according to the present invention detects the state of the driver by paying attention to the change in the pupil diameter. For example, when the pupil diameter becomes smaller than the reference pupil diameter, it is determined that the driver who has been driving in the normal state has fallen into a loose state, and when the pupil diameter of the driver becomes larger than the reference value, it is determined that the driver is in a tense state.

The pupil diameter reference value d0 described above varies depending on the light environment and individual differences. Therefore, the pupil diameter at the start of traveling is set to the reference value d0, and the driver state at the start of traveling is determined to be a normal state concentrated on driving. If the detected value of the optical sensor (not shown) has changed by a predetermined value or more, the determination routine shown below is stopped, and the process waits until the change level of the detected value is stabilized, and returns the determination routine from the normal state. And

A control processing procedure in the electronic control unit 31 in the first embodiment will be described with reference to flowcharts shown in FIGS. This control processing is performed for a predetermined time (for example, 100 m) for a predetermined main program.
This is executed as a timer interrupt process every sec). In step S101, the gaze direction data E calculated by the image processing unit 25 is input. In step S103, it is determined where the input gaze direction data E exists in the plurality of divided gaze direction regions D, that is, the corresponding region di with reference to the gaze direction region map. Step S105
Then, the gaze direction frequency distribution F in which the gaze direction data E exists in the region di in the frame of the immediately preceding predetermined time section or the distribution relating to the abundance ratio in each region di in the total number of frames is calculated. In step S107, the pupil diameter data dr detected by the pupil diameter detector 39 is input. In step S109, a change rate drv of the pupil diameter dr with respect to the reference value d0 in a predetermined time interval is calculated using the input pupil diameter data dr.

In the next step S111, the data Lf of the degree of approach to the preceding vehicle calculated by the degree-of-preceding-vehicle-approach calculating section 49 is input. The preceding vehicle approach degree data Lf is used for detecting the state of the driver. In step S113, the degree of approach Lf to the preceding vehicle is compared with a predetermined level Lf0. The preceding vehicle proximity Lf is equal to or less than a predetermined level Lf0 (LF
≦ Lf0), if an affirmative determination is made, step S11
Go to 5. In step S115, the gaze frequency F in the forward direction is calculated from the gaze direction frequency distribution F calculated in step S105.
It is determined whether or not f is larger than a predetermined value Ff0. If it is determined that the gaze frequency Ff> Ff0, the process proceeds to step S117.

In step S117, it is determined whether the current driver state (actually, the driver state detected in the previous frame) s is the normal state sn. Step S1
If the determination in step 17 is affirmative and the driver state s is the normal state sn, the process proceeds to step S119. If the pupil diameter change rate drv is a negative value and smaller than the predetermined value -d1, that is, if it is determined that the pupil diameter change rate drv <the predetermined value -d1 is continuously determined for a predetermined number of times or more, the process proceeds to step S121. It is determined that the person state s is the inattentive state s1, and the step S1
At 23, a soundless driving warning is issued.

On the other hand, if the determination in step S117 is negative, and the driver state s of the previous frame is not the normal state sn, the process proceeds to step S125. In the step S125, it is determined whether or not the driver state s is the absent-minded state s1. If the determination in step S125 is affirmative, and the driver's state s in the previous frame is the inattentive driving s1, the process proceeds to step S127. Pupil diameter change rate drv is a positive value and exceeds predetermined value d2, that is, pupil diameter change rate drv
If it is determined that the predetermined value d2 has continued for a predetermined number of times or more, the process proceeds to step S129, and it is determined that the driver state s has returned to the normal state sn. The absolute value | d1 | of the predetermined judgment value used in steps S119 and S127,
The relationship of | d2 | is | d1 | <| d2 |.

Further, if the determination in step S125 is negative, and the driver state s is not the inadvertent state s1, the driver state s in the previous frame was the side-by-side state s3, but it is assumed that the state has shifted to the normal state sn. (Step S
133). Therefore, the process proceeds to step S135 to release the aside warning.

In step S115, the frequency of gaze forward F
When it is determined that f is equal to or less than the determination predetermined value Ff0, FIG.
The process proceeds to step S137 shown in FIG. In step S137, from the gaze direction frequency distribution F calculated in step S105, the mirror gaze frequency Lm, which is the gaze frequency with respect to the mirror area in the gaze direction region D, is compared with a predetermined judgment value Lm0. Here, as the mirror area in the line-of-sight direction area D, for example, the room mirror d3 shown in FIG.
A left side mirror d4 and a right side mirror d5 can be set. If it is determined that the mirror gaze frequency Lm is greater than the predetermined value Lm0 (Lm> Lm0), the process proceeds to step S13.
Proceed to 9 and change the driver state s to the normal and surrounding environment confirmation state s
2a. If a negative determination is made in step S137, the process proceeds to step S141, in which the driver state s is looked aside and the state s3 is set.
Then, an aside warning is issued (step S143).

If the result of the determination in step S113 is negative, and the preceding vehicle proximity Lf is greater than the predetermined level Lf0, the flow proceeds to step S145 shown in FIG. In step S145, the forward gaze frequency Ff is equal to the predetermined value Ff0.
It is determined whether or not the value is greater than the value. If the determination is affirmative, the process proceeds to step S147 to determine whether or not the driver state s of the previous frame is the normal state sn. When the driver state s is the normal state sn, the process proceeds to step S149, and the pupil diameter change rate drv is a negative value and less than a predetermined value -d1, that is, drv <-d1 continues for a predetermined number of times or more. It is determined whether or not it has been done. If an affirmative determination is made in step S149, the driver state s is determined to be the inadvertent state s1 in step S151, the approach warning timing is changed in step S153, and the automatic brake operation timing is changed in step S155.

FIG. 10 shows the relationship between the time t, the degree of approach Lf to the preceding vehicle, the approach warning, and the automatic brake operation timing. As the time t elapses, the degree of approach Lf to the preceding vehicle increases. Here, time t1 is
It is assumed that the driving state is detected, w1 and w2 are alarm timings in the initial state, and B is automatic brake operation timing in the initial state. The timings w1 ', w2', and B 'are respectively changed so that the warning issuance and the automatic braking operation are started early, that is, while the approach Lf of the preceding vehicle is relatively small.

On the other hand, if a negative determination is made in step S149, the process proceeds to step S157, where the pupil diameter change rate drv
Is greater than a predetermined value d4, that is, whether drv> d4 has continued for a predetermined number of times or more. Step S
If the determination in step 157 is affirmative, in step S159, the driver state s is determined to be in the excessively tensed state s4, and the process proceeds to step S161 to activate the automatic brake. FIG.
2 shows the relationship between the time t, the degree of approach Lf to the preceding vehicle, and the automatic brake operation timing. As in the case of FIG. 10, as the time t elapses, the degree of approach Lf to the preceding vehicle increases. At this time, the automatic braking operation timing is changed from B to B ′, and the point t of the driver state detection is changed.
At 1, the automatic brake operation is started.

If it is determined in step S147 that the driver state s of the previous frame is not the normal state sn,
Proceeding to step S163, the driver's state s is in the indiscreet state s1
Is determined. If it is in the drunken state s1, the process proceeds to step S165 to compare the pupil diameter change rate drv with a predetermined value d2. In step S165, the pupil diameter change rate drv exceeds the predetermined value d2, that is, dr
When it is determined that v> d2 has been continued for the predetermined number of times or more, the process proceeds to step S167 to determine that the driver state s has returned to the normal state sn. In step S169, the approach warning timing is returned to the initial state, and in step S171, the automatic brake operation timing is returned to the initial state.

If a negative determination is made in step S163, the process proceeds to step S173 shown in FIG. 8, and it is determined whether the driver's state s is in an excessively tensed state s4. When it is determined that the driver state is the excessively tensed state s4, the process proceeds to step S175, and the pupil diameter change rate dr
v is a negative value and less than a predetermined value d3, that is,
It is determined whether pupil diameter change rate <predetermined value−d3 has been continued a predetermined number of times or more. If an affirmative determination is made in step S175, the process proceeds to step S177, where the driver state s is determined to be the normal state sn, and the automatic brake is released in step S179.

On the other hand, if it is determined in step S173 that the driver state s is not in the excessively tensed state s4, the process proceeds to step S181, where the driver state s is determined to be the normal state sn, and in step S183, the alarm timing is set. In step S185, the automatic brake operation timing is
At 187, the steering gear ratio is returned to the initial state.

If it is determined in step S145 shown in FIG. 7 that the forward gaze frequency Ff is equal to or less than the predetermined value Ff0, the process proceeds to step S189 shown in FIG. Step S
In 189, it is determined whether or not the driver state s is the normal state sn. If the driver state s is the normal state sn, the process proceeds to step S191 to compare the mirror gaze frequency Lm with the predetermined value Lm0. In step S191, the mirror watching frequency Lm>
If the predetermined value Lm0 is affirmatively determined, the driver state s is determined to be normal and lane change intention s2b in step S193, and the steering gear ratio is changed in step S195. The details of changing the steering gear ratio will be described later. If it is determined in step S191 that the mirror gaze frequency Lm is equal to or less than the predetermined value Lm0, the process proceeds to step S197, where the driver state s is determined to be the look-aside state s3, and step S1 is performed.
At 99, a warning is issued.

On the other hand, in step S189, the driver state s
Is not in the normal state sn, it is determined in step S201 whether or not the mirror gaze frequency Lm exceeds a predetermined value Lm0. If an affirmative determination is made in step S201, it is determined in step S203 that the driver state s is normal and lane change intention s2b, and the steering gear ratio is changed in step S205.

Here, a method of changing the steering gear ratio will be described. In order to improve the initial response and the convergence of the steering input by the steering operation, a term relating to the steering angular velocity is added as shown in the following (Equation 2).

Φ = (θ + kθv) × N (Equation 2) where φ is the actual steering angle of the front wheel, θ is the steering angle,
θv is the steering angular velocity, k is a proportional constant, and N is the steering gear ratio.

Step S195 or S205
When the steering gear ratio is changed by setting the proportional constant k in (Equation 2) as k> 1, the front wheel steering angle output with respect to the steering angle input is increased. When the steering gear ratio is returned to the initial value in step S187, the proportional constant k is set to 0. In addition,
Determination predetermined values d1, d2, d3, and d relating to pupil diameter change rate drv used as thresholds for driver state determination
The relation of 4 is | d1 | <| d2 |, | d3 | <| d4 |
And

As described above, the driver state detecting apparatus according to the first embodiment detects the driver's pupil diameter dr, and calculates the rate of change drv from the detected pupil diameter dr to the reference pupil diameter d0. Calculated. The driver's line of sight E
And the gaze frequency distribution F was calculated from this. Since the driver's state is detected based on the pupil diameter change rate drv and the gaze frequency distribution F, it is possible to reliably determine whether the driver is gazing at the front of the vehicle in a relaxed state or in a normal state.

The driver's condition also depends on the external environment such as the approaching speed of the preceding vehicle. By detecting the approaching speed Lf of the preceding vehicle and adding this to the criterion, the pupil diameter change rate drv and the gaze frequency are determined. It is possible to set a driver state that is difficult to determine only by the distribution F, and detect these. For example, when the degree of approach to the preceding vehicle is high, an excessively tensed state of the driver, which is seen when the vehicle rapidly approaches the preceding vehicle, can be set as the driver state to be detected.
In addition, even when it is determined that the vehicle is in the drunken state, it is possible to provide the driver with a driving support function according to each situation when the approaching level of the preceding vehicle is high or low. That is,
The algorithm for determining the driver state is different depending on whether the detected preceding vehicle approach degree exceeds a predetermined value or not, and it is possible to detect a more reliable driver state in consideration of the external environment. it can.

<< Second Embodiment >> In a second embodiment of the driver state detecting apparatus according to the present invention, the forward gaze frequency used for determining the driver state in the first embodiment is used. The predetermined value Ff0 relating to Ff and the predetermined values d1 to d4 relating to the pupil diameter change rate drv are determined by the preceding vehicle proximity L
It is set based on the rate of change ΔLf of f. The schematic configuration and the functional configuration in the second embodiment are shown in FIGS.
This is the same as the first embodiment shown in FIG. Hereinafter, differences from the first embodiment will be mainly described.

The preceding vehicle approach degree change rate ΔLf in a predetermined time is calculated from the preceding vehicle approach degree Lf calculated by the preceding vehicle approach degree calculating section 49. FIG. 12 shows a relationship between the preceding vehicle proximity Lf and the time t. Among the states a to c, the rate of change in the degree of proximity to the state a is the highest, and the rate of change in the degree of proximity to the state c is the lowest.

The forward fixation frequency F used in the first embodiment
Assuming that the predetermined determination value of f is Ff0b, the predetermined predetermined value Ff0 of the forward gaze frequency Ff according to the second embodiment is represented by the following (Equation 3).

Ff0 = KFf × Ff0b (0 <KFf <1) (Equation 3) Here, KFf is a coefficient related to the predetermined value Ff0b of the gaze frequency determination ahead, and the coefficient KFf and the preceding vehicle approach degree change rate ΔLf Have a relationship as shown in FIG. The coefficient KFf is set to decrease as the preceding vehicle approaching degree ΔLf increases, and accordingly, the forward gaze frequency determination predetermined value Ff0 also decreases.

Pupil diameter change rate d used in the first embodiment
Assuming that the rv determination predetermined values d1 to d4 are drv0b, the pupil diameter change rate drv determination predetermined value drv0 according to the second embodiment is expressed by the following (Equation 4).

Drv0 = Kdrv × drv0b (0 <Kdrv <1) (Equation 4) Here, Kdrv is a predetermined value of the pupil diameter change rate determination drv0b.
The coefficient Kdrv and the preceding vehicle approach degree Δ
Lf has a relationship as shown in FIG. The coefficient Kdrv is set to decrease as the preceding vehicle approaching degree ΔLf increases, and accordingly, the pupil diameter change rate determination predetermined value drv0 also decreases.

By changing each predetermined value in accordance with the preceding vehicle approach degree change rate ΔLf, if the preceding vehicle approach degree change rate ΔLf is high and the vehicle is approaching the preceding vehicle rapidly, the preceding vehicle It is possible to detect a slight change in the pupil diameter and detect an excessively tensed state of the driver, as compared with the case where the rate of change in proximity degree ΔLf is low.

In the above-described driver state detecting device,
The gaze direction frequency distribution is calculated from the gaze direction data, but the change in the absolute value of the pupil diameter when the driver watches a specific region for a certain period of time is calculated, and the driver state is detected. For example, the gaze direction frequency distribution need not be calculated.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of a driver state detecting device according to the present invention.

FIG. 2 is a block diagram showing a functional configuration of a driver state detecting device according to the present invention.

FIG. 3 is a diagram illustrating an example of a line-of-sight direction area divided into a plurality of rectangular shapes;

FIG. 4 is a diagram showing an example of a line-of-sight direction area targeting a plurality of specific vehicle parts;

FIG. 5 is a flowchart showing control processing in the electronic control unit 31 according to the first embodiment.

FIG. 6 is a flowchart illustrating a control process in the electronic control unit 31 according to the first embodiment.

FIG. 7 is a flowchart illustrating control processing in the electronic control unit 31 according to the first embodiment.

FIG. 8 is a flowchart illustrating control processing in the electronic control unit 31 according to the first embodiment.

FIG. 9 is a flowchart illustrating control processing in the electronic control unit 31 according to the first embodiment.

FIG. 10 is a diagram showing a change in timing of presentation of an alarm and timing of automatic braking operation when a drunken state is detected.

FIG. 11 is a diagram showing a change in the timing of the automatic braking operation when an excessive tension state is detected.

FIG. 12 is a diagram showing a relationship between a time t and a preceding vehicle proximity Lf according to the second embodiment.

FIG. 13 is a diagram showing a relationship between a preceding vehicle approach degree change rate ΔLf and a coefficient KFf relating to a predetermined value of a gaze frequency determination in front;

FIG. 14 is a diagram showing a relationship between a preceding vehicle approach degree change rate ΔLf and a coefficient Kdrv related to a predetermined value for determining a pupil diameter change rate.

FIG. 15 is a diagram showing the appearance of a vehicle provided with a driver state detecting device according to the present invention.

FIG. 16 is a diagram showing the vicinity of the driver's seat of a vehicle provided with the driver state detecting device according to the present invention.

[Explanation of symbols]

 21: CCD Camera 23: Infrared Irradiator 25: Image Processing Unit 31: Electronic Control Unit 43: Radar 47: Radar Processing Unit 57: Speaker 59: Brake Controller 61: Wheel Cylinder 63: Steering Gear Ratio Variable Controller 65: Steering Gear box

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) B60R 21/00 624 B60R 21/00 624D 624G 626C 626 27/00 27/00 B62D 6/00 B62D 6/00 101: 00 // B62D 101: 00 109: 00 109: 00 A61B 3/10 A F term (reference) 3D032 CC21 DA23 DA88 DA98 DB20 DC31 DD02 DD13 DE03 EC31 FF01 GG01 3D037 FA10 FA25 FA26 FB10 5H180 AA01 CC02 CC03 CC04 CC12 CC14 LL04 LL07 LL09 LL20

Claims (14)

[Claims] 1. A gaze direction detecting means for detecting a driver's gaze direction on a vehicle fixed projection plane; a pupil diameter detecting means for detecting a driver's pupil diameter; and a pupil diameter detected by the pupil diameter detecting means. A pupil diameter change rate calculating means for calculating a change rate in a predetermined time with respect to an absolute value of the pupil diameter; data on a gaze direction detected by the gaze direction detecting means; and a pupil calculated by the pupil diameter change rate calculating means. A driver state detection device comprising: driver state determination means for determining a driver state based on a diameter change rate. 2. The driver state detection device according to claim 1, wherein a plurality of line-of-sight direction data detected during a predetermined time is used.
The apparatus further includes a gaze direction frequency distribution calculation unit that calculates a gaze direction frequency distribution in a plurality of divided regions, and the driver state determination unit includes a gaze direction in a specific region calculated by the gaze direction frequency distribution calculation unit. A driver state detection device that determines a driver state based on a direction frequency and the pupil diameter change rate. 3. The driver state detecting device according to claim 1, further comprising a preceding vehicle approach degree detecting means for detecting an approach degree to a preceding vehicle, wherein said driver state determining means changes said pupil diameter. The determination of the driver state based on the rate is performed in consideration of the preceding vehicle approach degree detected by the preceding vehicle approach degree detection means, and the driver state determination means determines that the preceding vehicle approach degree is higher than a predetermined value. A different driver state is determined depending on whether the vehicle is approaching the vehicle or when the degree of approach of the preceding vehicle is smaller than a predetermined value. 4. The driver state detecting device according to claim 2, further comprising: a preceding vehicle approach degree detecting means for detecting an approach degree to a preceding vehicle; And determining the driver state based on the pupil diameter change rate in consideration of the preceding vehicle approach degree detected by the preceding vehicle approach degree detecting means. A driver state detection device that determines different driver states depending on whether the degree of approach is larger than a predetermined value or when the degree of approach of the preceding vehicle is smaller than a predetermined value. 5. The driver state detecting device according to claim 1, further comprising a preceding vehicle approaching degree detecting means for detecting an approaching degree to a preceding vehicle, wherein said preceding vehicle approaching degree detecting means detects the degree of approach to the preceding vehicle. Calculating a rate of change of the degree of approach of the preceding vehicle during a predetermined time of the degree of approach of the preceding vehicle, and detecting the driver state based on the rate of change of the pupil diameter by the driver state determining means. And the determination predetermined value, and the driver state determination unit changes the determination predetermined value related to the pupil diameter change rate based on the calculated preceding vehicle approach degree change rate. A driver state detecting device. 6. The driver status detecting device according to claim 2, further comprising a preceding vehicle approaching degree detecting means for detecting an approaching degree to a preceding vehicle, wherein said preceding vehicle approaching degree detecting means detects the degree of approach to the preceding vehicle. Means for calculating a preceding vehicle approach degree change rate in a predetermined time of the preceding vehicle approach degree, wherein the driver state determination means detects a driver state based on the gaze direction frequency and the pupil diameter change rate. The gaze direction frequency and the determination predetermined value thereof, and the pupil diameter change rate and the determination predetermined value thereof, and the driver state determination means determines the determination value relating to the gaze direction frequency, and / or And a determination value for the pupil diameter change rate is changed based on the calculated preceding vehicle approach degree change rate. 7. The driver state detecting device according to claim 3, wherein the preceding vehicle approaching degree detecting means calculates a preceding vehicle approaching degree change rate in a predetermined time of the detected preceding vehicle approaching degree. The driver state determining means detects a driver state based on the pupil diameter change rate based on the pupil diameter change rate and a determination predetermined value thereof. And a determination value for the pupil diameter change rate is changed based on the calculated preceding vehicle approach degree change rate. 8. The driver state detecting device according to claim 4, wherein the preceding vehicle approach degree detecting means calculates a preceding vehicle approach degree change rate in a predetermined time of the detected preceding vehicle approach degree. The detection of the driver state based on the gaze direction frequency and the pupil diameter change rate by the driver state determination means includes the gaze direction frequency and a predetermined determination value thereof, and the pupil diameter change rate and a predetermined determination thereof. The driver state determining means may determine the predetermined value relating to the gaze direction frequency and / or the predetermined value relating to the pupil diameter change rate by the calculated preceding vehicle change degree. A driver state detection device, wherein the change is made based on a rate. 9. The driver state detecting device according to claim 1, further comprising: a preceding vehicle approaching degree detecting means for detecting an approaching degree to a preceding vehicle, wherein the driver state determining means detects the detected state. A driver state detection device for determining one of two or more set driver states based on the pupil diameter change rate when the preceding vehicle approaching degree is equal to or more than a predetermined value. . 10. The driver state detecting device according to claim 2, further comprising: a preceding vehicle approaching degree detecting means for detecting an approaching degree to a preceding vehicle, wherein said driver state determining means detects said detected state. When the preceding vehicle approaching degree is equal to or more than a predetermined value, a driver state of two or more set driver states is determined based on the gaze direction frequency and the pupil diameter change rate. Driver status detection device. 11. The driver status detection device according to claim 3, wherein the driver status determination means is configured to determine that the detected preceding vehicle approach level is equal to or greater than a predetermined value. In this case, the driver state detection device determines one of two or more set driver states based on the pupil diameter change rate. 12. The driver status detection device according to claim 4, wherein the driver status determination means is configured to determine that the detected preceding vehicle approach level is equal to or greater than a predetermined value. In this case, a driver state detecting device determines one of two or more set driver states based on the gaze direction frequency and the pupil diameter change rate. 13. A driver state detecting apparatus according to claim 1, wherein: an alarm presenting means for presenting an alarm by voice or the like; and a braking force for generating a target deceleration. A braking force control unit that controls the vehicle, and performs an alarm issuance by the warning presenting unit and a braking force control by the braking force control unit in accordance with a driver state determined by the driver state determination unit. The warning state presenting means changes the warning presenting timing, and the braking force control means changes the braking force control start timing according to the determined state of the driver. . 14. A steering device according to claim 1, wherein a steering gear ratio which is a ratio of a front wheel steering angle output to a driver steering angle input is changed. A driver state detecting device further comprising a gear ratio variable unit, wherein the steering gear ratio variable unit changes a steering gear ratio according to the driver's state determined by the driver state determining unit. .