JP3815842B2 - Dozing alarm device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To zoom up a photographed image so as to heighten awake degree estimating accuracy by actuating an alarm at the time of judging lowering of an awake degree, setting a reference coordinate position and the deviation allowable value, and adjusting the focal length of a zoom lens. SOLUTION: An image filter extracts eyes accurately from an image of a CCD camera 2 and makes the output image binary. A doze detecting part 13 detects a wink from the inclination change of the corner of the eye on the basis of the binary image and judges lowering of an awake degree. An alarm part 15 actuates an alarm 3 in a cabin when the lowering of the awake degree is judged in the doze detecting part 14. In a bright state based on output data of an image processing part 13, the focal length of a zoom lens is adjusted so that deviation between the measured coordinate position of both eyes on the binary image and a reference coordinate position is within the allowable value. In a dark state, the coordinate position of both eyes on a retinal reflected image is measured, and the focal length is adjusted in the same way.

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an improvement in a device that detects and alerts a driver to fall asleep.
[0002]
[Prior art]
For the purpose of preventing accidents due to the driver's falling asleep, the driver's face (CCD camera, TV camera, etc.) is placed on the instrument panel and the driver is based on the eye information obtained from the image. It is known that the degree of arousal is estimated and an alarm is activated when the degree of arousal falls below a threshold value (Japanese Patent Laid-Open Nos. 7-32907, 7-156682, Mitsubishi Electric Technical Report Vol. .70 No.9 1996 “Dozing detection technology”).
[0003]
[Problems to be solved by the invention]
In such a conventional device, the most important thing for accurately estimating the driver's arousal level is how to accurately capture the driver's eye information. In the conventional method, when the driver changes the seat position (front / rear direction or vertical position of the driver's seat) or the driver changes, the driver's face is removed from the image (view) captured by the camera. As a countermeasure, it is required to set the shooting range of the camera at a wide angle. However, when the field of view of the camera is set to a wide angle in this way, the driver's face portion occupied in the camera image is reduced, and the eye information obtained therefrom is also reduced. That is, there is a problem that it is difficult to ensure estimation accuracy with a high arousal level.
[0004]
It is an object of the present invention to provide effective countermeasure means for such problems.
[0005]
[Means for Solving the Problems]
In the first invention, as shown in FIG. 5, means a for photographing a driver's face through a zoom lens, and means for extracting eyes from image processing for binarizing the output of the image using a special image filter. b, means c for measuring a closing eye time per fixed time based on eyes extracted by image processing, means d for setting a predetermined closing time as a threshold of arousal level, and a measurement value of the closing time A means e for determining a decrease in the driver's arousal level when the threshold is exceeded, a means f for controlling the driver's cab alarm to be activated when the decrease in the arousal level is determined, and the focus of the zoom range when the driver starts sitting Means m for setting the distance to the wide side, means k for adjusting the shooting position so that the center connecting the left and right eyes extracted by image processing is at the center of the image, and the coordinate position on the image of the eyes that are also extracted Means h for measuring and a predetermined zoom Means i for setting the reference coordinate position of the eye and an allowable value of the deviation on the image, and means j for adjusting the focal length of the zoom lens so that the deviation between the measured coordinate position of the eye and the reference coordinate position is within the allowable value And comprising.
[0006]
In the second invention, as shown in FIG. 6, means n for irradiating the driver's face with infrared light, means a for photographing the driver's face through a zoom lens, and means b for performing image processing thereof The means e for measuring the closing eye time per fixed time based on the retina reflection image, the means d for setting the predetermined closing time as a threshold value of the arousal level, and the driver when the measured value of the closing time exceeds the threshold value A means e for determining a decrease in the degree of arousal, a means f for controlling the driver's cab alarm to be activated when the decrease in the degree of arousal is determined, and setting the focal length of the zoom range to the wide side when the driver starts sitting Means m for adjusting the photographing position so that the center connecting the two retinal reflected lights is at the center of the image, means h for measuring the coordinate position of the retinal reflected light on the image, and a predetermined zoom image Upper eye reference coordinate position and deviation tolerance And means i for setting, means j for adjusting the focal length of the zoom lens so that the deviation between the measured coordinate positions and eye reference coordinate position of the retina reflected light fall within the allowable value.
[0007]
In 3rd invention, in FIG. 5 or FIG. 6, the means g which controls to stop a vehicle at the time of the fall determination of an arousal level is provided.
[0008]
【The invention's effect】
In the first invention, the driver's face is photographed through a zoom lens, and eyes are extracted by image processing. The driver's arousal level is determined based on the eyes, and a doze alarm is activated when the awakening level is reduced. The degree of arousal is determined as a decrease in consciousness level when this exceeds a threshold from the closing eye time per fixed time. When the driver starts sitting, the zoom lens is set to the wide side, and the photographing position is adjusted so that the center connecting the left and right eyes extracted by image processing is located at the center of the image. Therefore, even if the driver changes the seat position or changes the position of the face, the driver's face is adjusted so that the center of the left and right eyes is centered on the center of the image within a wide field of view. It ’s definitely caught. Then, the coordinate position on the image of both eyes is measured, and the zoom lens is set so that the deviation between the measurement position of the eye and the reference position falls within the allowable value based on the reference coordinate position on the predetermined zoom image and the allowable value of the deviation. The focal length is adjusted. As a result, the image taken by the driver is zoomed up with both eyes as the center, so that the estimation accuracy of the arousal level can be greatly increased.
[0009]
In the second invention, the driver's face is irradiated with infrared light, and the retina reflected light of the eye is photographed through the zoom lens. The driver's arousal level is determined based on the retina reflected light on the image, and a drowsiness alarm is activated when the arousal level decreases. The degree of arousal is determined as a decrease in consciousness level when this exceeds a threshold from the closing eye time per fixed time. When the driver starts seating, the zoom lens is set to the wide side, and the photographing position is adjusted so that the center connecting the left and right retinal reflected light on the image is at the center of the image. Therefore, even if the driver changes the seat position or changes the position of the face, the driver's face so that the center of the left and right retinal reflected light comes to the center of the image within a wide field of view. Can be captured accurately. Then, the coordinate position of the retinal reflected light on the image is measured, and the deviation between the measured position of the retinal reflected light and the reference position is within the allowable value based on the reference coordinate position on the predetermined zoom image and the allowable value of the deviation. The focal length of the zoom lens is adjusted to fit. As a result, the image taken by the driver is zoomed up with both eyes as the center, so that the estimation accuracy of the arousal level can be greatly increased.
[0010]
In the third invention, the vehicle is controlled to stop when the arousal level is lowered. Therefore, even when the driver does not respond to the alarm operation, the vehicle reaches a stopped state (stopped), so that the safety against the driver's sleep can be improved.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
In FIG. 1, a CCD camera 2 for photographing a driver's face and an alarm device 3 that operates when the driver's arousal level is lowered are installed on an instrument panel 1 in the driver's cab. A zoom lens 4 is attached to the CCD camera 2 as shown in FIG. 2, and a near infrared LED 5 is disposed around the zoom lens 4. The CCD camera 2 is attached to the instrument panel 2 through a support mechanism (not shown) so that the photographing position facing the driver (support angle in this example) can be adjusted.
[0012]
The controller 10 includes a light source control unit 11 that controls irradiation of the near-infrared LED 5, a camera control unit 12 that controls a support angle of the CCD camera 2 and a focal length of the zoom lens 4, and an image processing unit 13 of the CCD camera 2. The sleep detection unit 14 estimates the driver's arousal level and the alarm unit 15 controls the operation of the alarm device 3. The light source controller 11 performs control so that the low-intensity near-infrared LED 5 is operated toward the driver's face only in a dark state such as at night.
[0013]
The image processing unit 12 includes a special image filter. The image filter is used to accurately extract the eyes from the image of the CCD camera 2 in a bright state such as daytime. The image captured by the camera and a black region (such as the eyes) with a short vertical length of the face The difference from the image from which () is removed is output. The output image is binarized and becomes a binary image from which eyes and the like are extracted.
[0014]
The dozing detection unit 13 detects blinking from a change in the tilt of the corner of the eye based on the binary image. Then, it measures the closing time per eye (the inclination of the corner of the eye is gentler than the inclination of the opening eye), and compares the measured time with the threshold value of the degree of arousal to determine the decrease in the degree of arousal. .
[0015]
In a dark state such as at night, a retina reflection image of the eye is taken by the CCD camera 2 by the operation of the near infrared LED 5. The dozing detection unit 14 detects the presence or absence of retinal reflection shining brightly on the image via the image processing unit 13. Then, a closed eye (without retinal reflection) time per certain time is measured, and the measurement time is compared with a threshold value of the arousal level, and when the threshold value is exceeded, a decrease in the arousal level is determined.
[0016]
The alarm unit 15 activates the alarm device 3 in the driver's cab when the dozing detection unit 14 determines that the wakefulness level is lowered. In order to effectively avoid a drowsy driving, it is preferable to control the stop of the vehicle when the driver's arousal level decreases. An operation signal to the alarm device 3 can be used to start the stop control.
[0017]
The camera control unit 12 sets the zoom lens of the CCD camera 2 to the wide side when the driver starts sitting. The driver's face is then captured small in the wide image of the CCD camera 2. Then, based on the output data of the image processing unit 13, in a bright state such as daytime, the support angle of the CCD camera 2 is adjusted so that the center connecting the eyes of the binary image is at the center of the image.
[0018]
The reference coordinate position of the eye on the predetermined zoom image and the allowable value of the deviation are set in the memory of the camera control unit 12. Then, the coordinate position of both eyes on the binary image is measured, and the focal length of the zoom lens 4 is adjusted so that the deviation between the measured coordinate position and the reference coordinate position is within an allowable value.
[0019]
In a dark state such as at night, the support angle of the CCD camera 2 is adjusted based on the retina reflection image so that the center of both eyes is at the center of the image, while the coordinate position of both eyes on the retina reflection image is measured. The focal length of the zoom lens 4 is adjusted so that the deviation from the reference coordinate position falls within an allowable value.
[0020]
With this configuration, the driver's face is photographed by the CCD camera 2. The driver's arousal level is determined based on the eyes of the binary image or the retina reflection image, and a doze alarm is activated when the arousal level decreases. When the driver starts sitting, the zoom lens 4 is set to the wide side, and the support angle of the CCD camera 2 is adjusted so that the center connecting the eyes on the binary image or the retina reflection image is at the center of the image. Therefore, even if the driver changes the seat position or changes the position of the face, the driver's face is properly adjusted so that the center of both eyes is centered on the center of the image within a wide field of view. Captured.
[0021]
In addition, the zoom lens is configured so that the coordinate position on the images of both eyes is measured, and the deviation between the measurement position of the eye and the reference position falls within the allowable value based on the reference coordinate position on the predetermined zoom image and the allowable value of the deviation. 4 focal length is adjusted. Since the driver's photographed image is zoomed up around both eyes, the accuracy of estimating the arousal level can be greatly increased.
[0022]
FIG. 3 is a flowchart for explaining the control contents in a bright state. In step 1, the focal length of the CCD camera 2 is set to the wide side (shorter) when the driver is seated. In step 2, the driver's face is photographed by the CCD camera 2. In step 3, an image G1 from which a black region having a short vertical length is removed from the image G0 (original image) of the CCD camera 2 is obtained. In step 4, a difference G2 between the original image G0 and the image G1 from which the black area is removed is obtained. In step 5, an image G3 is obtained by subtracting the original image G0 from the image difference G2. In step 6, the eyes are extracted by binarizing the image G3.
[0023]
In steps 7 and 8, the support angle of the CCD camera 2 is adjusted so that the center connecting the eyes on the binary image is located at the center of the image. In step 9, the coordinate position of both eyes on the image is measured, and the deviation from the reference coordinate position is obtained. In steps 10 and 11, the focal length of the CCD camera 2 is adjusted to the zoom side (longer) so that the deviation falls within the allowable value. The left and right deviations are given as el and er absolute values of (elx ² + ery ² ) ^1/2 and (erx ² + ery ² ) ^1/2 .
[0024]
In step 12, the inclination α of the corner of the eye is measured. In step 13, it is determined whether α is less than a threshold value (which is a reference value for determining a closing eye and is preset in a memory). If yes, the closing flag f _{1 is set} to 1 in step 14. If no, the closing flag f _{1 is set} to 0 in step 15. In step 16, it is determined whether or not the closing flag f ₁ = 1. If yes, the process proceeds to step 17, and if no, the process jumps to step 18.
[0025]
In step 17, the duration of the closing eye is measured and the time is accumulated. In step 18 and step 19, when a certain time has elapsed, it is determined whether or not the cumulative time S ₋ f ₁ of the closing exceeds the arousal level threshold. If yes, the awakening level reduction flag f _{2 is set} to 1 in step 20. If no, the awakening level decrease flag f _{2 is set} to 0 in step 21.
[0026]
At step 22, closed eyes of the cumulative time S _- a f ₁ starts counting a predetermined time and at the same time to 0. In step 23 and step 24, when the arousal level reduction flag f ₂ = 1, the alarm device 3 (and vehicle stop control) is activated. Steps 2 to 24 are repeatedly executed at a predetermined control cycle.
[0027]
FIG. 4 is a flowchart for explaining the control contents in a dark state. In step 1, the focal length of the CCD camera 2 is set to the wide side (shorter) when the driver is seated. In step 2, the near-infrared LED 5 is operated. In step 3, the driver's face is photographed by the CCD camera 2. In step 4, a bright point is extracted from the retina reflection image G0 (original image) of the CCD camera 2 as an eye.
[0028]
In steps 5 and 6, the support angle of the CCD camera 2 is adjusted so that the center connecting both eyes on the retina reflection image is at the center of the image. In step 7, the coordinate position of both eyes on the image is measured, and the deviation from the reference coordinate position is obtained. In steps 8 and 9, the focal length of the CCD camera 2 is adjusted to the zoom side (longer) so that the deviation falls within the allowable value. The left and right deviations are given as el and er absolute values of (elx ² + ery ² ) ^1/2 and (erx ² + ery ² ) ^1/2 .
[0029]
In step 10, it is determined whether or not there is retinal reflex. If yes, the closing flag f _{1 is set} to 1 in step 11. If no, the closing flag f _{1 is set} to 0 in step 12. In step 13, it is determined whether or not the closing flag f ₁ = 1. If yes, the process proceeds to step 14, and if no, the process jumps to step 15.
[0030]
In step 14, the duration of the closing eye is measured and the total of the time is performed. Determining f ₁ is whether exceeds a threshold value of wakefulness _- step 15, in step 16, after a certain period of time, closing th cumulative time S. If yes, the awakening level reduction flag f _{2 is set} to 1 in step 17. If no, the awakening level reduction flag f _{2 is set} to 0 in step 18.
[0031]
At step 19, closed eyes of the cumulative time S _- a f ₁ starts counting a predetermined time and at the same time to 0. In step 20 and step 21, when the arousal level lowering flag f ₂ = 1, the alarm device 3 (and vehicle stop control) is activated. Steps 2 to 21 are repeatedly executed at a predetermined control cycle.
[Brief description of the drawings]
FIG. 1 is a schematic diagram illustrating an embodiment of the present invention.
FIG. 2 is a configuration diagram of a CCD camera.
FIG. 3 is a flowchart for explaining the control content in the same manner.
FIG. 4 is a flowchart for explaining the control contents in the same manner.
FIG. 5 is a diagram corresponding to claims of the present invention.
FIG. 6 is a diagram corresponding to claims of the present invention.
[Explanation of symbols]
2 CCD camera 3 Alarm 4 Zoom lens 5 Near infrared LED
10 controller 11 light source control unit 12 camera control unit 13 image processing unit 14 dozing detection unit 15 alarm unit

Claims (3)

Means for photographing the driver's face through a zoom lens, means for extracting eyes from image processing that binarizes the output using a special image filter, and eyes extracted by image processing Means for measuring the closing time per fixed time, means for setting a predetermined closing time as a threshold value of the awakening level, and means for determining a decrease in the driving level of the driver when the measured value of the closing time exceeds the threshold value And a means for controlling the driver's cab alarm to be activated when it is judged that the arousal level is lowered, a means for setting the focal length of the zoom range to the wide side when the driver starts sitting, and a left and right extracted by image processing Means for adjusting the shooting position so that the center of the eye is located at the center of the image, means for measuring the coordinate position on the extracted eye image, and the reference coordinate position and deviation of the eye on a predetermined zoom image Forgiveness Snooze alarm device comprising means for setting a value, and means for deviation of the measurement coordinate position and the reference coordinate position of the eye to adjust the focal length of the zoom lens to fall within the allowable value, further comprising: a. Means for irradiating the driver's face with infrared light, means for photographing the driver's face through a zoom lens, means for image processing, and closed eyes per fixed time based on a retina reflection image A means for measuring time, a means for setting a predetermined closing time as a threshold value of arousal level, a means for determining a decrease in the driver's arousal level when the measured value of the closing time exceeds the threshold value, and a reduction in the arousal level Means for controlling the alarm device in the cab at the time of determination, means for setting the focal length of the zoom range to the wide side when the driver starts sitting, and the center connecting the two retinal reflections is the center of the image Means for adjusting the photographing position so as to move, means for measuring the coordinate position of the retinal reflected light on the image, means for setting a reference coordinate position of the eye on the predetermined zoom image and an allowable value of the deviation, and retinal reflected light Measurement coordinate position and eye Snooze alarm device, wherein a deviation between the quasi-coordinate position is provided with means for adjusting the focal length of the zoom lens to fall within the allowable value. The dozing alarm device according to claim 1 or 2, further comprising means for controlling the vehicle to stop when it is determined that the degree of arousal is lowered.

Images