CN101803928A - Video-based driver fatigue detection device - Google Patents
Video-based driver fatigue detection device Download PDFInfo
- Publication number
- CN101803928A CN101803928A CN 201010119431 CN201010119431A CN101803928A CN 101803928 A CN101803928 A CN 101803928A CN 201010119431 CN201010119431 CN 201010119431 CN 201010119431 A CN201010119431 A CN 201010119431A CN 101803928 A CN101803928 A CN 101803928A
- Authority
- CN
- China
- Prior art keywords
- face
- human eye
- people
- pixel
- threshold value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a video-based driver fatigue detection device, which comprises an infrared illumination component, a lens, a filtering component, an image capturing component and an image processing unit, wherein the infrared illumination component is used for providing infrared illumination for the monitored scene; the lens is used for acquiring the infrared optical image signal of the monitored scene; the filtering component is used for carrying out filtering treatment on the received infrared optical image signal of the monitored scene so as to highlight the face and pupils in the infrared optical image signal of the monitored scene; the image capturing component is used for converting the filtered infrared optical image signal into a digital image signal; and the image processing unit is used for extracting eyes and pupils from the acquired digital image and judging whether the driver is tired. By utilizing pupils to judge whether the eyes are open or closed, the video-based driver fatigue detection device of the invention effectively improves the accuracy of judging the open/close state of eyes, and then accurately and rapidly detects whether the driver is tired.
Description
Technical field
The present invention relates to a kind of driver fatigue detection device, belong to Flame Image Process, field of video monitoring based on video.
Background technology
Driver fatigue, not have enough sleep be one of major incentive that causes severe traffic accidents, therefore supervises effectively and prevent that driver tired driving has crucial meaning.Fatigue detecting is the fatigue phenomenon that the driver occurs in driving to be detected and imposes the process of suitable warning in real time, and it has following requirement: 1) must be glitch-free; 2) must be real-time; 3) must be subjected to the influence of illumination less; 4) harmful radiation can not be arranged; 5) can not comprise mobile device.In existing various detection methods, can satisfy above requirement and effect and comparatively it is desirable to take in real time with video camera, detect the physical reactions of driver's eye then by Flame Image Process.
Application number is that 200610012623.6 Chinese patent application discloses a kind of method and device that utilizes Kalman wave filter and Mean Shift algorithm to detect degree of fatigue simultaneously.Publication number be US2006/0132319A1 U.S. Patent Application Publication a kind of driver fatigue equipment and the method estimated.Above-mentioned two patents all are to utilize face's characteristic (comprising eyebrow, eyes, nose, face) that driver's face image is carried out Flame Image Process earlier, and with the position of location eyes, whether closure judges whether the driver is in fatigue state by detecting eyes again.But owing to there be a lot of the interference in the actual scene, therefore often can not locate the position of eyes exactly, this has just influenced the judgement of later stage driver fatigue state.
Application number is that 200510037771.9 Chinese patent application discloses and a kind ofly utilizes Infrared that driver's eye is shone to obtain image, and the image that obtains is carried out Flame Image Process to detect the device and method of degree of fatigue.But the method for this patent is extracted the image inaccuracy of driver's eyes, under the brighter environment of light, causes easily and measures failure.
In sum, press for a kind of device that can detect driver fatigue state quickly and accurately of proposition at present.
Summary of the invention
In view of this, main purpose of the present invention is to provide a kind of device that can realize detecting quickly and accurately driver fatigue state.For achieving the above object, the invention provides a kind of driver fatigue detection device based on video, described driver fatigue detection device comprises:
At least one infrared illumination element is used for providing infrared illumination to monitoring scene;
Camera lens is used to obtain the infrared optics picture signal of monitoring scene;
Filter element is used for processings that filter of infrared optics picture signal to the monitoring scene that receives, with people's face and the human eye pupil part in the infrared optics picture signal of giving prominence to monitoring scene;
Image capturing component, the back that it is positioned at described camera lens is used for the infrared optics picture signal of optical processing after filtration is converted to data image signal; With
Graphics processing unit is used for extracting human eye and pupil from the digital picture of obtaining, and judges whether the driver is in fatigue state.
Wherein, described infrared illumination element is installed in individually to be used to drive the regional interior of area illumination or to be installed in and is used to drive in the zone of area illumination with ground connection being installed in that link to each other towards the described camera lens of driving the zone; Before described filter element is positioned at described camera lens, perhaps is positioned at the back of image capturing component or is positioned at described camera lens back and before described image capturing component;
Preferably, described infrared illumination element and described camera lens removably closely link together, and described camera lens, described filter element and described image capturing component removably closely link together each other.
Preferably, described infrared illumination element comprises at least one infrared illuminator.
Preferably, described filter element comprises the logical filter element of at least one band and/or at least one polarizing filter element.
Preferably, described image capturing component be ccd sensor, cmos sensor, CCD camera, CMOS camera, infrared imaging camera or infrared thermal release electric magazine one or more.
Preferably, described graphics processing unit comprises:
The candidate face acquisition module is used for extracting the candidate face zone from the digital picture of obtaining;
The human eye acquisition module is used to utilize eye detection method, detects human eye area from the candidate face zone that receives;
Whether human eye opens and closes determination module, be used for according to existing pupil to determine the switching of human eye in the human eye area; With
The driver fatigue judge module is used for the switching according to human eye, utilizes the PERCLOS value to judge whether the driver is in fatigue state.
Preferably, described candidate face acquisition module obtains the candidate face zone by following steps:
Step 511: go out the people face pixel of the higher pixel of brightness value by the digital picture of obtaining being carried out the threshold value segmented extraction, and obtain corresponding people's face threshold value split image as the candidate;
Step 512: the people's face pixel with the candidate in people's face threshold value split image is a template, utilizes method for detecting human face to obtain the candidate face zone on this template.
Preferably, the threshold value in the described step 511 is cut apart realization by the following method:
Suppose that (x, brightness value y) are that (x, y), (x, brightness value y) are I to pixel to I in people's face threshold value split image of digital picture to pixel in the digital picture
1(x, y), I wherein
1Initial value be I, (x, y) 〉=first threshold T1, then this pixel of labelling belongs to people's face pixel of candidate, keeps the brightness value I of this pixel in people's face threshold value split image if I
1(x, y) constant, otherwise this pixel of labelling is people's face pixel of non-candidate, with the brightness value I of this pixel in people's face threshold value split image
1(x y) gives 0, and wherein, 100≤T1≤200 and T1 are integer.
Preferably, method for detecting human face in the described step 512 is a method for marking connected region, described method for marking connected region is: at first the people's face pixel with the candidate in people's face threshold value split image is a target, all impact points in people's face threshold value split image are carried out connected component labeling, add up the number of people's face pixel of candidate in the connected region of labelling then, if this number is greater than the second threshold value T2, then this connected region of labelling is candidate's a human face region, otherwise with in this connected region be labeled as people's face pixel of non-candidate a little, with the brightness value I that is had a few in this connected region
1(x y) gives 0, and wherein, 300≤T2≤600 and T2 are integer.
Preferably, described human eye switching determination module judges that by following steps human eye opens and closes:
Step 531: in human eye area, extract pupil, at first the brightness value corresponding to the pixel in the human eye area in people's face threshold value split image is kept, give 0 with other the brightness value of pixel, then this people's face threshold value split image is carried out threshold value and cut apart, if I
1(x, y) 〉=the 3rd threshold value T3 thinks that then this pixel belongs to pupil, and pixel corresponding in the human eye area is labeled as pupil;
Step 532: if there is pupil in the human eye area, think that then human eye is in the state of opening, otherwise think that human eye is in closure state.
Preferably, described driver fatigue judge module judges by the following method whether the driver is in fatigue state:
The shared ratio f of human eye closing time in the statistical unit time:
Wherein, t
1Be the special time of a certain setting, t
2Be the time of human eye closure,, think that then the driver is in fatigue state as PERCLOS value f during greater than the 4th threshold value T4, wherein, 65%≤T4≤80%.
Compare with the technology that existing driver fatigue detects, driver fatigue detection device based on video provided by the present invention has following advantage: 1) owing to adopt filter element, the brightness value of pixel that belongs to the people face part in the digital picture that image capturing component is obtained is higher, therefore can extract human face region simply, reduce the amount of calculation of people's face detection algorithm; 2) method of traditional judgement human eye switching is to utilize the ratio of the height and the width of human eye area to realize, because the human eye area of actual extracting is accurate inadequately, therefore had a strong impact on the judgement of human eye open and-shut mode, and the present invention utilizes pupil to judge that the switching of human eye has improved the accuracy that the human eye open and-shut mode is judged effectively.
Description of drawings
Fig. 1 is the structural representation according to the driver fatigue detection device based on video of the present invention;
Fig. 2 is according to the schematic block diagram based on the graphics processing unit in the driver fatigue detection device of video of the present invention;
The specific embodiment
For making your auditor can further understand structure of the present invention, feature and other purposes, now be described in detail as follows in conjunction with appended preferred embodiment, illustrated preferred embodiment only is used to technical scheme of the present invention is described, and non-limiting the present invention.
Comprise according to the driver fatigue detection device 1 based on video of the present invention: at least one infrared illumination element 1 is used for providing infrared illumination to monitoring scene; Camera lens 2 is used to obtain the infrared optics picture signal of monitoring scene; Filter element 3 is used for processings that filter of infrared optics picture signal to the monitoring scene that receives, with people's face and the human eye pupil part in the infrared optics picture signal of giving prominence to monitoring scene; Image capturing component 4 is used for the infrared optics picture signal of optical processing after filtration is converted to data image signal; With graphics processing unit 5, be used for extracting human eye and pupil, and judge whether the driver is in fatigue state from the digital picture of obtaining.Wherein, infrared illumination element 1 is installed in individually and is used to drive the regional interior of area illumination or is used to drive in the zone of area illumination with ground connection being installed in that link to each other towards the camera lens 2 of driving the zone; Image capturing component 4 is positioned at the back of camera lens 2; Before filter element 3 is positioned at camera lens 2, perhaps is positioned at the back of image capturing component 4 or is positioned at camera lens 2 back and before image capturing component 4.Camera lens 2, filter element 3 and image capturing component 4 both can be self-existent, also can removably closely link together.
Preferably, infrared illumination element 1 can removably closely link together with camera lens 2, for example infrared illumination element 1 can be distributed in camera lens 2 around, also can be installed in individually and a certainly can be used to drive in the zone of area illumination, camera lens 2 is driven the zone towards this.As shown in Figure 1, Fig. 1 is the structural representation according to the driver fatigue detection device based on video of the present invention.As seen from Figure 1, infrared illumination element 1 removably closely links together with camera lens 2, camera lens 2 is towards driving the zone, and filter element 3 removably closely is connected the back of camera lens 2, and image capturing component 4 removably closely is connected the back of filter element 3.In addition, when practical application, also can adopt following connected mode between camera lens 2, filter element 3 and the image capturing component 4:
Connected mode one: before filter element 3 removably closely is connected camera lens 2, being used for that monitoring scene is carried out infrared filtering earlier handles, camera lens 2 is used for directly obtaining the infrared optics picture signal of the monitoring scene after the optical element 3 infrared filterings processing after filtration, image capturing component 4 removably closely is connected the back of camera lens 2, is used for the infrared optics picture signal of optical processing after filtration is converted to data image signal.
Connected mode two: before camera lens 2 removably closely is connected image capturing component 4, be used to obtain the infrared optics picture signal of monitoring scene, the infrared optics picture signal that image capturing component 4 is used for the monitoring scene that will obtain is converted to data image signal, filter element 3 removably closely is connected the back of image capturing component 4, is used for the processing that filters of infrared digital image signal to the monitoring scene that obtains.
Wherein, infrared illumination element 1 provides the infrared light of a certain wavelength, and for example wavelength is the infrared light of 850 ± 20nm or 940 ± 20nm, and this infrared light is used for the illumination of monitoring scene.This infrared illumination element 1 can comprise at least one infrared illuminator, and this infrared illuminator can be the infrared illumination lamp.The infrared light wavelength that this infrared illumination element 1 provides is in the optical filtering wave band that filter element 3 provides.Camera lens 2 provides it to filter element 3 after obtaining the infrared optics picture signal of monitoring scene.
The processing that filters of the infrared optics picture signal of the monitoring scene of 3 pairs of receptions of filter element, with people's face and the human eye pupil part in the infrared optics picture signal of outstanding monitoring scene, and the infrared optics picture signal that will filter after handling offers image capturing component 4.According to the present invention, described filter element 3 can comprise the logical filter element of at least one band and/or at least one polarizing filter element.That is to say that filter element 3 can be made of an independent logical filter element of band or an independent polarizing filter element, also can constitute by the logical filter element of a plurality of bands and/or a plurality of polarizing filter elements combination.During enforcement, the infrared wavelength of the optical filtering wave band of filter element 3 is corresponding with the infrared wavelength of the infrared light that infrared illumination element 1 provides.For example, filter element 3 can be made of a logical filter element of band, the logical filter element of this band can be that centre wavelength is in the narrow band pass filter in the 850-950nm scope, and the infrared light wavelength that infrared illumination element 1 provides is being in the infrared band at center with above-mentioned centre wavelength.Preferably, the infrared wavelength of the optical filtering wave band of filter element 3 is 850 ± 20nm or 940 ± 20nm.
The image capturing component 4 infrared optics picture signal of optical processing after filtration is converted to data image signal, and this data image signal is offered graphics processing unit 5.According to the present invention, image capturing component 4 can be that ccd sensor, cmos sensor or other can be converted to optical image signal the device of data image signal, perhaps can be CCD camera, CMOS camera, infrared imaging camera and infrared thermal release electric magazine one or more.For example, described image capturing component 4 is the CCD camera.In this case, when the CCD camera is set, in order to make the image that obtains more stable, some automatic functions of this CCD camera can be closed, as AWB, automatic gain, autoelectrinic shutter etc., and the manual adjustments electronic shutter, the time of exposure of this CCD camera is in the scope of 0.1-2.5ms.
Graphics processing unit 5 extracts human eye and pupil, and judges whether the driver is in fatigue state according to the digital picture that receives.Fig. 2 is according to the schematic block diagram based on the graphics processing unit in the driver fatigue detection device of video of the present invention, and as shown in Figure 2, this graphics processing unit 5 comprises:
Candidate face acquisition module 51 is used for extracting the candidate face zone from the digital picture of obtaining;
Human eye acquisition module 52 is used to utilize eye detection method, detects human eye area from the candidate face zone that receives;
Whether human eye opens and closes determination module 53, be used for according to existing pupil to determine the switching of human eye in the human eye area;
Driver fatigue judge module 54 is used for the switching according to human eye, utilizes the PERCLOS value to judge whether the driver is in fatigue state.
According to the present invention, candidate face acquisition module 51 obtains the candidate face zone by following steps:
Step 511: utilize threshold segmentation method to obtain people's face pixel of candidate.Because the brightness value of the pixel of people face part is than higher in the digital picture that image capturing component 4 provides, therefore cut apart and to extract the people face pixel of the higher pixel of brightness value by the digital picture of obtaining being carried out threshold value, and obtain corresponding people's face threshold value split image as the candidate.Wherein, the method that threshold value is cut apart is: suppose that (x, brightness value y) are that (x, y), (x, brightness value y) are I to pixel to I in people's face threshold value split image of digital picture to pixel in the digital picture
1(x, y) (I wherein
1Initial value be I
1=I), if I (x, y) 〉=first threshold T1, then this pixel of labelling belongs to people's face pixel (brightness value I that promptly keeps this pixel in people's face threshold value split image of candidate
1(x, y) constant) (is about to the brightness value I of this pixel in people's face threshold value split image otherwise this pixel of labelling is people's face pixel of non-candidate
1(x y) gives 0).Wherein, 100≤T1≤200 and T1 are integer.During enforcement, for fear of with possible people's face pixel filtering, T1 can be got a little bit smaller, T1=120 for example.
Step 512: the people's face pixel with the candidate in people's face threshold value split image is a template, utilizes method for detecting human face to obtain candidate's human face region on this template.Wherein, described method for detecting human face can adopt method for marking connected region, also can request for utilization number be the method for detecting human face that is provided in 200910077430.2 the patent application (applicant by the application is submitted to), can also adopt comparatively popular method for detecting human face at present based on AdaBoost.When using existing method for detecting human face to obtain candidate's human face region, directly in people's face threshold value split image, extract candidate's human face region, owing to be partitioned into people's face pixel of candidate in people's face threshold value split image, therefore reduced of the influence of complicated scene greatly, also relatively reduced the amount of calculation that people's face detects people's face detection accuracy.
In the enforcement,, can adopt simple method for marking connected region to obtain candidate's human face region in order to reduce the amount of calculation that people's face detects.At first the people's face pixel (being the pixel of the brightness value non-0 in people's face threshold value split image) with the candidate in people's face threshold value split image is an impact point, and all impact points in people's face threshold value split image are carried out connected component labeling.Then, the number of people's face pixel of candidate in the connected region of statistics labelling, if this number＞second threshold value T2, then this connected region of labelling is candidate's a human face region, otherwise people's face pixel that institute in this connected region is labeled as non-candidate a little (is about to the interior brightness value I that is had a few of this connected region in people's face threshold value split image
1(x y) gives 0).Wherein, 300≤T2≤600 and T2 are integer.
Method for marking connected region can be realized by four connected domain methods or eight connected domain methods.Four methods that connect the connection labelling of/eight connected domains are: at first, people's face threshold value split image is implemented to line by line scan, find first impact point in a unmarked zone, the people face pixel of this first impact point for not being labeled, this point of labelling; Four companies 8 company's territory points and the labelling of checking this gauge point satisfy connectivity platform and impact point that be not labeled as yet, simultaneously newly-increased gauge point are noted the seed points as " region growing ".In follow-up labeling process, constantly from the array that writes down seed points, take out a seed points, implement above-mentioned operation, so circulation, when the array of record seed points was sky, a connected component labeling finished.Follow the next unlabelled zone of labelling, all connected regions all are labeled in people's face threshold value split image again.
Human eye acquisition module 52 is a template with the candidate's of candidate face acquisition module 51 outputs human face region, utilizes eye detection method to obtain human eye area on this template.Wherein, eye detection method can utilize existing human eye detection algorithm to realize, for example adopting application number is the human eye detection algorithm that is provided in the patent application (applicant by the application is submitted to) of 200910077436.X, also can adopt document " Zhong Wei; Liu Zhiming, Zhou Jiliu. the pinpoint research of eyes during people's face detects, computer engineering and application; 2004,36:73-76 " in the method that provided.Wherein, the human eye detection algorithm that is provided in the patent application of application number for 200910077436.X is: the first step: determine the human eye region of search, the position of human eye of in next frame, predicting, according to the prediction position of human eye point about respectively enlarge two eye distances from fault value 4, up enlarge two eye distances from fault value 5, down enlarge two eye distances from fault value 6, to form a rectangular area, choose this rectangular area as the human eye region of search; Second step, determine true human eye area, with the FRST algorithm FRST algorithm computation is carried out in described human eye region of search, to obtain corresponding map image, and obtain the max pixel value of this map image, then with the fault value 7 of max pixel value as cutting apart the fault value, described human eye region of search is cut apart, to obtain the bianry image zone, judge in this bianry image zone whether have a pair of vertical area again, if exist, think that then the following zone of vertical area is true human eye area, the upper zone of vertical area is an eyebrow, if do not exist, thinks that then this vertical area is true human eye area.Wherein, the concrete numerical value of the fault value 4-fault in said method value 7 can application reference number be the patent application of 200910077436.X.
Utilize the FRST algorithm, each connected region that has the eyes hole location of symmetry feature in the calculated candidate human face region, be aided with ellipse fitting again to obtain five FRST characteristic parameters of above-mentioned symmetry feature, and the geometric distribution that whether meets eyes and nostril according to described five FRST characteristic parameters judgement concerns, if meet, then write down the result of human eye detection; If do not meet, think that then this zone is false areas and filtering, wherein, described five FRST characteristic parameters comprise: the upper end position of the left position of hole, the right end position of hole, hole, the lower end position of hole and the center of hole.
According to the present invention, human eye opens and closes determination module 53 and judges that by following steps human eye opens and closes:
Step 531: in human eye area, extract pupil.At first the brightness value corresponding to the pixel in the human eye area (human eye acquisition module 52 obtains) in the people's face threshold value split image (step 511 obtains) is kept, with other the brightness value I of pixel
1(x y) gives 0; Then this people's face threshold value split image is carried out threshold value and cut apart, if I
1(x, y) 〉=the 3rd threshold value T3 thinks that then this pixel belongs to pupil, and pixel corresponding in the human eye area is labeled as pupil.Wherein, the 3rd threshold value T3 can obtain by the method for calculating the maximum between-cluster variance of pixel in the human eye area.The method of maximum between-cluster variance can list of references " OTSC N.Athreshold selection method from gray-level histogram[J] .IEEE Trans, 1979, SMC-9:62-69 ".
Step 532: if there is pupil in the human eye area, think that then human eye is in the state of opening, otherwise think that human eye is in closure state.
Driver fatigue judge module 54 utilizes the PERCLOS value to judge whether the driver is in fatigue state according to the state of the human eye switching of human eye switching determination module 53 outputs.Utilize the PERCLOS value judge that the method for driver fatigue state can list of references " D F Dinges; R Grace.PERCLOS:A Valid Psycho physiological Measure of Alertnessas Assessed by Psychomotor Vigilance[R] .Report No FHWA2MCRT2982OO6.Federal Highway Administration.Office of MotorCarriers; 1998 ", perhaps list of references " Laurence T, Nick M.Review of fatiguedetection and prediction technologies.http: //www.nrtc.gov.au.2000-09 ".As preferred embodiment a kind of, the applicant adopts the shared ratio f of human eye closing time in the following formula statistical unit time with reference to the principle idea of the method for putting down in writing in above-mentioned two pieces of lists of references:
Wherein, t
1Be the special time of a certain setting, t
2Be the time of human eye closure, f is the PERCLOS value.As PERCLOS value f during, think that then the driver is in fatigue state greater than the 4th threshold value T4.Wherein, 65%≤T4≤80%, t
2Open and close determination module 53 statistics at time t by human eye
1In belong to the human eye closure state accumulated time obtain.
Be scene in one embodiment of the invention with the night, the infrared light wavelength 850nm that infrared illumination element 1 provides, the optical filtering wave band 850nm of filter element 3, first threshold T1=120, the second threshold value T2=400, the 4th threshold value T4=75%, t
1=5s, and calculate the 3rd threshold value T3, t
2And f, and by judging whether f judges greater than the 4th threshold value T4 whether the driver is in fatigue state.
Compare with the technology that existing driver fatigue detects, driver fatigue detection device based on video provided by the present invention has following advantage: 1) owing to adopt filter element, the brightness value of pixel that belongs to the people face part in the digital picture that image capturing component is obtained is higher, therefore can extract human face region simply, reduce the amount of calculation of people's face detection algorithm; 2) method of traditional judgement human eye switching is to utilize the ratio of the height and the width of human eye area to realize, because the human eye area of actual extracting is accurate inadequately, therefore had a strong impact on the judgement of human eye open and-shut mode, and the present invention utilizes pupil to judge that the switching of human eye has improved the accuracy that the human eye open and-shut mode is judged effectively.
What need statement is that the foregoing invention content and the specific embodiment are intended to prove the practical application of technical scheme provided by the present invention, should not be construed as the qualification to protection domain of the present invention.Those skilled in the art are in spirit of the present invention and principle, when doing various modifications, being equal to and replacing or improve.Protection scope of the present invention is as the criterion with appended claims.
Claims (11)
1. the driver fatigue detection device based on video is characterized in that, described driver fatigue detection device comprises:
At least one infrared illumination element is used for providing infrared illumination to monitoring scene;
Camera lens is used to obtain the infrared optics picture signal of monitoring scene;
Filter element is used for processings that filter of infrared optics picture signal to the monitoring scene that receives, with people's face and the human eye pupil part in the infrared optics picture signal of giving prominence to monitoring scene;
Image capturing component, the back that it is positioned at described camera lens is used for the infrared optics picture signal of optical processing after filtration is converted to data image signal; With
Graphics processing unit is used for extracting human eye and pupil from the digital picture of obtaining, and judges whether the driver is in fatigue state;
Wherein, described infrared illumination element is installed in individually and is used to drive the regional interior of area illumination or is used to drive in the zone of area illumination with ground connection being installed in that link to each other towards the described camera lens of driving the zone; Before described filter element is positioned at described camera lens, perhaps is positioned at the back of image capturing component or is positioned at described camera lens back and before described image capturing component.
2. driver fatigue detection device according to claim 1, it is characterized in that, described infrared illumination element and described camera lens removably closely link together, and described camera lens, described filter element and described image capturing component removably closely link together each other.
3. driver fatigue detection device according to claim 1 is characterized in that, described infrared illumination element comprises at least one infrared illuminator.
4. driver fatigue detection device according to claim 1 is characterized in that, described filter element comprises the logical filter element of at least one band and/or at least one polarizing filter element.
5. driver fatigue detection device according to claim 1 is characterized in that, described image capturing component be ccd sensor, cmos sensor, CCD camera, CMOS camera, infrared imaging camera or infrared thermal release electric magazine one or more.
6. driver fatigue detection device according to claim 1 is characterized in that, described graphics processing unit comprises:
The candidate face acquisition module is used for extracting the candidate face zone from the digital picture of obtaining;
The human eye acquisition module is used to utilize eye detection method, detects human eye area from the candidate face zone that receives;
Whether human eye opens and closes determination module, be used for according to existing pupil to determine the switching of human eye in the human eye area; With
The driver fatigue judge module is used for the switching according to human eye, utilizes the PERCLOS value to judge whether the driver is in fatigue state.
7. driver fatigue detection device according to claim 6 is characterized in that, described candidate face acquisition module obtains the candidate face zone by following steps:
Step 511: go out the people face pixel of the higher pixel of brightness value by the digital picture of obtaining being carried out the threshold value segmented extraction, and obtain corresponding people's face threshold value split image as the candidate;
Step 512: the people's face pixel with the candidate in people's face threshold value split image is a template, utilizes method for detecting human face to obtain the candidate face zone on this template.
8. driver fatigue detection device according to claim 7 is characterized in that, threshold value described in the step 511 is cut apart realization by the following method:
Suppose that (x, brightness value y) are that (x, y), (x, brightness value y) are I to pixel to I in people's face threshold value split image of digital picture to pixel in the digital picture
1(x, y), I wherein
1Initial value be I, (x, y) 〉=first threshold T1, then this pixel of labelling belongs to people's face pixel of candidate, keeps the brightness value I of this pixel in people's face threshold value split image if I
1(x, y) constant; Otherwise this pixel of labelling is people's face pixel of non-candidate, with the brightness value I of this pixel in people's face threshold value split image
1(x y) gives 0, and wherein, 100≤T1≤200 and T1 are integer.
9. driver fatigue detection device according to claim 7 is characterized in that, method for detecting human face described in the step 512 is a method for marking connected region, and described method for marking connected region comprises:
At first the people's face pixel with the candidate in people's face threshold value split image is a target, and all impact points in people's face threshold value split image are carried out connected component labeling;
Add up the number of people's face pixel of candidate in the connected region of labelling then, if this number is greater than the second threshold value T2, then this connected region of labelling is candidate's a human face region, otherwise with in this connected region be labeled as people's face pixel of non-candidate a little, with the brightness value I that is had a few in this connected region
1(x y) gives 0;
Wherein, 300≤T2≤600 and T2 are integer.
10. driver fatigue detection device according to claim 6 is characterized in that, described human eye opens and closes determination module and judges that by following steps human eye opens and closes:
Step 531: in human eye area, extract pupil, at first will keep corresponding to the brightness value of the pixel in the human eye area in people's face threshold value split image, give 0 with other the brightness value of pixel; Then this people's face threshold value split image is carried out threshold value and cut apart, if I
1(x, y) 〉=the 3rd threshold value T3 thinks that then this pixel belongs to pupil, and pixel corresponding in the human eye area is labeled as pupil;
Step 532: if there is pupil in the human eye area, think that then human eye is in the state of opening, otherwise think that human eye is in closure state.
11. driver fatigue detection device according to claim 6 is characterized in that, described driver fatigue judge module judges by the following method whether the driver is in fatigue state:
The shared ratio f of human eye closing time in the statistical unit time:
Wherein, t
1Be the special time of a certain setting, t
2Be the time of human eye closure,, think that then the driver is in fatigue state as PERCLOS value f during greater than the 4th threshold value T4, wherein, 65%≤T4≤80%.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010119431 CN101803928A (en) | 2010-03-05 | 2010-03-05 | Video-based driver fatigue detection device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010119431 CN101803928A (en) | 2010-03-05 | 2010-03-05 | Video-based driver fatigue detection device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101803928A true CN101803928A (en) | 2010-08-18 |
Family
ID=42606005
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010119431 Pending CN101803928A (en) | 2010-03-05 | 2010-03-05 | Video-based driver fatigue detection device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101803928A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102324166A (en) * | 2011-09-19 | 2012-01-18 | 深圳市汉华安道科技有限责任公司 | Fatigue driving detection method and device |
| CN102657513A (en) * | 2012-04-10 | 2012-09-12 | 中国航空无线电电子研究所 | Method for detecting pupil position from human eye infrared image |
| CN103077370A (en) * | 2011-08-25 | 2013-05-01 | 株式会社电装 | Face image detection apparatus |
| CN103211605A (en) * | 2013-05-14 | 2013-07-24 | 重庆大学 | Psychological testing system and method |
| CN103347446A (en) * | 2010-12-10 | 2013-10-09 | Tk控股公司 | System for monitoring a vehicle driver |
| CN104013414A (en) * | 2014-04-30 | 2014-09-03 | 南京车锐信息科技有限公司 | Driver fatigue detecting system based on smart mobile phone |
| CN104382607A (en) * | 2014-11-26 | 2015-03-04 | 重庆科技学院 | Fatigue detecting method based on driver video images in vehicle working condition |
| CN105361851A (en) * | 2014-08-29 | 2016-03-02 | 阿尔卑斯电气株式会社 | Sight line detection device |
| CN105718844A (en) * | 2014-10-13 | 2016-06-29 | 由田新技股份有限公司 | blink detection method and device |
| CN107847130A (en) * | 2015-07-17 | 2018-03-27 | 索尼公司 | Eyeball observation device, glasses type terminal, method for detecting sight line and program |
| CN109496309A (en) * | 2018-08-07 | 2019-03-19 | 深圳市汇顶科技股份有限公司 | Detection method, device and the equipment of fatigue state |
| CN109767602A (en) * | 2019-03-14 | 2019-05-17 | 钧捷智能（深圳）有限公司 | A kind of round-the-clock driver fatigue monitor system camera |
| CN111493897A (en) * | 2020-06-01 | 2020-08-07 | 南京国科医工科技发展有限公司 | Intelligent health monitoring and promoting system for automobile driver |
| CN113302912A (en) * | 2018-12-19 | 2021-08-24 | 法雷奥舒适驾驶助手公司 | Image capture device for monitoring a driver and related system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1830389A (en) * | 2006-04-21 | 2006-09-13 | 太原理工大学 | Device for monitoring fatigue driving state and its method |
| CN101540090A (en) * | 2009-04-14 | 2009-09-23 | 华南理工大学 | Driver fatigue monitoring device based on multivariate information fusion and monitoring method thereof |
| CN101599207A (en) * | 2009-05-06 | 2009-12-09 | 深圳市汉华安道科技有限责任公司 | A kind of fatigue driving detection device and automobile |
-
2010
- 2010-03-05 CN CN 201010119431 patent/CN101803928A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1830389A (en) * | 2006-04-21 | 2006-09-13 | 太原理工大学 | Device for monitoring fatigue driving state and its method |
| CN101540090A (en) * | 2009-04-14 | 2009-09-23 | 华南理工大学 | Driver fatigue monitoring device based on multivariate information fusion and monitoring method thereof |
| CN101599207A (en) * | 2009-05-06 | 2009-12-09 | 深圳市汉华安道科技有限责任公司 | A kind of fatigue driving detection device and automobile |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103347446A (en) * | 2010-12-10 | 2013-10-09 | Tk控股公司 | System for monitoring a vehicle driver |
| CN103347446B (en) * | 2010-12-10 | 2016-10-26 | Tk控股公司 | For monitoring the system of vehicle driver |
| CN103077370A (en) * | 2011-08-25 | 2013-05-01 | 株式会社电装 | Face image detection apparatus |
| CN103077370B (en) * | 2011-08-25 | 2015-09-30 | 株式会社电装 | Face image detecting apparatus |
| CN102324166A (en) * | 2011-09-19 | 2012-01-18 | 深圳市汉华安道科技有限责任公司 | Fatigue driving detection method and device |
| CN102657513A (en) * | 2012-04-10 | 2012-09-12 | 中国航空无线电电子研究所 | Method for detecting pupil position from human eye infrared image |
| CN103211605A (en) * | 2013-05-14 | 2013-07-24 | 重庆大学 | Psychological testing system and method |
| CN103211605B (en) * | 2013-05-14 | 2015-02-18 | 重庆大学 | Psychological testing system and method |
| CN104013414A (en) * | 2014-04-30 | 2014-09-03 | 南京车锐信息科技有限公司 | Driver fatigue detecting system based on smart mobile phone |
| CN104013414B (en) * | 2014-04-30 | 2015-12-30 | 深圳佑驾创新科技有限公司 | A kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone |
| CN105361851A (en) * | 2014-08-29 | 2016-03-02 | 阿尔卑斯电气株式会社 | Sight line detection device |
| CN105718844A (en) * | 2014-10-13 | 2016-06-29 | 由田新技股份有限公司 | blink detection method and device |
| CN105718844B (en) * | 2014-10-13 | 2019-11-26 | 由田新技股份有限公司 | Blink detection method and device |
| CN104382607B (en) * | 2014-11-26 | 2016-08-24 | 重庆科技学院 | Driver's video image fatigue detection method towards real vehicle operating mode |
| CN104382607A (en) * | 2014-11-26 | 2015-03-04 | 重庆科技学院 | Fatigue detecting method based on driver video images in vehicle working condition |
| CN107847130A (en) * | 2015-07-17 | 2018-03-27 | 索尼公司 | Eyeball observation device, glasses type terminal, method for detecting sight line and program |
| CN109496309A (en) * | 2018-08-07 | 2019-03-19 | 深圳市汇顶科技股份有限公司 | Detection method, device and the equipment of fatigue state |
| WO2020029093A1 (en) * | 2018-08-07 | 2020-02-13 | 深圳市汇顶科技股份有限公司 | Fatigue state detection method, apparatus, and device |
| US10969864B2 (en) | 2018-08-07 | 2021-04-06 | Shenzhen GOODIX Technology Co., Ltd. | Method, apparatus and device for detecting fatigue state |
| CN113302912A (en) * | 2018-12-19 | 2021-08-24 | 法雷奥舒适驾驶助手公司 | Image capture device for monitoring a driver and related system |
| US11845335B2 (en) | 2018-12-19 | 2023-12-19 | Valeo Comfort And Driving Assistance | Image capture device and associated system for monitoring a driver |
| CN109767602A (en) * | 2019-03-14 | 2019-05-17 | 钧捷智能（深圳）有限公司 | A kind of round-the-clock driver fatigue monitor system camera |
| CN111493897A (en) * | 2020-06-01 | 2020-08-07 | 南京国科医工科技发展有限公司 | Intelligent health monitoring and promoting system for automobile driver |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101803928A (en) | Video-based driver fatigue detection device | |
| CN101593425B (en) | Machine vision based fatigue driving monitoring method and system | |
| CN103646250B (en) | Pedestrian monitoring method and device based on distance image head and shoulder features | |
| CN106446926A (en) | Transformer station worker helmet wear detection method based on video analysis | |
| CN102867417B (en) | Taxi anti-forgery system and taxi anti-forgery method | |
| CN106295551A (en) | A kind of personal security cap wear condition real-time detection method based on video analysis | |
| CN108376246A (en) | A kind of identification of plurality of human faces and tracking system and method | |
| CN104091176A (en) | Technology for applying figure and head portrait comparison to videos | |
| Almagambetov et al. | Mobile standards-based traffic light detection in assistive devices for individuals with color-vision deficiency | |
| CN102164270A (en) | Intelligent video monitoring method and system capable of exploring abnormal events | |
| CN105243386A (en) | Face living judgment method and system | |
| CN106331605B (en) | A kind of floods detection system and method based on video | |
| CN110866479A (en) | Method, device and system for detecting that motorcycle driver does not wear helmet | |
| CN109190475A (en) | A kind of recognition of face network and pedestrian identify network cooperating training method again | |
| CN102915433A (en) | Character combination-based license plate positioning and identifying method | |
| CN107948585A (en) | Video recording labeling method, device and computer-readable recording medium | |
| CN102193918A (en) | Video retrieval method and device | |
| KR102434154B1 (en) | Method for tracking multi target in traffic image-monitoring-system | |
| CN108090459A (en) | A kind of road traffic sign detection recognition methods suitable for vehicle-mounted vision system | |
| CN111401310B (en) | Kitchen sanitation safety supervision and management method based on artificial intelligence | |
| CN101178769B (en) | Health protecting equipment and realization method thereof | |
| CN104463232A (en) | Density crowd counting method based on HOG characteristic and color histogram characteristic | |
| CN104850841A (en) | Elder abnormal behavior monitoring method with combination of RFID and video identification | |
| CN103390151A (en) | Face detection method and device | |
| CN112287823A (en) | Facial mask identification method based on video monitoring |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20100818 |