CN107016336A - The facial characteristics point location detected for fatigue driving is corrected errors the method and device of identification - Google Patents
The facial characteristics point location detected for fatigue driving is corrected errors the method and device of identification Download PDFInfo
- Publication number
- CN107016336A CN107016336A CN201710092629.7A CN201710092629A CN107016336A CN 107016336 A CN107016336 A CN 107016336A CN 201710092629 A CN201710092629 A CN 201710092629A CN 107016336 A CN107016336 A CN 107016336A
- Authority
- CN
- China
- Prior art keywords
- face
- face shape
- image frame
- facial feature
- feature point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000001815 facial effect Effects 0.000 title claims abstract description 122
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000001514 detection method Methods 0.000 claims abstract description 77
- 230000004807 localization Effects 0.000 claims description 48
- 208000029152 Small face Diseases 0.000 claims description 22
- 238000012549 training Methods 0.000 claims description 14
- 238000010606 normalization Methods 0.000 claims description 4
- 230000001960 triggered effect Effects 0.000 claims description 4
- 238000007493 shaping process Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 4
- 230000005484 gravity Effects 0.000 description 2
- 241001282135 Poromitra oscitans Species 0.000 description 1
- 206010048232 Yawning Diseases 0.000 description 1
- 230000004397 blinking Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The present invention proposes the method and device for identification of being corrected errors for the facial characteristics point location of fatigue driving detection, and this method includes：Via video acquisition device collect predetermined time period in the face-image comprising driver n picture frame, and therewith be directed to each picture frame execution face feature point positioning action with determine respectively with an associated n face shape S being made up of multiple face feature points in the n picture frame1, S2..., Sn；) it is based on the n face shape S1, S2..., SnDetermine benchmark face shape；The picture frame for obtaining the current face image comprising driver in real time is used as target image frame, and the face shape associated with the target image frame is determined, the face shape being associated with the target image frame is compared with the benchmark face shape to determine whether the face shape associated with the target image frame be effective therewith.The method disclosed in the present and device can accurately differentiate whether current facial characteristics point location is correct.
Description
Technical Field
The invention relates to a method and a device for positioning and identifying facial feature points, in particular to a method and a device for positioning and identifying facial feature points for fatigue driving detection.
Background
Currently, with the increasing development and popularization of vehicles, it is becoming more and more important to detect fatigue phenomena (e.g., blinking, yawning, etc.) occurring in driving of a driver in real time to avoid occurrence of an accident.
In prior art solutions, fatigue driving detection is typically implemented by: (1) acquiring an original video image of a driver in real time through a vehicle-mounted camera device; (2) extracting each image frame containing a face image of a driver in the original video image and taking the image frame as a target image frame; (3) a facial feature point location operation is performed on the target image frame to determine facial feature points, whereupon a target sub-image is truncated from the target image frame based on the resulting facial feature points and analyzed to determine whether the driver is currently driving fatigue.
However, the above prior art solutions have the following problems: when the vehicle is affected by uneven light, strong light, weak light, and a change in the posture of the driver's head during driving, the recognition of the facial feature points becomes inaccurate, resulting in inaccuracy (i.e., false alarm) of the detection result for fatigue driving.
Thus, there is a need for: provided are a method and a device for identifying the position of facial feature points for fatigue driving detection, which can accurately judge whether the position of the facial feature points is correct.
Disclosure of Invention
In order to solve the problems existing in the prior art, the invention provides a method and a device for identifying the positioning correctness and the inaccuracy of facial feature points for fatigue driving detection, which can accurately judge whether the positioning of the facial feature points is correct.
The purpose of the invention is realized by the following technical scheme:
a method for facial feature point localization true-false recognition for fatigue driving detection, comprising the steps of:
(A1) collecting n image frames containing facial images of a driver over a predetermined period of time via a video capture device, and then performing a facial feature point location operation for each image frame to determine n facial shapes S of a plurality of facial feature points respectively associated with one of the n image frames1，S2，…，SnWherein n is a positive integer;
(A2) based on the n face shapes S1，S2，…，SnDetermining a reference face shape;
(A3) acquiring an image frame containing a current face image of a driver as a target image frame in real time, performing a facial feature point positioning operation for the target image frame to determine a face shape associated with the target image frame, subsequently comparing the face shape associated with the target image frame with the reference face shape to determine whether the face shape associated with the target image frame is valid, and performing a subsequent fatigue driving detection operation only if the face shape associated with the target image frame is valid.
In the aspect disclosed above, illustratively, the n face shapes S1，S2，…，SnAre each made up of p coordinate points, i.e. Si＝{(xi1，yi1)，(xi2，yi2)，...，(xip，yip)}。
In the above disclosed solution, exemplarily, the step (a2) further includes: obtaining the n-plane shape S1，S2，…，SnThen, the area a of the circumscribed rectangle of each face shape is calculated1，a2，…，an。
In the above disclosed solution, exemplarily, the step (a2) further includes: by shaping each face to shape SiGeometric center (x) ofc，yc) Shift to coordinates (0, 0) and S for each face shapeiPerforming a normalization operation such thatWhereinIs to SiThe normalized result is executed.
In the above disclosed solution, exemplarily, the step (a2) further includes: based on the obtainedAnd calculates the average face shape according to the following formula:
wherein,is the average face shape.
In the above disclosed solution, exemplarily, the step (a2) further includes: calculating the n face shapes S1，S2，…，SnThe difference between each of the normalized results of (a) and the average face shape:
wherein,is a normalized resultAnd average facial shapeThe difference of (a).
In the above disclosed solution, exemplarily, the step (a2) further includes: area a of circumscribed rectangle based on each face shape1，a2，…，anThe average area is calculated as follows:
wherein,is the average area.
In the above disclosed solution, exemplarily, the step (a2) further includes: defining a symbolic function associated with the occurrence of a "large face" or "small face" phenomenon based on the following formula:
wherein,is the average area.
In the above disclosed solution, exemplarily, the step (a2) further includes: calculate and countAnd a negative first threshold T1 and a positive second threshold T2 are determined based on the statistical results, so that the numerical region between the first threshold T1 and the second threshold T2 constitutes a confidence interval in discriminating that the face shape is valid.
In the above disclosed solution, exemplarily, the step (a3) further includes: (1) shape the average faceAs the reference face shape, and calculating a normalized value of the face shape associated with the target image frame, whereupon a difference between the normalized value and the reference face shape is calculated according to formula (1), and an area of a circumscribed rectangle of the face shape associated with the target image frame is calculated, whereupon a value of a sign function associated with the target image frame is calculated according to formula (2); (2) calculating a product of the obtained difference value and the value of the sign function, and determining that the facial shape associated with the target image frame is valid and a subsequent fatigue driving detection operation is followed if the product is within a confidence interval constituted by the first threshold T1 and the second threshold T2, otherwise terminating the subsequent fatigue driving detection operation for the target image frameAnd (5) continuing the operation.
In the above disclosed solution, exemplarily, the step (a3) further includes: if the product is less than the first threshold T1, it is determined that a "small-face case" has occurred, and if the product is greater than the second threshold T2, it is determined that a "large-face case" has occurred.
In the above disclosed solution, exemplarily, the step (a3) further includes: when it is determined that the "small face condition" has occurred, a predetermined small face processing program is started to recalculate the average face shape to correct the reference face shape.
In the above disclosed solution, exemplarily, the step (a3) further includes: when it is determined that a "large face condition" has occurred, the current detection process is terminated and a new face detection process is triggered accordingly.
The purpose of the invention can also be realized by the following technical scheme:
an apparatus for facial feature point localization true-false recognition for fatigue driving detection, comprising:
a model training unit configured to collect, via a video capture device, n image frames containing facial images of a driver over a predetermined period of time, and then perform a facial feature point localization operation for each image frame to determine n facial shapes S composed of a plurality of facial feature points respectively associated with one of the n image frames1，S2，…，SnWhere n is a positive integer, and based on the n face shapes S1，S2，…，SnDetermining a reference face shape;
a real-time discrimination unit configured to acquire an image frame containing a current face image of a driver as a target image frame in real time, and perform a facial feature point positioning operation for the target image frame to determine a face shape associated with the target image frame, whereupon the face shape associated with the target image frame is compared with the reference face shape to determine whether the face shape associated with the target image frame is valid, and perform a subsequent fatigue driving detection operation only if the face shape associated with the target image frame is valid.
The method and the device for identifying the facial feature point positioning correctness and inaccuracy for detecting the fatigue driving disclosed by the invention have the following advantages that: since whether the shape of the currently acquired face is correct can be detected in real time, the accuracy of fatigue driving detection can be significantly improved, and thus vehicle driving safety can be enhanced.
Drawings
The features and advantages of the present invention will be better understood by those skilled in the art when considered in conjunction with the accompanying drawings, in which:
FIG. 1 is a flow diagram of a method of facial feature point localization misidentification for fatigue driving detection according to an embodiment of the present invention.
Fig. 2 is a schematic configuration diagram of an apparatus for facial feature point localization misrecognition for fatigue driving detection according to an embodiment of the present invention.
Detailed Description
FIG. 1 is a flow diagram of a method of facial feature point localization misidentification for fatigue driving detection according to an embodiment of the present invention. As shown in fig. 1, the method for identifying facial feature point location errors for fatigue driving detection disclosed by the invention comprises the following steps: (A1) n image frames containing a facial image of a driver over a predetermined period of time (e.g., 10 minutes) are collected via a video capture device, and then a facial feature point location operation is performed for each image frame to determine n facial shapes of a plurality of facial feature points respectively associated with one of the n image framesS1，S2，…，SnWherein n is a positive integer; (A2) based on the n face shapes S1，S2，…，SnDetermining a reference face shape; (A3) acquiring an image frame containing a current face image of a driver as a target image frame in real time, performing a facial feature point positioning operation for the target image frame to determine a face shape associated with the target image frame, subsequently comparing the face shape associated with the target image frame with the reference face shape to determine whether the face shape associated with the target image frame is valid, and performing a subsequent fatigue driving detection operation only if the face shape associated with the target image frame is valid. Illustratively, the video capture device is an onboard camera located above the driving seat or any in-vehicle video device capable of capturing images of the driver's head in real time (such as a smartphone, tablet computer, etc. communicating with an electronic control unit in the vehicle over a wired or wireless physical channel), such as, but not limited to, a camera unit based on a CCD sensor or CMOS sensor. .
Illustratively, in the method for facial feature point localization false-positive recognition for fatigue driving detection disclosed in the present invention, the n facial shapes S1，S2，…，SnAre each made up of p coordinate points, i.e. Si＝{(xi1，yi1)，(xi2，yi2)，...，(xip，yip)}。
Illustratively, in the method for facial feature point localization false positive identification for fatigue driving detection disclosed in the present invention, the step (a2) further comprises: obtaining the n-plane shape S1，S2，…，SnThen, the area a of the circumscribed rectangle of each face shape is calculated1，a2，…，an。
Illustratively, in the method for facial feature point localization false positive identification for fatigue driving detection disclosed in the present invention, the step (a2) further comprises: by mixingEach face shape SiGeometric center (x) ofc，yc) Shift to coordinates (0, 0) and S for each face shapeiPerforming a normalization operation such thatWhereinIs to SiThe normalized result is executed.
Exemplarily, xcAnd ycIs determined by the following formula:
illustratively, in the method for facial feature point localization false positive identification for fatigue driving detection disclosed in the present invention, the step (a2) further comprises: based on the obtainedAnd calculates the average face shape according to the following formula:
wherein,is the average face shape.
Illustratively, in the method for facial feature point localization false positive identification for fatigue driving detection disclosed in the present invention, the step (a2) further comprises: calculating the n face shapes S1，S2，…，SnThe difference between each of the normalized results of (a) and the average face shape:
wherein,is a normalized resultAnd average facial shapeThe difference of (a).
Illustratively, in the method for facial feature point localization false positive identification for fatigue driving detection disclosed in the present invention, the step (a2) further comprises: area a of circumscribed rectangle based on each face shape1，a2，…，anThe average area is calculated as follows:
wherein,is the average area.
Illustratively, in the method for facial feature point localization false positive identification for fatigue driving detection disclosed in the present invention, the step (a2) further comprises: defining a sign function associated with the occurrence of a "large face" or "small face" phenomenon (if the normalized value of the area of the circumscribed rectangle of the current face shape differs from the area of the average face shape by less than a certain threshold, then the occurrence of a "small face condition" (i.e., the current face shape is smaller than the actual face shape, e.g., due to excessive head deflection), and vice versa the occurrence of a "large face condition" (i.e., the current face shape is larger than the actual face shape, e.g., due to changes in lighting conditions)) based on the following formula:
wherein,is the average area.
Illustratively, in the method for facial feature point localization false positive identification for fatigue driving detection disclosed in the present invention, the step (a2) further comprises: calculate and countSuch that the numerical area between said first threshold T1 and said second threshold T2 constitutes a confidence interval (e.g. according to experimental data) in which the "facial shape is valid" is judged, on the basis of statistical results, such that the driver is in a normal driving situation in most cases, i.e. the positioning of the facial feature points is accurate, and only in few cases either a "large face situation" or a "small face situation" occurs, i.e. it approximately follows a gaussian distribution, whereby, on the basis of statistical results, a negative first threshold T1 and a positive second threshold T2 are determined, such that the area of the values between said first threshold T1 and said second threshold T2 constitutes a confidence interval in whichAssuming that 90% of the cases are normal, the confidence interval of the statistics may be set to 90%, so as to obtain the threshold T1 where the "small face case" occurs and the threshold T2 where the "large face case" occurs).
Illustratively, in the method for facial feature point localization false positive identification for fatigue driving detection disclosed in the present invention, the step (a3) further comprises: (1) shape the average faceAs the reference face shape, and calculating a normalized value of the face shape associated with the target image frame, whereupon a difference between the normalized value and the reference face shape is calculated according to formula (1), and an area of a circumscribed rectangle of the face shape associated with the target image frame is calculated, whereupon a value of a sign function associated with the target image frame is calculated according to formula (2); (2) calculating a product of the obtained difference value and a value of the sign function, and determining that a facial shape associated with the target image frame is valid and a subsequent fatigue driving detection operation is then performed if the product is within a confidence interval constituted by the first threshold T1 and the second threshold T2, otherwise terminating the subsequent operation for the target image frame.
Illustratively, in the method for facial feature point localization false positive identification for fatigue driving detection disclosed in the present invention, the step (a3) further comprises: if the product is less than the first threshold T1, it is determined that a "small-face case" has occurred, and if the product is greater than the second threshold T2, it is determined that a "large-face case" has occurred.
Illustratively, in the method for facial feature point localization false positive identification for fatigue driving detection disclosed in the present invention, the step (a3) further comprises: when it is determined that the "small face condition" has occurred, a predetermined small face processing program is started to recalculate the average face shape to correct the reference face shape.
Illustratively, in the method for facial feature point localization false positive identification for fatigue driving detection disclosed in the present invention, the step (a3) further comprises: when it is determined that a "big face condition" has occurred, at which point the center of gravity of the face is not already reliable, the current detection process is terminated and a new face detection process is triggered accordingly.
From the above, the method for identifying the facial feature point positioning error disclosed by the invention has the following advantages: since whether the shape of the currently acquired face is correct can be detected in real time, the accuracy of fatigue driving detection can be significantly improved, and thus vehicle driving safety can be enhanced.
The present application also discloses a fatigue driving determination method using the facial feature point localization misidentification method for fatigue driving detection described in any of the above examples.
Fig. 2 is a schematic configuration diagram of an apparatus for facial feature point localization misrecognition for fatigue driving detection according to an embodiment of the present invention. As shown in fig. 2, the facial feature point positioning and error recognition device for fatigue driving detection disclosed by the invention comprises a model training unit 1 and a real-time discrimination unit 2. The model training unit 1 is configured to collect, via a video capturing device, n image frames containing facial images of a driver over a predetermined period of time (e.g., 10 minutes), and then perform a facial feature point localization operation for each image frame to determine n facial shapes S composed of a plurality of facial feature points respectively associated with one of the n image frames1，S2，…，SnWhere n is a positive integer, and based on the n face shapes S1，S2，…，SnA reference face shape is determined. The real-time discrimination unit 2 is configured to acquire an image frame containing a current face image of a driver as a target image frame in real time, and perform a face feature point positioning operation for the target image frame to determine a face shape associated with the target image frame, whereupon the face shape associated with the target image frame is compared with the reference face shape to determine whether the face shape associated with the target image frame is valid, and perform a subsequent fatigue driving detection operation only in a case where the face shape associated with the target image frame is valid. Illustratively, the video capture device is an onboard camera located above the driver seat or any in-vehicle video device capable of capturing images of the driver's head in real time (such as a smart phone communicating with an electronic control unit in the vehicle through a wired or wireless physical channel)Tablet, etc.), such as but not limited to a CCD sensor or CMOS sensor based camera unit.
Illustratively, in the device for facial feature point localization false-positive recognition for fatigue driving detection disclosed in the present invention, the n facial shapes S1，S2，…，SnAre each made up of p coordinate points, i.e. Si＝{(xi1，yi1)，(xi2，yi2)，...，(xip，yip)}。
Exemplarily, in the apparatus for facial feature point localization and false recognition for fatigue driving detection disclosed in the present invention, the model training unit 1 is further configured to obtain the n facial shapes S1，S2，…，SnThen, the area a of the circumscribed rectangle of each face shape is calculated1，a2，…，an。
Exemplarily, in the apparatus for facial feature point localization and false recognition for fatigue driving detection disclosed in the present invention, the model training unit 1 is further configured to correct the false recognition by assigning each face shape SiGeometric center (x) ofc，yc) Shift to coordinates (0, 0) and S for each face shapeiPerforming a normalization operation such thatWhereinIs to SiThe normalized result is executed.
Exemplarily, xcAnd ycIs determined by the following formula:
exemplarily, in the apparatus for facial feature point localization and misrecognition for fatigue driving detection disclosed in the present invention, the model training unit 1 is further configured to perform a training process based on the obtained facial feature pointsAnd calculates the average face shape according to the following formula:
wherein,is the average face shape.
Exemplarily, in the apparatus for facial feature point localization false-positive recognition for fatigue driving detection disclosed in the present invention, the model training unit 1 is further configured to calculate the n facial shapes S1，S2，…，SnThe difference between each of the normalized results of (a) and the average face shape:
wherein,is a normalized resultAnd average facial shapeThe difference of (a).
Exemplarily, in the apparatus for facial feature point localization false-positive recognition for fatigue driving detection disclosed in the present invention, the model training unit 1 is further configured to determine the face shape based on the area a of the circumscribed rectangle of each face shape1，a2，…，anThe average area is calculated as follows:
wherein,is the average area.
Exemplarily, in the apparatus for facial feature point localization false recognition for fatigue driving detection disclosed in the present invention, the model training unit 1 is further configured to define a sign function associated with occurrence of a "large face" or a "small face" phenomenon based on the following formula (if a normalized value of an area of a circumscribed rectangle of a current facial shape differs from an area of the average facial shape by less than a certain threshold, it is interpreted that a "small face case" has occurred (i.e., the current facial shape is smaller than an actual facial shape, e.g., due to excessive head deflection), and otherwise a "large face case" has occurred (i.e., the current facial shape is larger than an actual facial shape, e.g., due to a change in lighting conditions)):
wherein,is the average area.
Exemplarily, in the apparatus for facial feature point localization false recognition for fatigue driving detection disclosed in the present invention, the model training unit 1 is further configured to calculate and calculateStatistics ofSuch that the numerical area between said first threshold T1 and said second threshold T2 constitutes a confidence interval (e.g. according to experimental data) in which the "facial shape is valid" is judged, on the basis of statistical results, such that the driver is in a normal driving situation in most cases, i.e. the positioning of the facial feature points is accurate, and only in few cases either a "large face situation" or a "small face situation" occurs, i.e. it approximately follows a gaussian distribution, whereby, on the basis of statistical results, a negative first threshold T1 and a positive second threshold T2 are determined, such that the area of the values between said first threshold T1 and said second threshold T2 constitutes a confidence interval in whichAssuming that 90% of the cases are normal, the confidence interval of the statistics may be set to 90%, so as to obtain the threshold T1 where the "small face case" occurs and the threshold T2 where the "large face case" occurs).
Exemplarily, in the apparatus for facial feature point localization and false recognition for fatigue driving detection disclosed in the present invention, the real-time discriminating unit 2 is further configured to perform the following operations: (1) shape the average faceAs the reference face shape, and calculating a normalized value of the face shape associated with the target image frame, whereupon a difference between the normalized value and the reference face shape is calculated according to formula (1), and an area of a circumscribed rectangle of the face shape associated with the target image frame is calculated, whereupon a value of a sign function associated with the target image frame is calculated according to formula (2); (2) calculating a product of the obtained difference value and a value of the sign function, and determining that a facial shape associated with the target image frame is valid and a subsequent fatigue driving detection operation is then performed if the product is within a confidence interval constituted by the first threshold T1 and the second threshold T2, otherwise terminating the subsequent operation for the target image frame.
Exemplarily, in the apparatus for facial feature point localization and false recognition for fatigue driving detection disclosed in the present invention, the real-time discriminating unit 2 is further configured to perform the following operations: if the product is less than the first threshold T1, it is determined that a "small-face case" has occurred, and if the product is greater than the second threshold T2, it is determined that a "large-face case" has occurred.
Exemplarily, in the apparatus for facial feature point localization and false recognition for fatigue driving detection disclosed in the present invention, the real-time discriminating unit 2 is further configured to: when it is determined that the "small face condition" has occurred, a predetermined small face processing program is started to recalculate the average face shape to correct the reference face shape.
Exemplarily, in the apparatus for facial feature point localization and false recognition for fatigue driving detection disclosed in the present invention, the real-time discriminating unit 2 is further configured to: when it is determined that a "big face condition" has occurred, at which point the center of gravity of the face is not already reliable, the current detection process is terminated and a new face detection process is triggered accordingly.
The model training unit 1 and the real-time discrimination unit 2 may both be located in a central controller of a vehicle (e.g., an electric vehicle) or each be located in any other type of separate or integrated controller, such as, but not limited to, an Electronic Control Unit (ECU), a video signal processor, a data processing unit, etc., and the model training unit 1 and the real-time discrimination unit 2 may be implemented as any form of entity or program, such as, but not limited to, software, firmware, or application specific integrated circuits, etc.
From the above, the facial feature point positioning and error identifying device for fatigue driving detection disclosed by the invention has the following advantages: since whether the shape of the currently acquired face is correct can be detected in real time, the accuracy of fatigue driving detection can be significantly improved, and thus vehicle driving safety can be enhanced.
In addition, the invention also discloses a vehicle comprising the facial feature point positioning error recognition device.
In addition, the invention also discloses a fatigue driving determination system which can comprise any one of the facial feature point positioning error recognition devices described above.
Although the present invention has been described in connection with the preferred embodiments, its mode of implementation is not limited to the embodiments described above. It should be appreciated that: various changes and modifications can be made by one skilled in the art without departing from the spirit and scope of the invention.
Claims (17)
1. A method for facial feature point localization true-false recognition for fatigue driving detection, comprising the steps of:
(A1) collecting n image frames containing facial images of a driver over a predetermined period of time via a video capture device, and then performing a facial feature point location operation for each image frame to determine n facial shapes S of a plurality of facial feature points respectively associated with one of the n image frames1，S2，…，SnWherein n is a positive integer;
(A2) base ofAt the n face shapes S1，S2，…，SnDetermining a reference face shape;
(A3) acquiring an image frame containing a current face image of a driver as a target image frame in real time, performing a facial feature point positioning operation for the target image frame to determine a face shape associated with the target image frame, subsequently comparing the face shape associated with the target image frame with the reference face shape to determine whether the face shape associated with the target image frame is valid, and performing a subsequent fatigue driving detection operation only if the face shape associated with the target image frame is valid.
2. The method of claim 1, wherein the n facial feature points are located in a correct and incorrect manner, and wherein the n facial shapes S are determined based on the determined location of the facial feature points1，S2，…，SnAre each made up of p coordinate points, i.e. Si＝{(xi1，yi1)，(xi2，yi2)，...，(xip，yip)}。
3. The method for fatigue driving detection of facial feature point localization misidentification of claim 2, wherein the step (a2) further comprises: obtaining the n-plane shape S1，S2，…，SnThen, the area a of the circumscribed rectangle of each face shape is calculated1，a2，…，an。
4. The method for fatigue driving detection of facial feature point localization misidentification of claim 3, wherein the step (A2) further comprises: by shaping each face to shape SiGeometric center (x) ofc，yc) Shift to coordinates (0, 0) and S for each face shapeiPerforming a normalization operation such thatWhereinIs to SiThe normalized result is executed.
5. The method for fatigue driving detection of facial feature point localization misidentification of claim 4, wherein the step (A2) further comprises: based on the obtainedAnd calculates the average face shape according to the following formula:
wherein,is the average face shape.
6. The method for fatigue driving detection of facial feature point localization misidentification of claim 5, wherein the step (A2) further comprises: calculating the n face shapes S1，S2，…，SnThe difference between each of the normalized results of (a) and the average face shape:
wherein,is a normalized resultAnd average facial shapeThe difference of (a).
7. The method for fatigue driving detection of facial feature point localization misidentification of claim 6, wherein the step (A2) further comprises: area a of circumscribed rectangle based on each face shape1，a2，…，anThe average area is calculated as follows:
wherein,is the average area.
8. The method for fatigue driving detection of facial feature point localization misidentification of claim 7, wherein the step (A2) further comprises: defining a symbolic function associated with the occurrence of a "large face" or "small face" phenomenon based on the following formula:
wherein,is the average area.
9. The method for fatigue driving detection of facial feature point localization misidentification of claim 8, wherein the step (a2) further comprises: calculate and countAnd a negative first threshold T1 and a positive second threshold T2 are determined based on the statistical results, so that the numerical region between the first threshold T1 and the second threshold T2 constitutes a confidence interval in discriminating that the face shape is valid.
10. The method for fatigue driving detection of facial feature point localization misidentification of claim 9, wherein the step (a3) further comprises: (1) shape the average faceAs the reference face shape, and calculating a normalized value of the face shape associated with the target image frame, whereupon a difference between the normalized value and the reference face shape is calculated according to formula (1), and an area of a circumscribed rectangle of the face shape associated with the target image frame is calculated, whereupon a value of a sign function associated with the target image frame is calculated according to formula (2); (2) calculating a product of the obtained difference value and a value of the sign function, and determining that a facial shape associated with the target image frame is valid and a subsequent fatigue driving detection operation is then performed if the product is within a confidence interval constituted by the first threshold T1 and the second threshold T2, otherwise terminating the subsequent operation for the target image frame.
11. The method for fatigue driving detection of facial feature point localization misidentification of claim 10, wherein the step (a3) further comprises: if the product is less than the first threshold T1, it is determined that a "small-face case" has occurred, and if the product is greater than the second threshold T2, it is determined that a "large-face case" has occurred.
12. The method for fatigue driving detection of facial feature point localization misidentification of claim 11, wherein the step (a3) further comprises: when it is determined that the "small face condition" has occurred, a predetermined small face processing program is started to recalculate the average face shape to correct the reference face shape.
13. The method for fatigue driving detection of facial feature point localization misidentification of claim 12, wherein the step (a3) further comprises: when it is determined that a "large face condition" has occurred, the current detection process is terminated and a new face detection process is triggered accordingly.
14. An apparatus for facial feature point localization true-false recognition for fatigue driving detection, comprising:
a model training unit configured to collect, via a video capture device, n image frames containing facial images of a driver over a predetermined period of time, and then perform a facial feature point localization operation for each image frame to determine n facial shapes S composed of a plurality of facial feature points respectively associated with one of the n image frames1，S2，…，SnWhere n is a positive integer, and based on the n face shapes S1，S2，…，SnDetermining a reference face shape;
a real-time discrimination unit configured to acquire an image frame containing a current face image of a driver as a target image frame in real time, and perform a facial feature point positioning operation for the target image frame to determine a face shape associated with the target image frame, whereupon the face shape associated with the target image frame is compared with the reference face shape to determine whether the face shape associated with the target image frame is valid, and perform a subsequent fatigue driving detection operation only if the face shape associated with the target image frame is valid.
15. A vehicle comprising the facial feature point location misidentification device of claim 14.
16. A fatigue driving determination method using the facial feature point localization misidentification method for fatigue driving detection as claimed in any one of claims 1-13.
17. A fatigue driving determination system comprising the facial feature point location misidentification device of claim 14.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710092629.7A CN107016336A (en) | 2017-02-21 | 2017-02-21 | The facial characteristics point location detected for fatigue driving is corrected errors the method and device of identification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710092629.7A CN107016336A (en) | 2017-02-21 | 2017-02-21 | The facial characteristics point location detected for fatigue driving is corrected errors the method and device of identification |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107016336A true CN107016336A (en) | 2017-08-04 |
Family
ID=59440734
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710092629.7A Pending CN107016336A (en) | 2017-02-21 | 2017-02-21 | The facial characteristics point location detected for fatigue driving is corrected errors the method and device of identification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107016336A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112949345A (en) * | 2019-11-26 | 2021-06-11 | 北京四维图新科技股份有限公司 | Fatigue monitoring method and system, automobile data recorder and intelligent cabin |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102254151A (en) * | 2011-06-16 | 2011-11-23 | 清华大学 | Driver fatigue detection method based on face video analysis |
-
2017
- 2017-02-21 CN CN201710092629.7A patent/CN107016336A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102254151A (en) * | 2011-06-16 | 2011-11-23 | 清华大学 | Driver fatigue detection method based on face video analysis |
Non-Patent Citations (2)
| Title |
|---|
| 张波等: "《基于人脸3D模型的驾驶人头部姿态检测》", 《汽车工程》 * |
| 邵黄芳等: "《一种基于二维信息匹配的ASM定位算法》", 《系统仿真学报》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112949345A (en) * | 2019-11-26 | 2021-06-11 | 北京四维图新科技股份有限公司 | Fatigue monitoring method and system, automobile data recorder and intelligent cabin |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106778737B (en) | A kind of license plate antidote, device and a kind of video acquisition device | |
| US9158738B2 (en) | Apparatus for monitoring vicinity of a vehicle | |
| WO2015052896A1 (en) | Passenger counting device, passenger counting method, and program recording medium | |
| US9202106B2 (en) | Eyelid detection device | |
| US10178382B2 (en) | Method for performing diagnosis of a camera system of a motor vehicle, camera system and motor vehicle | |
| US11017552B2 (en) | Measurement method and apparatus | |
| WO2012042580A1 (en) | Line-of-sight estimation device | |
| US8983147B2 (en) | Eyelid detection device | |
| US10521679B2 (en) | Human detection device, human detection system, human detection method, and human detection program | |
| WO2018095194A1 (en) | Image analysis method and device | |
| JP2014009975A (en) | Stereo camera | |
| CN108198431A (en) | Vehicle parking detection device | |
| JP2012123642A (en) | Image identifying device and program | |
| TW201738806A (en) | Image recognition system and self-adaptive learning method thereof with a learning unit to receive the output picture, and calculate by the set of Weak Classifier parameters and a set of Strong Classifier parameters to judge whether the output image conforms to the preset target or not | |
| CN108074263B (en) | Visual positioning method and system | |
| CN112104838B (en) | Image distinguishing method, monitoring camera and monitoring camera system thereof | |
| CN112766023B (en) | Method, device, medium and equipment for determining gesture of target object | |
| US8503737B2 (en) | Visual line estimating apparatus | |
| KR101236223B1 (en) | Method for detecting traffic lane | |
| KR101949765B1 (en) | Apparatus and method for recognizing vehicle location information using number plate recognition | |
| CN107016336A (en) | The facial characteristics point location detected for fatigue driving is corrected errors the method and device of identification | |
| CN106919916A (en) | For the face front attitude parameter method of estimation and device of driver status detection | |
| WO2022130709A1 (en) | Object identification device and object identification method | |
| JP4947592B2 (en) | Vehicle detection device | |
| KR102270678B1 (en) | Apparatus for safety-driving of vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170804 |