CN101968918B - Feedback type fatigue detection system - Google Patents
Feedback type fatigue detection system Download PDFInfo
- Publication number
- CN101968918B CN101968918B CN2010105267839A CN201010526783A CN101968918B CN 101968918 B CN101968918 B CN 101968918B CN 2010105267839 A CN2010105267839 A CN 2010105267839A CN 201010526783 A CN201010526783 A CN 201010526783A CN 101968918 B CN101968918 B CN 101968918B
- Authority
- CN
- China
- Prior art keywords
- fatigue
- driver
- fatigue detecting
- voice
- detecting unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000001514 detection method Methods 0.000 title abstract description 18
- 230000003287 optical effect Effects 0.000 claims abstract description 15
- 238000006757 chemical reactions by type Methods 0.000 claims description 33
- 230000009471 action Effects 0.000 claims description 32
- 230000002452 interceptive effect Effects 0.000 claims description 14
- 230000004044 response Effects 0.000 claims description 13
- 230000011664 signaling Effects 0.000 claims description 13
- 210000003128 head Anatomy 0.000 claims description 11
- 101100433727 Caenorhabditis elegans got-1.2 gene Proteins 0.000 claims description 2
- 238000005070 sampling Methods 0.000 claims description 2
- 230000003993 interaction Effects 0.000 abstract description 5
- 238000005516 engineering process Methods 0.000 description 8
- 238000000034 method Methods 0.000 description 8
- 230000001815 facial effect Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 238000005259 measurement Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 230000011514 reflex Effects 0.000 description 3
- 241000406668 Loxodonta cyclotis Species 0.000 description 2
- 241000282376 Panthera tigris Species 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000001276 controlling effect Effects 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000035479 physiological effects, processes and functions Effects 0.000 description 2
- 244000287680 Garcinia dulcis Species 0.000 description 1
- 241001282135 Poromitra oscitans Species 0.000 description 1
- 206010048232 Yawning Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000002490 cerebral effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000000875 corresponding effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000004424 eye movement Effects 0.000 description 1
- 239000012467 final product Substances 0.000 description 1
- 238000013100 final test Methods 0.000 description 1
- 230000004886 head movement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000004297 night vision Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 210000001747 pupil Anatomy 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000011435 rock Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Landscapes
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a feedback type fatigue detection system which is characterized by comprising a sensing module, a fatigue detection module and a voice alarm and feedback module. The sensing module is used for collecting image optical signals and/or voice signals from a driver. The fatigue detection module is connected with the sensing module and used for judging the fatigue state of the driver according to the collected image optical signals and/or voice signals. And the voice alarm and feedback module is connected with the fatigue detection module and used for sending out a voice alarm signal and a voice interaction signal according to the fatigue state of the driver and feeding back the related information of the voice interaction signal to the fatigue detection module to be used as a reference standard for judging the fatigue state of the driver. The invention has high accuracy for fatigue detection of drivers and can effectively prevent false alarms and missed alarms.
Description
Technical field
The present invention relates to vehicle safety and Monitoring and Controlling field, particularly a kind of reaction type fatigue detecting system.
Background technology
Fatigue driving is one of important hidden danger of current traffic safety.The driver is when fatigue, and therefore its perception to surrounding environment, situation judgement and the ability of controlling of vehicle all had decline in various degree be easy to occurrence of traffic accident.Detection early warning technology about safe conditions such as driver's fatigue and attention dispersion; Owing to it receives the attention of various countries' height in the development prospect aspect the traffic accident prevention, the researchist has carried out many-sided research at physiology and operational characteristic according to the driver when tired.The detection method of driver's fatigue state can roughly be divided into based on driver's operation behavior, based on car status information, based on driver's physiological signal, based on four kinds of detection methods of driver's physiological reaction characteristic.
Driver's fatigue state recognition technology based on driver's operation behavior is meant through driver's operation behavior and infers driver's fatigue state.For example detect, and detected data are handled, thereby draw the service data of driver's bearing circle and the relation between the fatigue data through bearing circle operation to the driver.
Based on the fatigue state recognition technology of car status information, be meant and utilize vehicle driving trace to change and lane line such as departs from the fatigue state that the vehicle driving information is inferred the driver.This method is the same with fatigue state recognition technology based on driver's operation behavior, all is the basis with the vehicle conventional device, need not add too much hardware device, and can not cause interference to driver's normal driving, therefore has higher utility.
But; Driver's operation behavior and car status information are except outside the Pass the fatigue state with the driver has; Also receive the influence of factors such as personal habits, travel speed, road environment, operative skill; Therefore based on driver's operation behavior with based on the fatigue detecting method of car status information, there is the higher shortcoming of faults rate in it.
Detection method based on driver's physiological signal starts from physiology the earliest.Correlative study shows that the physical signs of driver under fatigue state can depart from the index of normal condition.Therefore can judge whether the driver gets into fatigue state through driver's physical signs.At present comparatively ripe detection method comprises the measurement of EEG signals EEG to the driver, core signal ECG etc.For example when getting into fatigue state, the delta ripple among the EEG signals EEG and the activity of theta ripple can increase substantially, and the activity of alpha ripple has and increases by a small margin, and core signal ECG can significantly clocklike descend.Therefore through just can know driver's fatigue state to the measurement of EEG signals EEG, core signal ECG.
Though the detection method based on driver's physiological signal is higher to the accuracy of fatigue judgement; But physiological signal often need adopt contact type measurement; Need wear the helmet all the time during such as measure cerebral electric signal EEG, thus bigger to individual degree of dependence, a lot of limitation is arranged when being actually used in driver's fatigue detecting; Even checkout equipment can cause interference to driving, therefore the application of this detection method difficulty very.
Be meant that based on driver's physiological reaction characteristic the eye movement characteristic of utilizing the driver, head movement characteristic etc. infer driver's fatigue state, it is to use maximum a kind of detection methods at present.For example the patent of invention of Chinese patent numbers 200610031817.0 discloses a kind of driver fatigue monitoring system and method based on image processing and information fusion technology; Its principle is following: at first by the continuous digital image signal of camera collection; Detect people's face position, and then the position and the size of definite eyes, pupil, face; Confirm that again the driver's eyes closure time accounts for the size of rule, eye gaze direction and time four fatigue characteristics that the number percent (PERCLOS) of T.T., yawning frequency, head rocks, and convert the four measuring value into degree of fatigue; Last exploit information integration technology merges the information of four fatigue characteristics of driver, judges the current fatigue conditions of driver; When degree of fatigue surpasses setting value, alarm will start warning.This system is through merging a plurality of fatigue characteristics, thereby reaches the effect that reduces rate of false alarm and rate of failing to report.
But; This fatigue state detection method based on driver's physiological reaction characteristic all need be gathered digital image signal; And tend to receive the bigger influence of objective environment factor when will gather digital image signal, for example night, insufficient light or strong solar light irradiation all can affect to IMAQ.And for example some driver habits are worn sunglasses, then can't collect the image information of eye, thereby cause occurring false alarm or fail to report the police.
In sum, there is the high problem of error rate in existing fatigue detecting system.
Summary of the invention
The purpose of this invention is to provide a kind of reaction type fatigue detecting system, have the high problem of error rate to solve existing fatigue detecting system.
The present invention proposes a kind of reaction type fatigue detecting system, it is characterized in that, comprises sensing module, fatigue detecting module and audio alert and feedback module.Sensing module is used to gather image optical signalling and/or the voice signal that comes from the driver.The fatigue detecting module links to each other with sensing module, is used for judging driver's fatigue state according to the image optical signalling and/or the voice signal that collect.Audio alert and feedback module; Link to each other with the fatigue detecting module; Be used for sending audio alert signal and interactive voice signal according to driver's fatigue state; And the response of the action command information of interactive voice signal prompt or system's expectation fed back to the fatigue detecting module, this fatigue detecting module combines the voice signal and/or the image optical signalling of response and driver's response of said action command information or system's expectation, as the normative reference of judging driver's fatigue state.
Wherein, this sensing module comprises a voice sensing unit, is used to gather the voice signal that comes from the driver; This fatigue detecting module comprises the fatigue detecting unit of a computing voice energy; It links to each other with feedback module with this voice sensing unit and this audio alert respectively; The response of the system's expectation that is used for sending according to this audio alert and feedback module; And combine the voice signal that the driver sends in a period of time that this voice sensing unit collects, obtain its fatigue state value through the speech energy that calculates the driver.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, sensing module comprises image sensing cell, is used to gather certainly in driver's image optical signalling.The fatigue detecting module comprises the fatigue detecting unit of identification maneuver; It links to each other with feedback module with image sensing cell and audio alert respectively; Be used for the action command information that sends according to audio alert and feedback module; And the image optical signalling of combining image sensing unit driver's action in a period of time of collecting, calculate driver's fatigue state value.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, the fatigue detecting unit of identification response actions calculates its fatigue state value through identification driver's the action of shaking the head, specifically according to following computing formula:
Wherein, x is the horizontal range between two centers of driver, and y is the vertical ranges of two centers of driver to the mouth center, and g is a characteristic quantity, p
1Be fatigue state value based on action.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, the fatigue detecting unit of computing voice energy is specifically according to following computing formula:
Wherein, N is a current time, and M is the time span of sampling, x
nBe the amplitude of voice signal, E
NBe the speech energy of driver's current time, Eo is the energy of driver's normal voice, P
2Be fatigue state value based on speech energy.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, sensing module comprises image sensing cell.The fatigue detecting module comprises the fatigue detecting unit of identification maneuver and the fatigue detecting unit of recognizing voice content; The fatigue detecting unit of identification maneuver links to each other with image sensing cell, and the fatigue detecting unit of recognizing voice content links to each other with the voice sensing unit respectively.Integrated unit; It links to each other with feedback module with the fatigue detecting unit of identification maneuver, the fatigue detecting unit of computing voice energy, the fatigue detecting unit and the audio alert of recognizing voice content respectively; Be used for the fatigue state that each fatigue detecting unit calculates is merged, and finally calculate driver's degree of fatigue.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, integrated unit is specifically according to following computing formula:
Wherein, Y is driver's a degree of fatigue, and n is the number of the fatigue detecting unit of participation, and the n value is 4 here; The fatigue state that Xi calculates for each fatigue detecting unit is got 1 when tired, otherwise is got 0; Ii gets 1, otherwise gets 0 for the indieating variable that enables of each fatigue detecting unit when enabling.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, the fatigue detecting module also further comprises the fatigue detecting unit of face image, and it links to each other with image sensing cell, is used for calculating its fatigue state value according to driver's face image.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, what the fatigue detecting unit of face image calculated is the percent of shared time of driver's eyes closed in a period of time, and it is specifically according to following computing formula:
Wherein, f is the percent of shared time of driver's eyes closed in a period of time, and t1 is that eyes open for closed 20% time fully; T2 is that eyes open for closed 80% time fully; T3 is that eyes are opened 20% the time of next time opening fully; T4 is that eyes are opened 80% the time of next time opening fully.
With respect to prior art, the invention has the beneficial effects as follows:
1, the reaction type mechanism that system of the present invention confirms through specific action and particular problem answer affirmation is such is carried out fatigue identification, can judge driver's fatigue state more exactly, effectively avoids the generation of false-alarm, false dismissal situation.
2, system of the present invention is through merging the output of a plurality of fatigue detecting unit, and draws final testing result, can eliminate the flase drop that causes because of uncertain factors such as environment maybe, have very high accuracy.
3, the present invention fully use this nature of voice, under driving condition, the driver is disturbed minimum media of communication, produce interaction with the driver, can reach the effect of the alleviation driving fatigue of nature.
Description of drawings
Fig. 1 is a kind of Organization Chart of reaction type fatigue detecting system of the present invention;
Fig. 2 is first kind of example structure figure of reaction type fatigue detecting system of the present invention;
Face's location synoptic diagram that Fig. 3 adopts when shaking the head action for the present invention's identification;
The characteristic quantity oscillogram that Fig. 4 adopts when shaking the head action for the present invention's identification;
Fig. 5 is second kind of example structure figure of reaction type fatigue detecting system of the present invention;
Fig. 6 is the third example structure figure of reaction type fatigue detecting system of the present invention;
Fig. 7 is the 4th kind of example structure figure of reaction type fatigue detecting system of the present invention;
Fig. 8 is the measuring principle figure of the fatigue detecting unit of face image of the present invention to the PERCLOS value;
Fig. 9 is the 5th kind of example structure figure of reaction type fatigue detecting system of the present invention;
Figure 10 is the 6th kind of example structure figure of reaction type fatigue detecting system of the present invention.
Embodiment
Main thought of the present invention is to make full use of this media of communication of voice; Make system and driver produce interaction, thereby not only can effectively avoid false-alarm (false alarm, the driver is in fatigue state; System has but produced alarm) and false dismissal (driver is in fatigue state; System does not but produce alarm), improve the correct recognition rata of fatigue detecting system, and can play the effect of alleviating driving fatigue.
See also Fig. 1, it is for a kind of structural drawing of reaction type fatigue detecting system of the present invention.It comprises sensing module 100, fatigue detecting module 200 and audio alert and feedback module 300.Fatigue detecting module 200 links to each other with feedback module 300 with sensing module 100 and audio alert respectively.
Before system's entering work, need carry out initialization earlier.Because the present invention image and the voice signal that are based on being collected in the driver carry out analytical calculation, judge its fatigue state, thereby need to gather in advance under the normal conditions (non-fatigue state), driver's image and voice messaging are with standard as a reference.The eyes size of for example measuring the driver and the energy of position, normal voice signal etc.Initialized process is simple, only needs the positive face of driver facing to fatigue detecting equipment, accomplishes once simple dialogue and gets final product.
During work, sensing module 100 can be gathered and come from driver's image optical signalling and/or voice signal, and is transferred to fatigue detecting module 200.The information that fatigue detecting module 200 is sent according to sensing module 100 is judged driver's fatigue state, and testing result is sent to audio alert and feedback module 300.Audio alert and feedback module 300 can send audio alert signal and interactive voice signal according to driver's fatigue state.When sending the interactive voice signal, audio alert and feedback module 300 can feed back to fatigue detecting module 200 with the relevant information of interactive voice signal, as the normative reference of judging driver's fatigue state.Simultaneously, sensing module 100 can be gathered once more and come from driver's image optical signalling and/or voice signal, and is transferred to fatigue detecting module 200.Fatigue detecting module 200 can be come according to sensing module 100 transmission and audio alert is evaluated driver's fatigue state with the next information of feedback module 300 feedbacks again; And give audio alert and feedback module 300 with result transmission, with grade and the kind that determines the audio alert that it sends.So cycle detection realizes the real-time monitoring to driver's fatigue strength.
With several representative embodiment, specify the present invention below.
Embodiment one
See also Fig. 2; This reaction type fatigue detecting system; Its sensing module 100 comprises image sensing cell 101; Its fatigue detecting module 200 comprises the fatigue detecting unit 201 of identification maneuver, and the fatigue detecting unit 201 of identification maneuver links to each other with feedback module 300 with image sensing cell 101 and audio alert respectively.
In the present embodiment, the input of image sensing cell 101 is facial image optics signals, and output is digitized facial image.Image sensing cell 101 can adopt near infrared camera image sensor, and under the night vision environment, this camera still can normally obtain facial image.System outputs in the fatigue detecting unit 201 of identification maneuver after collecting facial image through image sensing cell 101.
The input of the fatigue detecting unit 201 of identification maneuver has two: come from continuous people's face digital picture of image sensing cell 101, and the action command information of audio alert and feedback module 300 feedbacks.The identification of specific action is realized in the fatigue detecting unit 201 of identification maneuver, confirms whether human pilot has accomplished corresponding action according to voice suggestion.For example, 300 promptings of audio alert and feedback module: " please do the action of shaking the head ", and command information fed back to the fatigue detecting unit 201 of identification maneuver.People's face video of the 201 pairs of image sensing cells in fatigue detecting unit, 101 inputs of identification maneuver is analyzed at this moment, and affirmation is whether the driver has accomplished the action of shaking the head.
Existing is that example specifies computation process with the action of shaking the head.The fatigue detecting unit 201 of identification maneuver at first needs the mouth center of Fine Mapping people face and the coordinate of eye center in computation process.As shown in Figure 3, X is two horizontal ranges between the center, and Y is the vertical ranges of two centers to the mouth center, through the accurate location of human face, and recording feature amount g=X/Y.Accomplish in the action of shaking the head the people so, the X/Y characteristic can present wave form varies pattern as shown in Figure 4.
Height point expression people face in the waveform is in the positive face state of level, and this moment, character numerical value was maximum; Low spot representes that people's face is in the state of side face in the waveform, and this moment, character numerical value was minimum.In a period of time, search characteristics maximal value gmax and minimum value gmin, calculate the fatigue state value P1 based on action then:
Confirm through experience, usually, when P1＞0.3, can think that the driver has accomplished the level that the cooperates action of shaking the head, and promptly is in non-fatigue state.Otherwise then be in fatigue state, and send the relevant voice alarm by audio alert and feedback module 300.
Embodiment two
See also Fig. 5; This reaction type fatigue detecting system; Its sensing module 100 comprises voice sensing unit 102; Its fatigue detecting module 200 comprises the fatigue detecting unit 202 of computing voice energy, and the fatigue detecting unit 202 of computing voice energy links to each other with feedback module 300 with voice sensing unit 102 and audio alert respectively.
The design starting point of present embodiment is to attempt to constitute interactive voice with the driver, and judges whether to fail alternately whether the driver is in fatigue state.Wherein the fatigue detecting unit 202 of computing voice energy is used for judging through the speech energy that calculates the driver driver's fatigue state; Its input be the voice signal that voice sensing unit 102 is gathered, and from the feedback information of audio alert and feedback module 300.
System can use level and smooth square of energy in realizing, its computing formula is:
Implication directly perceived is that current time N with preceding M signal amplitude Xn square summation constantly of current time, just can be used as a kind of sign of signal energy EN.Because driver's voice responsive duration of the expectation in the system is all shorter, in the enforcement, get and generally get 0.5 second, like this according to SF, just can calculate the size of M.
Because the actual environment for use more complicated of system, thereby can use relative value to weigh driver's fatigue state.In initialization procedure, when the energy of the normal voice that measures human pilot is then to remember the fatigue state value based on speech energy
Rule of thumb, work as P
2＜0.3 o'clock, can think the response that does not detect the driver, therefore, it is in fatigue state decidable.
What need replenish is to use P here
2Value is meant as criterion, the P of maximum in certain observation period
2Value.This observation period is that the system prompt driver sends certain voice.For example, 300 couples of drivers of audio alert and feedback module have said a joke, can investigate P in a period of time after joke has been said
2Perhaps 300 couples of drivers of audio alert and feedback module have carried out an enquirement, can regard the observation period as in a period of time after the enquirement.
Certainly, except adopting speech energy, also can use other index to weigh.Calculate instantaneous energy etc. after for example using wave-shape amplitude, use time-frequency conversion.
Embodiment three
See also Fig. 6; This reaction type fatigue detecting system; Its sensing module 100 comprises voice sensing unit 102; Its fatigue detecting module 200 comprises the fatigue detecting unit 203 of recognizing voice content, and the fatigue detecting unit 203 of recognizing voice content links to each other with feedback module 300 with voice sensing unit 102 and audio alert respectively.
The same with the fatigue detecting unit 202 of computing voice energy, the input of the fatigue detecting unit 203 of recognizing voice content also is the voice signal gathered of voice sensing unit 102 and from the feedback information of audio alert and feedback module 300.Judge whether to fail alternately whether human pilot is in fatigue state too.Different is that the formation base of present embodiment is based on the mutual of voice content.
The fatigue detecting unit 203 of recognizing voice content utilizes speech recognition technology; The voice messaging that the response of comparison system expectation and audio alert and feedback module 300 collect; If it is consistent to contrast both, thinks that then the driver is not in fatigue state, otherwise then be in fatigue state.
For example, audio alert and feedback module 300 provide an enquirement: " numeral 5 is bigger than numeral 2, is it right? " When human pilot answer " to "; The fatigue detecting unit 203 of recognizing voice content just thinks that it is not in fatigue state, otherwise, think that then it is in fatigue state; Similar problem, " area code in Shanghai is 021, is it right? in addition ", " build of elephant is bigger than tiger, is it right? ", " than 5 big what are than 7 little numerals? " Deng.
Consider that the system applies environment maybe be comparatively noisy, comparatively simple with interaction content design in the system implementation.Include only: " to ", " mistake " and " 0 to 9 numeral ".Wherein, " to " and " mistake " also designed synonym, " yes ", " not to ", " quite right " or the like.
Embodiment four
See also Fig. 7, this reaction type fatigue detecting system, its sensing module 100 comprises image sensing cell 101.Its fatigue detecting module 200 comprises the fatigue detecting unit 201 of identification maneuver, the fatigue detecting unit 204 and the integrated unit 205 of face image.The fatigue detecting unit 201 of identification maneuver and the fatigue detecting unit 204 of face image link to each other with image sensing cell 101 and integrated unit 205 respectively; Integrated unit 205 links to each other with feedback module 300 with audio alert, and audio alert and feedback module 300 are reversely connected to the fatigue detecting unit 201 of identification maneuver.
During work; Image sensing cell 101 can be transferred to the fatigue detecting unit 201 of identification maneuver and the fatigue detecting unit 204 of face image respectively with the image optical signalling that collects; Calculate driver's fatigue state respectively by the fatigue detecting unit 201 of identification maneuver and the fatigue detecting unit 204 of face image, and together input to integrated unit 205.The fatigue state value that integrated unit 205 can calculate both merges, and audio alert and feedback module 300 are exported in final fatigue identification.Audio alert and feedback module 300 send relevant voice and report to the police or the interactive voice signal according to the tired identifying information that integrated unit 205 sends, and simultaneously the interactive voice signal feedback are given the fatigue detecting unit 201 of identification maneuver.
In the present embodiment, what the fatigue detecting unit 204 of face image was imported is digitized facial image, and output is the fatigue state value.It realizes the detection to the PERCLOS value of facial image.Its calculating principle is through the accurate location of human face, to calculate shared ratio of eyes closed time in a certain period, in order to weigh people's degree of fatigue.Fig. 8 is the measuring principle figure of the 204 pairs of PERCLOS values in fatigue detecting unit of face image of the present invention.Curve is eyes closed and open the degree of opening in process curve over time among the figure, can obtain certain degree of eyes closed of required measurement or open the lasting time according to this curve, thereby calculate the PERCLOS value.T1 is that eyes open for closed 20% time fully among the figure; T2 is that eyes open for closed 80% time fully; T3 is that eyes are opened 20% the time of next time opening fully; T4 is that eyes are opened 80% the time of next time opening fully.Through measuring t1 just can calculate PERCLOS to the value of t4 value:
In the formula, f is the percent of shared a certain special time of eyes closed time.It is generally acknowledged, when PERCLOS value f＞0.15, think that the driver is in fatigue state.
In the present embodiment, system does not directly use the criterion of f value as degree of fatigue, also combines the output of the fatigue detecting unit 201 of identification maneuver, judges jointly whether the driver is in fatigue state.
Embodiment five
See also Fig. 9, this reaction type fatigue detecting system, its sensing module 100 comprises voice sensing unit 102.Its fatigue detecting module 200 comprises the fatigue detecting unit 202 of computing voice energy, the fatigue detecting unit 203 and the integrated unit 205 of recognizing voice content.The fatigue detecting unit 202 of computing voice energy and the fatigue detecting unit 203 of recognizing voice content link to each other with voice sensing unit 102 and integrated unit 205 respectively; Integrated unit 205 links to each other with feedback module 300 with audio alert, and audio alert and feedback module 300 are reversely connected to the fatigue detecting unit 202 of computing voice energy and the fatigue detecting unit 203 of recognizing voice content.
Present embodiment is that embodiment two is combined with embodiment three to use; Voice sensing unit 102 can be transferred to the fatigue detecting unit 202 of computing voice energy and the fatigue detecting unit 203 of recognizing voice content respectively with the voice signal that collects; Calculate driver's fatigue state respectively by the fatigue detecting unit 202 of computing voice energy and the fatigue detecting unit 203 of recognizing voice content, and together input to integrated unit 205.The fatigue state value that integrated unit 205 can calculate both merges, and audio alert and feedback module 300 are exported in final fatigue identification.The tired identifying information that audio alert and feedback module 300 send according to integrated unit 205; Send relevant voice and report to the police or the interactive voice signal, give the fatigue detecting unit 202 of computing voice energy and the fatigue detecting unit 203 of recognizing voice content with the interactive voice signal feedback simultaneously.
Embodiment six
Present embodiment is that embodiment four and embodiment five are combined to use.See also Figure 10, the sensing module 100 of this system comprises image sensing cell 101 and voice sensing unit 102.Fatigue detecting module 200 comprises the fatigue detecting unit 201 of identification maneuver, the fatigue detecting unit 204 of face image, the fatigue detecting unit 202 of computing voice energy, the fatigue detecting unit 203 and the integrated unit 205 of recognizing voice content.Image sensing cell 101 links to each other with the fatigue detecting unit 201 of identification maneuver and the fatigue detecting unit 204 of face image respectively.Voice sensing unit 102 links to each other with the fatigue detecting unit 202 of computing voice energy and the fatigue detecting unit 203 of recognizing voice content respectively.The fatigue detecting unit 202 of the fatigue detecting unit 201 of identification maneuver, the fatigue detecting unit 204 of face image, computing voice energy and the fatigue detecting unit 203 of recognizing voice content all link to each other with integrated unit 205.Integrated unit 205 links to each other with feedback module 300 with audio alert.Audio alert and feedback module 300 are reversely connected to the fatigue detecting unit 201 of identification maneuver, the fatigue detecting unit 202 of computing voice energy and the fatigue detecting unit 203 of recognizing voice content respectively.
The system of present embodiment is divided into two identified branches, and one is image recognition, makes one to be speech recognition.Two branches can work alone, and also can work simultaneously, and two branches finally merge through integrated unit 205, and recognition result is exported to audio alert and feedback module 300 the most at last.
More than all fatigue detecting unit, the user all can initiatively close.For example, the driver is wearing the situation of sunglasses, can initiatively the fatigue detecting unit 204 of face image be closed.Four fatigue detecting unit can be exported the driver respectively and whether be in tired judgement under the condition that enables, and through voting mechanism, merging becomes final fatigue identification output.When certain fatigue detecting unit does not enable, think that this module is absent from voting.
The input of four fatigue detecting unit of note is respectively Xi, and value Xi is 1 when thinking fatigue, otherwise the Xi value is 0; Enabling indieating variable does, when value is 1, and this fatigue detecting cell enable, otherwise this fatigue detecting unit does not enable, i.e. abstention.Integrated unit 205 final output Y are:
Therefore, the span of Y is between 0 to 1.Possible numerical value is 0,0.25,0.5,0.75 and 1, have five ranks altogether.For each output Y, indieating variable
and can be used as the confidence level index.
is big more; Explain that the fatigue detecting unit of participating in ballot is many more, vice versa.
The input of audio alert and feedback module 300 is the recognition results that merge, and audio alert can adopt different output with feedback module 300 for different fusion results.Generally speaking, it has three types output:
(1) general voice reminder.The Typical Disposition of this type prompting is as shown in table 1:
| Degree of fatigue | Voice reminder | |
| 0 | " please note safety ", " please focus one's attention on " | |
| 0.25 | " Don't Drive When Tired ": | |
| 0.5 | " you possibly be in fatigue state, please focus one's attention on, and drive with caution ": | |
| 0.75 | " you possibly be in serious fatigue state, the rest of please stopping ": | |
| 1 | " rest of please stopping immediately ": " dripping " (alarm tone) |
Table 1
Audio alert and feedback module 300 also can be considered confidence level simultaneously, and when confidence level higher (the fatigue detecting element number that promptly enables is more), the multiplicity of voice output increases.This type voice reminder is unidirectional output, do not have and the driver between interactive with exchange.
(2) action command is reported to the police.Usually, when driver's degree of fatigue＞=0.5 was judged by system, audio alert and feedback module 300 can provide the action command warning at random.This type report to the police the starting point of design be the requirement driver with correct response actions, confirm its not be in fatigue state.Otherwise system can think that it is in fatigue state.
At present, in the system implementation case, two kinds of actions of more employing, " please do the action of shaking the head " and " please do nodding action ".In principle, can also be easy to expansion and use other action command, for example, " please use hand combing hair once ".But, because the singularity of driving environment when designing this type action, reduce the interference to driving behavior itself as far as possible.
(3) voice dialogue is reported to the police.This type warning is divided into two types, promptly
A, the audio alert of expectation coupling answer;
B, the audio alert that still content is not had particular requirement is responded in expectation.
For the category-A audio alert, typical problem is a digital issue, the perhaps simple matter of right and wrong.Here take the example in some problem bases: " elephant is bigger than tiger, is it right? ", " area code in Shanghai is 021, is it right? ", " than 5 big what are than 7 little numerals? ", " in the time of thunderstorm, usually can lightning, is that right? "In order to control the accuracy of speech recognition well, thereby can the answer of problem to be controlled at be in the not sum digital scope.
For the category-B audio alert, system checks on one's answers and no requirement (NR).Typical conversation content is to say a joke, think hard zig zag, guessing riddles etc.
Various types of voice is reported to the police enable the fatigue detecting unit and feedback signal as shown in table 2:
Table 2
Can be found out that by table 2 audio alert and feedback module 300 can adopt the form of coding to each fatigue detecting unit feedack, generally speaking, there are following three types of feedback informations as shown in table 3 in system:
Table 3
System is after the voice alarm of sending the answer of expectation coupling, and except being non-coding (011 and 012) and the digital answer coding (020 to 029), expectation speech response (030) feedback information exists simultaneously.
System of the present invention through with driver's interactive voice, reach and detect accuracy rate more accurately, can effectively avoid the generation of false-alarm, false dismissal.More than disclosedly be merely several specific embodiment of the present invention, but the present invention is not limited thereto, any those skilled in the art can think variation, all should drop in protection scope of the present invention.
Claims (8)
1. a reaction type fatigue detecting system is characterized in that, comprising:
One sensing module is used to gather the image optical signalling and/or the voice signal that come from the driver;
One fatigue detecting module links to each other with this sensing module, is used for judging driver's fatigue state according to the image optical signalling and/or the voice signal that collect;
One audio alert and feedback module; Link to each other with this fatigue detecting module; Be used for sending audio alert signal and interactive voice signal according to driver's fatigue state; And the response of the action command information of interactive voice signal prompt or system's expectation fed back to this fatigue detecting module, this fatigue detecting module combines the voice signal and/or the image optical signalling of response and driver's response of said action command information or system's expectation, as the normative reference of judging driver's fatigue state;
Wherein, this sensing module comprises a voice sensing unit, is used to gather the voice signal that comes from the driver; This fatigue detecting module comprises the fatigue detecting unit of a computing voice energy; It links to each other with feedback module with this voice sensing unit and this audio alert respectively; The response of the system's expectation that is used for sending according to this audio alert and feedback module; And combine the voice signal that the driver sends in a period of time that this voice sensing unit collects, obtain its fatigue state value through the speech energy that calculates the driver.
2. reaction type fatigue detecting system as claimed in claim 1 is characterized in that,
This sensing module comprises an image sensing cell, is used to gather certainly in driver's image optical signalling;
This fatigue detecting module comprises the fatigue detecting unit of an identification maneuver; It links to each other with feedback module with this image sensing cell and this audio alert respectively; Be used for the action command information that sends according to audio alert and feedback module; And combine the image optical signalling of driver's action in a period of time that this image sensing cell collects, calculate driver's fatigue state value.
3. reaction type fatigue detecting system as claimed in claim 2 is characterized in that, the fatigue detecting unit of this identification maneuver calculates its fatigue state value through identification driver's the action of shaking the head, specifically according to following computing formula:
Wherein, x is the horizontal range between two centers of driver, and y is the vertical ranges of two centers of driver to the mouth center, and g is a characteristic quantity, p
1Be fatigue state value based on action.
4. reaction type fatigue detecting system as claimed in claim 1 is characterized in that, the fatigue detecting unit of this computing voice energy is specifically according to following computing formula:
Wherein, N is a current time, and M is the time span of sampling, x
nBe the amplitude of voice signal, E
NBe the speech energy of driver's current time, Eo is the energy of driver's normal voice, P
2Be fatigue state value based on speech energy.
5. reaction type fatigue detecting system as claimed in claim 1 is characterized in that,
This sensing module comprises an image sensing cell;
This fatigue detecting module comprises the fatigue detecting unit of an identification maneuver, the fatigue detecting unit and an integrated unit of a recognizing voice content; The fatigue detecting unit of this identification maneuver links to each other with this image sensing cell, and the fatigue detecting unit of this recognizing voice content links to each other with this voice sensing unit;
This integrated unit; It links to each other with feedback module with the fatigue detecting unit of this identification maneuver, the fatigue detecting unit of this computing voice energy, fatigue detecting unit and this audio alert of this recognizing voice content respectively; Be used for the fatigue state that each fatigue detecting unit calculates is merged, and finally calculate driver's degree of fatigue.
6. reaction type fatigue detecting system as claimed in claim 5 is characterized in that, this integrated unit is specifically according to following computing formula:
Wherein, Y is driver's a degree of fatigue, and n is the number of the fatigue detecting unit of participation; The fatigue state that Xi calculates for each fatigue detecting unit is got 1 when tired, otherwise is got 0; Ii gets 1, otherwise gets 0 for the indieating variable that enables of each fatigue detecting unit when enabling.
7. like claim 2,3,5,6 each described reaction type fatigue detecting system; It is characterized in that; This fatigue detecting module also further comprises the fatigue detecting unit of a face image; It links to each other with this image sensing cell, is used for calculating its fatigue state value according to driver's face image.
8. reaction type fatigue detecting system as claimed in claim 7; It is characterized in that; What the fatigue detecting unit of this face image calculated is the percent of shared time of driver's eyes closed in a period of time, and it is specifically according to following computing formula:
Wherein, f is the percent of shared time of driver's eyes closed in a period of time, and t1 is that eyes open for closed 20% time fully; T2 is that eyes open for closed 80% time fully; T3 is that eyes are opened 20% the time of next time opening fully; T4 is that eyes are opened 80% the time of next time opening fully.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010105267839A CN101968918B (en) | 2010-11-01 | 2010-11-01 | Feedback type fatigue detection system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010105267839A CN101968918B (en) | 2010-11-01 | 2010-11-01 | Feedback type fatigue detection system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101968918A CN101968918A (en) | 2011-02-09 |
| CN101968918B true CN101968918B (en) | 2012-05-23 |
Family
ID=43548065
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010105267839A Expired - Fee Related CN101968918B (en) | 2010-11-01 | 2010-11-01 | Feedback type fatigue detection system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101968918B (en) |
Families Citing this family (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102868951A (en) * | 2012-09-03 | 2013-01-09 | 陈火 | Sensing type early-warning earphone reminder |
| EP2708793B1 (en) * | 2012-09-15 | 2016-02-03 | ABB Technology AG | Safety device for a technical facility or a technical process |
| CN103009669B (en) * | 2012-12-11 | 2014-12-17 | 金丰（中国）机械工业有限公司 | Punch operator fatigue judgement system and punch operator fatigue judgement method |
| DE102013206212A1 (en) * | 2013-04-09 | 2014-10-09 | Ford Global Technologies, Llc | Method for controlling a vehicle with temporarily autonomous vehicle guidance and control device |
| JP2015184563A (en) * | 2014-03-25 | 2015-10-22 | シャープ株式会社 | Interactive household electrical system, server device, interactive household electrical appliance, method for household electrical system to interact, and program for realizing the same by computer |
| KR20150139229A (en) * | 2014-06-03 | 2015-12-11 | 현대모비스 주식회사 | Hmd utilizing driver's drowsiness warning apparatus and method thereof |
| CN104828095B (en) * | 2014-09-02 | 2018-06-19 | 北京宝沃汽车有限公司 | Detect the method, apparatus and system of driver's driving condition |
| CN104952209A (en) * | 2015-04-30 | 2015-09-30 | 广州视声光电有限公司 | Driving prewarning method and device |
| CN107633657A (en) * | 2016-07-18 | 2018-01-26 | 中兴通讯股份有限公司 | The based reminding method and terminal of a kind of fatigue driving |
| CN106657648A (en) * | 2016-12-28 | 2017-05-10 | 上海斐讯数据通信技术有限公司 | Mobile terminal for preventing fatigue driving and realization method thereof |
| CN107103294A (en) * | 2017-04-20 | 2017-08-29 | 上海耐相智能科技有限公司 | A kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone |
| CN107038433A (en) * | 2017-05-15 | 2017-08-11 | 柳州译海网络科技有限公司 | A kind of fingerprint speech recognition weariness working warning device |
| CN108099816A (en) * | 2017-12-25 | 2018-06-01 | 芜湖皖江知识产权运营中心有限公司 | A kind of vehicle audio input control method for intelligent travel |
| CN108162755A (en) * | 2017-12-25 | 2018-06-15 | 芜湖皖江知识产权运营中心有限公司 | A kind of fatigue driving identification control method applied in intelligent vehicle |
| CN108099915A (en) * | 2017-12-25 | 2018-06-01 | 芜湖皖江知识产权运营中心有限公司 | A kind of fatigue driving control system for identifying applied in intelligent vehicle |
| CN108831460A (en) * | 2018-06-15 | 2018-11-16 | 浙江吉利控股集团有限公司 | A kind of interactive voice control system and method based on fatigue monitoring |
| KR102529919B1 (en) * | 2018-11-28 | 2023-05-08 | 현대자동차주식회사 | Apparatus, system and method for managing drowsy driving |
| CN110303884B (en) * | 2019-07-10 | 2020-11-06 | 初速度（苏州）科技有限公司 | Fatigue driving prevention method, system and device |
| CN112396805B (en) * | 2019-07-31 | 2022-10-25 | Oppo广东移动通信有限公司 | Fatigue driving reminding method, device, terminal and storage medium |
| CN111354162A (en) * | 2020-03-07 | 2020-06-30 | 深圳市松恩电子科技有限公司 | Monitoring system for health fatigue and prevention of driver |
| CN112434563A (en) * | 2020-11-04 | 2021-03-02 | 芜湖职业技术学院 | Driving fatigue relieving system and method |
| CN116092267A (en) * | 2023-01-29 | 2023-05-09 | 浙江吉利控股集团有限公司 | Vehicle fatigue driving reminding method, device, equipment and storage medium |
| CN118220202B (en) * | 2024-05-07 | 2024-09-20 | 江苏益展研创智能科技有限公司 | Automatic reminding system for high-risk driving behavior of vehicle driver |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6313749B1 (en) * | 1997-01-04 | 2001-11-06 | James Anthony Horne | Sleepiness detection for vehicle driver or machine operator |
| DE10039551A1 (en) * | 2000-08-09 | 2002-02-21 | Vinzenz Von Holle | Sleep prevention device for automobile driver provides acoustic signal within seconds of driver falling asleep |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3362664B2 (en) * | 1998-04-24 | 2003-01-07 | ソニー株式会社 | Condition monitoring device |
| US7652583B2 (en) * | 2007-03-20 | 2010-01-26 | Deere & Company | Method and system for maintaining operator alertness |
-
2010
- 2010-11-01 CN CN2010105267839A patent/CN101968918B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6313749B1 (en) * | 1997-01-04 | 2001-11-06 | James Anthony Horne | Sleepiness detection for vehicle driver or machine operator |
| DE10039551A1 (en) * | 2000-08-09 | 2002-02-21 | Vinzenz Von Holle | Sleep prevention device for automobile driver provides acoustic signal within seconds of driver falling asleep |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101968918A (en) | 2011-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101968918B (en) | Feedback type fatigue detection system | |
| CN104146722B (en) | Driving fatigue detecting and grading early warning method based on head signals | |
| CN108052888B (en) | A kind of driver's replacement system | |
| Dong et al. | Driver inattention monitoring system for intelligent vehicles: A review | |
| CN106956591A (en) | A kind of system for being used to judge that human pilot drives authority | |
| CN101583313B (en) | Awake state judging model making device, awake state judging device, and warning device | |
| Shi et al. | A review on fatigue driving detection | |
| CN105595996B (en) | A kind of fatigue driving eeg monitoring method of electricity and brain electricity comprehensive judgement | |
| CN105869356A (en) | Railway driver work state evaluation intervening system and method | |
| CN106691443A (en) | Electroencephalogram-based wearable anti-fatigue intelligent monitoring and pre-warning system for driver | |
| CN106778618A (en) | A kind of fatigue driving monitoring early warning system | |
| CN104757981A (en) | Method and device for high-sensitively receiving and transmitting integrated infrared detection of driver's fatigue | |
| CN104183091A (en) | System for adjusting sensitivity of fatigue driving early warning system in self-adaptive mode | |
| CN106913333B (en) | Method for selecting sensitivity characteristic index of continuous attention level | |
| Sheng et al. | A case study of trust on autonomous driving | |
| CN110334592A (en) | A kind of monitoring of driver's abnormal behaviour and safety control system and safety control method | |
| CN102098955A (en) | Method and device for the detection of microsleep events | |
| CN106343977A (en) | UAV (unmanned aerial vehicle) operator state monitoring system based on multi-sensor fusion | |
| CN106236047A (en) | The control method of driver fatigue monitoring system | |
| CN112258790A (en) | Rail transit train driver on-duty fatigue level and body state detection system | |
| US11738763B2 (en) | Fatigue monitoring system for drivers tasked with monitoring a vehicle operating in an autonomous driving mode | |
| CN101426418A (en) | Incapacity monitor | |
| CN109875583A (en) | A kind of fatigue driving detecting system and method based on AR technology | |
| Alam et al. | Active vision-based attention monitoring system for non-distracted driving | |
| CN118097636A (en) | Image vision method and system for automatically detecting fatigue degree of driver based on artificial intelligence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120523 Termination date: 20131101 |