CN117770821A - Human-caused intelligent cabin driver fatigue detection method, device, vehicle and medium - Google Patents
Human-caused intelligent cabin driver fatigue detection method, device, vehicle and medium Download PDFInfo
- Publication number
- CN117770821A CN117770821A CN202311865457.4A CN202311865457A CN117770821A CN 117770821 A CN117770821 A CN 117770821A CN 202311865457 A CN202311865457 A CN 202311865457A CN 117770821 A CN117770821 A CN 117770821A
- Authority
- CN
- China
- Prior art keywords
- driver
- fatigue
- state
- data
- actual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 96
- 238000000034 method Methods 0.000 claims abstract description 38
- 230000003542 behavioural effect Effects 0.000 claims abstract description 17
- 230000006399 behavior Effects 0.000 claims description 57
- 230000008859 change Effects 0.000 claims description 34
- 230000008921 facial expression Effects 0.000 claims description 21
- 238000000605 extraction Methods 0.000 claims description 13
- 210000004556 brain Anatomy 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000013473 artificial intelligence Methods 0.000 claims description 3
- 210000003128 head Anatomy 0.000 description 17
- 238000012545 processing Methods 0.000 description 12
- 238000004891 communication Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 9
- 208000032140 Sleepiness Diseases 0.000 description 8
- 206010041349 Somnolence Diseases 0.000 description 8
- 230000000875 corresponding effect Effects 0.000 description 8
- 230000000694 effects Effects 0.000 description 8
- 230000001276 controlling effect Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 4
- 230000001419 dependent effect Effects 0.000 description 4
- 238000000540 analysis of variance Methods 0.000 description 3
- 230000004424 eye movement Effects 0.000 description 3
- 210000005036 nerve Anatomy 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000012880 independent component analysis Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 210000003205 muscle Anatomy 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000037321 sleepiness Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 241001282135 Poromitra oscitans Species 0.000 description 1
- 206010048232 Yawning Diseases 0.000 description 1
- 230000036982 action potential Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000036772 blood pressure Effects 0.000 description 1
- 230000036760 body temperature Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000010219 correlation analysis Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000002567 electromyography Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000004399 eye closure Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000004886 head movement Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000006996 mental state Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004118 muscle contraction Effects 0.000 description 1
- 230000003183 myoelectrical effect Effects 0.000 description 1
- 210000001087 myotubule Anatomy 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000001020 rhythmical effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Landscapes
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The application relates to the technical field of fatigue detection, in particular to a method, a device, a vehicle and a medium for detecting fatigue of a driver of an artificial intelligent cabin, wherein the method comprises the following steps: collecting physiological data and behavior data of a driver; extracting at least one argument feature of said driver from said physiological data and said behavioral data; and identifying the actual fatigue state of the driver according to the at least one independent variable characteristic, and controlling the vehicle to carry out state reminding on the driver based on the actual fatigue state. According to the method and the device for identifying the fatigue state of the driver, the at least one independent variable characteristic of the driver can be extracted according to the physiological data and the behavior data of the driver, so that the actual fatigue state of the driver is identified, the driver is timely reminded of the state, the accuracy of fatigue state identification of the driver is improved, the reliability of an identification result is improved, the experience of the driver and the viscosity of the client are enhanced, and different requirements of the driver under different scenes are met.
Description
Technical Field
The application relates to the technical field of fatigue detection, in particular to a fatigue detection method, device, vehicle and medium for a human-caused intelligent cabin driver.
Background
In the related art, fatigue driving is always a major hidden trouble of automobile driving safety, and at present, fatigue driving detection of a driver mainly detects whether the driver is in a fatigue driving state according to facial expression and related road information of the driver detected by a camera.
However, in the related art, the driving state of the driver is detected according to the facial expression and the road information of the driver alone, so that the fatigue detection mode and the recognition result are single, the actual driving state of the driver cannot be accurately recognized, the accuracy and the reliability of the detection result are reduced, the application range is narrow, the actual detection requirement of the driver cannot be met, the use experience of the driver is reduced, and the problem to be solved is urgent.
Disclosure of Invention
The application provides a human-caused intelligent cabin driver fatigue detection method, device, vehicle and medium to solve and detect the driving state of driver according to driver's facial expression and road information alone in the correlation technique, lead to fatigue detection mode and recognition result comparatively singleness, unable accurate discernment driver's actual driving state reduces the accuracy and the reliability of detection result, and application scope is narrower, can't satisfy driver's actual detection demand, reduces the problem such as driver's use experience.
An embodiment of a first aspect of the present application provides a method for detecting fatigue of a driver of an artificial intelligent cabin, including the following steps: collecting physiological data and behavior data of a driver; extracting at least one argument feature of said driver from said physiological data and said behavioral data; and identifying the actual fatigue state of the driver according to the at least one independent variable characteristic, and controlling the vehicle to carry out state reminding on the driver based on the actual fatigue state.
Through the technical scheme, at least one independent variable characteristic of the driver can be extracted by combining the physiological data and the behavior data of the driver, so that the actual fatigue state of the driver is identified by utilizing the at least one independent variable characteristic, the driver is further reminded of the state based on the actual fatigue state, the physiological data and the behavior data of the driver are comprehensively utilized to identify and remind the fatigue state of the driver, the purpose of multi-dimensional detection is achieved, the accuracy and the reliability of detection are improved, the applicability and the practicability of the detection are improved, the actual detection requirement of the driver is effectively met, the use experience of the driver is guaranteed, and the viscosity of a client is improved.
Illustratively, the collecting physiological data and behavioral data of the driver includes: collecting at least one change data of heart rate, brain electrical signals and electromyographic signals of the driver in the preset duration as the physiological data; and collecting at least one change data of the head gesture, blink frequency, eye opening and closing state and facial expression of the driver in a preset time period as the behavior data.
By the technical scheme, physiological data and behavior data of the driver within a certain period of time can be acquired, wherein the physiological data comprise at least one change data of heart rate, brain electrical signals and electromyographic signals, the behavior data comprise at least one change data of head gestures, blink frequency, eye opening and closing states and facial expressions, the data are used as data supports of subsequent steps, and the detection accuracy is effectively guaranteed, and the detection method is reliable and easy to realize.
Illustratively, the extracting at least one argument feature of the driver from the physiological data and the behavioral data comprises: preprocessing the physiological data and the behavior data to obtain processed acquisition data; and extracting at least one energy ratio index from the acquired data as the at least one independent variable characteristic.
By the technical scheme, physiological data and behavior data can be preprocessed, the effectiveness of the data is guaranteed, and redundant operation is reduced, so that at least one energy ratio index is extracted as at least one independent variable characteristic, the detection accuracy is further improved, invalid or redundant data is screened out, and the detection efficiency is effectively improved.
Illustratively, said identifying an actual fatigue state of said driver from said at least one argument feature comprises: inputting the at least one independent variable characteristic into a pre-constructed fatigue state identification model, and outputting the actual fatigue state, wherein the fatigue state identification model is jointly constructed by KSS scale scores before and after driving and multiple fatigue states.
Through the technical scheme, the at least one independent variable characteristic can be input into the fatigue state recognition model and the actual fatigue state is output, the fatigue state recognition model is pre-built through professional KSS scale grading and various fatigue states, the model is pre-trained in an off-line training stage by utilizing the recognition mode of the model, the convergence of the model is ensured, and the detection feasibility and reliability are further improved, and the method is reliable and easy to realize.
Illustratively, said identifying an actual fatigue state of said driver from said at least one argument feature comprises: acquiring the current working condition and/or the current environment of the vehicle; generating the weight of each independent variable characteristic in the at least one independent variable characteristic according to the current working condition and/or the current environment; and determining the actual fatigue state according to the at least one independent variable characteristic and the corresponding weight.
According to the technical scheme, the current working condition and/or the current environment of the vehicle can be obtained when the actual fatigue state of the driver is identified, the weight of each independent variable characteristic in at least one independent variable characteristic is generated, so that the actual fatigue state of the driver is determined, the identification applicability is improved, the physiological data and the behavior data of the driver are collected, the current working condition and/or the current environment of the vehicle are combined for determination, the detection accuracy is effectively improved, and the requirements of the driver are met.
Illustratively, the alerting the driver of the state based on the actual fatigue state includes: judging whether the actual fatigue state meets a preset reminding condition or not; and if the actual fatigue state meets the preset reminding condition, matching a target reminding type and a reminding signal according to the actual fatigue state, and controlling the vehicle to carry out state reminding based on the reminding signal according to the target reminding type.
Through the technical scheme, certain reminding conditions can be preset, so that when the actual fatigue state meets certain conditions, namely at necessary time, a driver is reminded of the state through the generated reminding type and the reminding signal, redundant operation is reduced, the use requirement of the driver in an actual scene is met, and the use experience of the driver is improved.
Illustratively, the preset alert condition is, but not limited to, that the fatigue level of the actual fatigue state is greater than a preset level.
Through the technical scheme, certain reminding conditions and fatigue grades can be preset, so that certain reminding conditions are judged to be met when the actual fatigue grade is greater than the certain fatigue grade, a driver is reminded, the driver is prevented from continuing to drive when the driver is overstired, safety accidents are caused, driving safety is guaranteed, the actual use requirements of the driver are met, and the viscosity of a client is guaranteed.
Embodiments of the second aspect of the present application provide a human factor intelligent cabin driver fatigue detection device, comprising: the acquisition module is used for acquiring physiological data and behavior data of the driver; an extraction module for extracting at least one argument feature of the driver from the physiological data and the behavioral data; and the detection module is used for identifying the actual fatigue state of the driver according to the at least one independent variable characteristic and controlling the vehicle to remind the driver of the state based on the actual fatigue state.
Through the technical scheme, the acquisition module can acquire physiological data and behavior data of the driver; the extraction module can extract at least one independent variable characteristic of the driver according to the physiological data and the behavior data; the detection module can identify the actual fatigue state of the driver according to at least one independent variable characteristic, controls the vehicle to carry out state reminding on the driver, combines physiological data and behavior data of the driver to identify and remind the fatigue state of the driver, realizes multidimensional detection, improves the accuracy and reliability of detection, and simultaneously improves the applicability and practicality of detection, effectively meets the actual detection requirement of the driver, effectively ensures the use experience of the driver, and improves the viscosity of clients.
Illustratively, the acquisition module includes: the first acquisition unit is used for acquiring at least one change data of heart rate, brain electrical signals and electromyographic signals of the driver in the preset duration as the physiological data; the second acquisition unit is used for acquiring at least one of change data of head gestures, blink frequency, eye opening and closing states and facial expressions of the driver in a preset duration as the behavior data.
Through the technical scheme, the first acquisition unit can acquire physiological data of the driver within a certain period of time, wherein the physiological data comprise at least one change data of heart rate, brain electrical signals and electromyographic signals; the second acquisition unit can acquire behavior data of the driver within a certain period of time, wherein the behavior data comprise at least one of change data of head gestures, blink frequency, eye opening and closing states and facial expressions, and the change data are used as support of follow-up data, so that the detection accuracy is effectively ensured, and the detection is reliable and easy to realize.
Illustratively, the extraction module includes: the first processing unit is used for preprocessing the physiological data and the behavior data to obtain processed acquisition data; and the extraction unit is used for extracting at least one energy ratio index from the acquired data as the at least one independent variable characteristic.
By the technical scheme, the first processing unit can preprocess the physiological data and the behavior data to obtain processed acquisition data, so that the validity of the data is ensured, and redundant operation is reduced; the extraction unit can extract at least one energy ratio index from the acquired data as at least one independent variable characteristic, so that the detection accuracy is further improved, invalid or redundant operation is screened out, and the detection efficiency is effectively improved.
Illustratively, the detection module includes: and the second processing unit is used for inputting the at least one independent variable characteristic into a pre-constructed fatigue state identification model and outputting the actual fatigue state, wherein the fatigue state identification model is jointly constructed by KSS scale scores before and after driving and a plurality of fatigue states.
Through the technical scheme, the second processing unit can input the at least one independent variable characteristic into the fatigue state recognition model and output the actual fatigue state, the fatigue state recognition model is constructed in advance through professional KSS scale scoring and various fatigue states, the model is trained in advance in an off-line training stage by utilizing the recognition mode of the model, the convergence of the model is ensured, and the detection feasibility is further improved, and the method is reliable and easy to realize.
Illustratively, the detection module includes: the acquisition unit is used for acquiring the current working condition and/or the current environment of the vehicle; the generating unit is used for generating the weight of each independent variable characteristic in the at least one independent variable characteristic according to the current working condition and/or the current environment; and the determining unit is used for determining the actual fatigue state according to the at least one independent variable characteristic and the corresponding weight.
Through the technical scheme, the acquisition unit can acquire the current working condition and/or the current environment of the vehicle; the generating unit can generate the weight of each independent variable characteristic in at least one independent variable characteristic according to the current working condition and/or the current environment; the determining unit can determine the actual fatigue state according to at least one independent variable characteristic and the corresponding weight, so that the applicability of identification is improved, the behavior data and the physiological data of the collected driver are determined by combining the current working condition and/or the current environment of the vehicle, the detection accuracy is effectively improved, and the requirements of the driver are met.
Illustratively, the detection module includes: the judging unit is used for judging whether the actual fatigue state meets a preset reminding condition or not; and the control unit is used for matching a target reminding type and a reminding signal according to the actual fatigue state when the actual fatigue state meets the preset reminding condition, and controlling the vehicle to carry out state reminding based on the reminding signal according to the target reminding type.
Through the technical scheme, the judging unit can judge whether the actual fatigue state meets certain reminding conditions; the control unit can control the vehicle to remind the driver of the state by matching the generated reminding type and the reminding signal when the actual fatigue state meets certain reminding conditions, so that redundant operation is reduced, the use requirement of the driver in an actual scene is met, and the use experience of the driver is improved.
Illustratively, the human factor intelligent cockpit driver fatigue detection apparatus further includes: the setting module is used for presetting a reminding condition that the fatigue grade of the actual fatigue state is larger than a preset grade.
Through the technical scheme, the setting module can preset certain reminding conditions and fatigue levels, so that the driver is reminded when the actual fatigue level is greater than the certain fatigue level, the driver is prevented from continuing driving when the driver is tired excessively, safety accidents are caused, driving safety is guaranteed, actual use requirements of the driver are met, and customer viscosity is guaranteed.
A third aspect of the present application provides a personal intelligence cabin comprising: the human-caused intelligent cabin driver fatigue detection device of the embodiment.
An embodiment of a fourth aspect of the present application provides a vehicle, including: the system comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to realize the human-caused intelligent cabin driver fatigue detection method according to the embodiment.
A fifth aspect of the present application provides a computer readable storage medium storing a computer program which when executed by a processor implements a human-induced intelligent cockpit driver fatigue detection method as above.
In the above embodiment, the physiological data and the behavior data of the driver are collected; extracting at least one argument feature of said driver from said physiological data and said behavioral data; the actual fatigue state of the driver is identified according to the at least one independent variable characteristic, and the vehicle is controlled to remind the driver of the state based on the actual fatigue state, so that the accuracy of fatigue state identification of the driver is improved, the reliability of an identification result is improved, the experience of the driver and the viscosity of the client are enhanced, and different requirements of the driver under different scenes are met. From this, solved among the correlation technique and detected driver's driving state according to driver's facial expression and road information alone, lead to tired detection mode and recognition result comparatively singleness, unable accurate discernment driver's actual driving state reduces the accuracy and the reliability of detection result, application scope is narrower, can't satisfy driver's actual detection demand, reduces the problem such as driver's use experience.
Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
fig. 1 is a flowchart of a human factor intelligent cabin driver fatigue detection method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of the present application for identifying and monitoring fatigue status of a driver;
fig. 3 is a schematic structural diagram of a human factor intelligent cabin driver fatigue detection device according to an embodiment of the present application;
fig. 4 is a schematic structural view of a vehicle according to an embodiment of the present application.
Description:
10-controlling device of air in the vehicle; 100-acquisition module, 200-extraction module and 300-detection module; 401-memory, 402-processor and 403-communication interface.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and intended for the purpose of explaining the present application and are not to be construed as limiting the present application.
The following describes a human-induced intelligent cabin driver fatigue detection method, a device, a vehicle and a medium according to the embodiments of the present application with reference to the accompanying drawings. Aiming at the problems that in the related art mentioned in the background technology center, the driving state of a driver is detected according to the facial expression and the road information of the driver alone, so that the fatigue detection mode and the recognition result are single, the actual driving state of the driver cannot be recognized accurately, the accuracy and the reliability of the detection result are reduced, the application range is narrow, the actual detection requirement of the driver cannot be met, and the use experience of the driver is reduced, the application provides a human-factor intelligent cabin driver fatigue detection method. Therefore, the problems that in the related art, the driving state of a driver is detected according to the facial expression and road information of the driver, so that the fatigue detection mode and the recognition result are single, the actual driving state of the driver cannot be accurately recognized, the accuracy and the reliability of the detection result are reduced, the application range is narrow, the actual detection requirement of the driver cannot be met, the use experience of the driver is reduced and the like are solved.
Specifically, fig. 1 is a flowchart of a method for detecting fatigue of a driver of a human-induced intelligent cabin according to an embodiment of the present application.
As shown in fig. 1, the fatigue detection method for the human-caused intelligent cockpit driver comprises the following steps:
in step S101, physiological data and behavioral data of the driver are collected.
It is understood that physiological data herein refers to physiological signals of the relevant human body, such as body temperature, heart rate, blood pressure, etc., that enable the status recognition of the driver; behavior data refers herein to data of related behaviors capable of performing state recognition of the driver, such as some gestures of the head, or some actions of the hand, or the like.
According to the method and the device for detecting the physiological data of the driver, the physiological data and the behavior data of the driver can be collected, and because the physiological and behavior indexes of the driver can change to a certain extent in the long-time driving process, the physiological data and the behavior indexes of the driver are collected to serve as the basis for subsequent data processing, and the physiological data and the behavior indexes of the driver can be further divided, so that the detection accuracy is improved.
As one example, collecting physiological data and behavioral data of a driver includes: collecting at least one change data of heart rate, brain electrical signals and electromyographic signals of a driver in a preset time period as physiological data; at least one of change data of head posture, blink frequency, eye opening and closing state and facial expression of a driver in a preset time period is collected and used as behavior data.
It will be appreciated by those skilled in the art that physiological data, as referred to herein, is a physiological signal that may be used to measure a person's mental state, including, but not limited to: the behavior data refers to data records of behaviors of a human body and when the behaviors occur, and refer to behaviors generated by a driver in a driving process and recorded data; the preset duration is a preset certain time, for example, ten minutes.
The fatigue state of the driver can be effectively identified through various physiological and behavioral indexes, such as brain waves, eye movements, head gestures, heart rate variability and the like; in the long-time driving process, physiological and behavioral indexes of a driver can change to a certain extent, such as changes of eye movement frequency, head posture and heart rate variability, and the like, the changes are positively correlated with fatigue states, and an effective fatigue state identification model can be constructed by collecting and analyzing various physiological and behavioral indexes of the driver.
By way of example, heart rate refers to the average of the heart rate times, and the heart rate of drivers of different ages, sexes, health conditions and habits also varies, and the heart rate of drivers in different driving states, for example, the heart rate in a normal driving state and the heart rate in a fatigue driving state, so that the heart rate of the driver in ten minutes during driving can be collected and used as a piece of physiological data of the driver.
For example, electroencephalogram (EEG) is an electrical signal generated by brain neuron activity, and the electroencephalogram signal of a driver during driving varies with different activities, and after the electroencephalogram signals of different drivers are collected, the electroencephalogram signals of different drivers are processed by using an artificial neural network, mainly the typical characteristics of different electroencephalograms of different wavebands are extracted and classified, and thus, whether the driver is tired or not is judged, so that the electroencephalogram signal of the driver during driving can be collected and used as a physiological data of the driver.
Illustratively, the electromyographic signal (EMG) is a superposition of the Motion Unit Action Potentials (MUAP) in numerous muscle fibers in time and space. The surface electromyogram Signal (SEMG) is the comprehensive effect of the electric activity of the superficial muscle EMG and the nerve trunk on the skin surface, can reflect the activity of nerve muscles to a certain extent, is an electric signal accompanied by muscle contraction, and can also carry out different contraction activities along with different actions of a body in the driving process of a driver, so that the electromyogram signal can be changed along with the different actions of the body, and can be used for collecting the electromyogram signal of the driver in ten minutes in the driving process and be used as physiological data of the driver.
Illustratively, head gestures are generally divided into three categories: raising, shaking and turning. The head movement of the driver during driving is understood to mean that, for example, the head may hang down involuntarily or the head may be deviated sideways when the driver is drowsy, so that the head posture of the driver during driving within ten minutes may be collected and used as a behavior data of the driver.
For example, the blink frequency is the number of blinks of the driver in a certain period of time during driving, and the number of blinks, the time of slow blinks and the number of times of the human being are different from those in a normal state when the human being is tired, so that the blink frequency of the driver during driving, for example, the number of blinks of the driver in ten minutes, can be collected and used as a behavior data of the driver.
The eye opening and closing state is a state of eyes of the driver during driving, and whether the driver is in a tired state is determined, for example, a detected closing time of the eyes of the driver in a unit time (generally 1min or 30 s) can be calculated, and if the eyes are closed for about 80% of the time in the time period, the driver can be determined to be in the tired state at the moment; it is also possible to calculate the actual pixel values of the detected eyes of the driver in the lateral and longitudinal directions, and to calculate the aspect ratio of the eyes, which is relatively fixed for the open or closed state of the same person, but the different persons have in common this value: i.e. the eye closure value is small (less than 0.3), e.g. the driver has a thirty second duration in a state where the aspect ratio of the eyes is less than 0.3 in a minute, then the driver is in a tired state at this time. The open/close state of the eyes of the driver in one minute during driving can be acquired and used as a behavior data of the driver.
For example, the facial expression is an expression of the driver, which is greatly different from the normal driving state in the process of driving, such as yawning, foggy, or the physiological tear due to fatigue flow, and the like, and is in a fatigue state at this time, so that the facial expression of the driver in ten minutes during driving can be collected and used as a behavior data of the driver.
According to the method and the device for detecting the fatigue state of the driver, different physiological data and behavior data of the driver in a certain period of time can be collected, the fatigue state of the driver is identified through enough physiological data and behavior data to serve as a data support for the subsequent steps, the detection accuracy is effectively guaranteed, and the identification realizability and the result reliability are improved.
Step S102, at least one independent variable characteristic of the driver is extracted according to the physiological data and the behavior data.
It is understood that the independent variable refers to a factor or condition that is manually actively controllable to cause a change in the dependent variable, and the independent variable can be regarded as a factor of the dependent variable, for example, the independent variable is that the driver is closed for a long time, and the dependent variable is that the system determines that the driver is in fatigue driving, and the factor is that the driver is closed for a long time, and the result is that the driver is determined to be in fatigue driving.
For example, the present application may extract at least one independent variable feature of the driver according to at least one change data of the collected physiological data such as heart rate, brain electrical signal, electromyographic signal, and the like in a certain period of time, and at least one change data of the behavior data such as head posture, blink frequency, eye opening and closing state, and facial expression, and the like in a certain period of time.
According to the method and the device for detecting the vehicle body, the at least one independent variable characteristic of the driver can be extracted from the physiological data and the behavior data, and the next detection is carried out according to the independent variable characteristic, so that the accuracy of detection is effectively ensured.
As one example, extracting at least one argument feature of a driver from physiological data and behavioral data, comprises: preprocessing physiological data and behavior data to obtain processed acquisition data; at least one energy ratio indicator is extracted from the acquired data as at least one argument feature.
Illustratively, data preprocessing: EOG artifact removal-Independent Component Analysis (ICA), noise frequency domain filtering-bandpass filtering, data downsampling-Nyquist sampling theorem may be utilized, but are not limited to.
Illustratively, the characteristic index is screened: and extracting a characteristic index which can most represent the driving fatigue from the preprocessed collected data, wherein the energy ratio index can effectively represent the fatigue state of a driver. Extracting energy characteristics by frequency decomposition of the rhythmic wave through wavelet packet transformation; searching an index which is most relevant to the fatigue state by utilizing grey correlation analysis, and screening out the preferable characteristics for researching the driving fatigue.
Illustratively, feature difference significance analysis: the validity analysis of the fatigue characteristic parameters of the driver is to check whether the parameters have obvious differences in different fatigue states of the driver. And calculating the average value of the parameters under different fatigue states, and comparing the differences between the average values to judge. Analysis of variance (Analysis of Variance) is used to verify whether this characteristic parameter varies from driver to driver. According to analysis of variance theory, the differences in characteristic parameters between different states are derived from two aspects:
(1) The parameter error caused by fatigue, i.e. the difference caused by the variation of the fatigue state of the driver, is called inter-group error. By calculating the sum of squares of the deviations between the mean value of the characteristic parameters in each group and the total mean value of the samples, denoted E Inter-group 。
(2) Random errors, i.e. deviations caused by disturbances in feature extraction, are called intra-group differences. Represented by summing the squares of the average value of the characteristic parameter within each group and the difference between the parameters within the group, denoted E Within a group 。
E Inter-group And E is Within a group Dividing the mean square value by the degree of freedom of each of the two groups to obtain mean square values, wherein the mean square values are respectively: inter-group Mean Square (MS) Inter-group ) Sum-intra-group Mean Square (MS) Within a group ). When the two parameters satisfy the following first formula, the different fatigue states have no effect on the parameter values, i.e. each sample comes from a population. When different fatigue states have a more obvious effect on the extracted characteristic parameter values, MS Inter-group Is caused by both driver fatigue and random errors, i.e. the samples originate from different populations. At this time MS Inter-group And MS (MS) Within a group The relationship of (2) is shown in the second formula below.
MS Inter-group >>MS Within a group
At this time, MS Inter-group And MS (MS) Within a group The ratio constitutes an F distribution, and comparing the F value with its threshold value can determine whether each sample is derived from the same population. Assuming n driver fatigue samples, the original assumption is set as: the mean value is the same for each fatigue state, namely:
u 1 =u 2 =u 3 =u
wherein u is i Average value indicating the ith fatigue state. The original assumption indicates that n samples originate from one population (i.e., μ and σ are the same). If the result of the calculation appears in the second formula, namely from MS Within a group Far smaller than MS Inter-group The method comprises the steps of carrying out a first treatment on the surface of the Then, it is explained that different fatigue states of the driver cause differences between the characteristic parameters and the average value, namely:
F>F 0.05 (f Inter-group ,f Within a group )
Where f is the degree of freedom in brackets, p <0.05 is considered to be statistically different, p <0.01 is significantly statistically different, and p <0.001 is extremely significantly statistically different.
And finally, counting the effective data for use.
According to the method and the device, the physiological data and the behavior data can be preprocessed before at least one independent variable characteristic is extracted from the physiological data and the behavior data, the validity of the detected data is guaranteed, invalid or redundant data is screened out, at least one energy ratio index is extracted from the obtained valid data and used as at least one independent variable characteristic, the data detection efficiency is effectively improved, and the accuracy of the data and the validity of the data are guaranteed.
Step S103, the actual fatigue state of the driver is identified according to at least one independent variable characteristic, and the vehicle is controlled to carry out state reminding on the driver based on the actual fatigue state.
For example, after at least one independent variable characteristic is obtained as described in the above example, the actual fatigue state of the driver may be identified, for example, the heart rate variation data collected and preprocessed for a certain period of time is used as the independent variable characteristic, and when the driver drives for a long time, the heart rate may decrease, so that it may be identified that the driver is in the fatigue state.
For example, after recognizing that the driver is in a tired state, the tired state of the driver may be further classified, and defined as an actual tired state, and the driver may be reminded according to the actual tired state of the driver, for example, by performing a corresponding state voice alarm through a voice alarm system, or by performing a reminder through icons or characters of an in-vehicle display screen, adjustment of in-vehicle lights, and the like.
According to the method and the device for identifying the actual fatigue state of the driver, the actual fatigue state of the driver can be identified through at least one independent variable characteristic, the vehicle is controlled to conduct state reminding to the driver according to the identification result, accuracy of the identification result is guaranteed, relevant reminding is conducted, fatigue driving of the driver is avoided, and use requirements of actual scenes of the driver are met.
As one example, identifying an actual fatigue state of the driver from the at least one argument feature comprises: inputting at least one independent variable characteristic into a pre-constructed fatigue state identification model, and outputting an actual fatigue state, wherein the fatigue state identification model is jointly constructed by KSS scale scores before and after driving and multiple fatigue states.
It can be appreciated based on other related embodiments that the fatigue state recognition model herein is a model built in advance through training and is built up from a KSS scale score before and after driving and a plurality of fatigue states together. As shown in fig. 2, for a schematic diagram of fatigue state recognition and monitoring of a driver in the present application, the method for constructing a fatigue state recognition model may be as follows:
Illustratively, model variable control: independent variable: head posture change, blink frequency change, eye opening and closing state change, facial expression change, heart rate change, electroencephalogram signal change, electromyographic signal change and the like; dependent variables: fatigue status (non-fatigue, mild fatigue, moderate fatigue, severe fatigue).
Illustratively, model objects and devices: 9 adults with driving experience were chosen as model subjects and the subjects were required to meet the following conditions: the ages are 18-60 years old, the body is healthy, and the disease history in psychological, nerve or vision and the like is avoided; the system comprises a driving simulation system x 1, an electroencephalograph x 1, an EEG electroencephalograph analysis module x 1, an eye movement analyzer x 1, a General basic physiological analyzer x 1, a wireless high-precision physiological recording system x 1, a camera system x 1, a head posture detector x 1, a wearable surface myoelectricity measurement system x 1, an EMG myoelectricity analysis module x 1, a computer system x 1 and a stopwatch x 3.
Illustratively, model data acquisition: after the operation explanation and notice explanation are carried out on the model object, a tester sets a test condition, clearly acquires the data type, and after the tester completes debugging and assembling of the data acquisition device, the tester wears the data acquisition equipment on the model object, so that the physiological signal acquisition equipment is well connected and the continuous acquisition of behavior and physiological data is started after a video system is opened.
Wherein, test condition sets up: simulating a driving environment: long-time single-speed high-speed driving, urban road driving, rural road driving and the like; driving time: 30-240min; fatigue degree: consciousness, mild fatigue, moderate fatigue, severe fatigue; model object own characteristics: age, sex, age of drive, driving experience, seat angle adjustment, etc. The type of data collected: behavior data: head posture change, blink frequency change, eye opening and closing state change, facial expression change, and the like; physiological data: heart rate changes, brain electrical signal changes, electromyographic signal changes, and the like.
Illustratively, the fatigue state labeling method: and recording the fatigue state and the generation time of the model object in the driving process by adopting a mode of combining a model object subjective evaluation kalina sleep scale (Karolinska Sleepiness Scale, KSS) and a state recording button, so as to determine different fatigue state labels corresponding to the acquired data samples.
Five grades with scores of 1 to 5 are taken as a section in which a driver is in a normal state, and 6 to 10 are taken as sections for judging fatigue, wherein 6 and 7 belong to mild fatigue, 8 belong to moderate fatigue, and 9 and 10 belong to severe fatigue. The model object is combined with the KSS scale to judge the change of the state of the model object, when the state is mild fatigue, moderate fatigue and severe fatigue respectively, a recorder is respectively indicated, the recorder records time nodes, the segmentation of signals such as electroencephalogram, electrocardiograph and the like is facilitated, after the model object achieves the severe fatigue, the data is continuously collected for 10min, the collected data is stored and exported, and the data collection is completed.
It should be noted that the subjective scale is used as an auxiliary means, and mainly adopts a state record labeling method. The model object needs to carry out subjective evaluation of a KSS scale before and after the simulated driving, and the time points of the state change of the model object are recorded and combined with the states before and after the driving displayed by the scale, so that the model object is in four different states and respectively corresponds to the time periods, the acquired data are marked, and finally the fatigue state recognition model is obtained.
Wherein, the grade of fatigue is divided into 10 levels in the KSS scale, the higher the grade is, the deeper the fatigue degree is, and the table 1 is the grade of the KSS scale and various fatigue state tables, as follows:
TABLE 1
Wherein 1 means extremely alert, 2 means very alert, 3 means alert, 4 means somewhat alert, 5 means neither alert nor somnolence, 6 means some sign of somnolence, 7 means somnolence but still awake, 8 means somnolence and requires effort to stay awake, 9 means very somnolence and requires effort to stay awake and try to overcome and 10 means extremely somnolence has not been able to stay awake; further divided by state: 1-5 is non-fatigue, 6-7 is mild fatigue, 8 is moderate fatigue and 9-10 indicates severe fatigue.
According to the embodiment of the application, at least one independent variable characteristic can be input into the fatigue state identification model which is built through pre-training, the actual fatigue state is output, the fatigue state identification model is pre-built through professional KSS scale scoring and various fatigue states, different fatigue grades are divided, the convergence and reliability of the model are guaranteed through offline training, and the feasibility and reliability of the application are further improved.
As one example, identifying an actual fatigue state of the driver from the at least one argument feature comprises: acquiring the current working condition and/or the current environment of the vehicle; generating the weight of each independent variable characteristic in at least one independent variable characteristic according to the current working condition and/or the current environment; and determining the actual fatigue state according to at least one independent variable characteristic and the corresponding weight.
It is understood that the current working condition of the vehicle refers to the condition that the vehicle is running on the current road, for example, whether the vehicle is driven at a monotonic high speed, or is driven on an urban road, or is driven on a rural road, etc.
For example, the current operating conditions and the current environment of the vehicle may have an impact on the driving state of the driver. For example, in a state where the vehicle is in a current condition, such as monotonically driving at a high speed, the driver is in a relaxed state, and the frequency of electromyography at this time shows a decreasing trend, but the generation of fatigue and the increase of the fatigue degree are not performed; in contrast, when the vehicle is under intense light, the eyes of the driver cannot be consciously squinted for a long time due to intense light, and the closing degree of the eyes is deepened, but the driver does not feel sleepiness, namely, is not moderately tired.
For example, since different working conditions and different environments affect physiological data and behavioral data of the driver, the weight of each of the at least one independent variable feature may be generated according to the current working condition and/or the current environment, for example, the weight of the myoelectric signal needs to be reduced when the current working condition is better, the weight of the open/close state of the eyes needs to be reduced when the current working condition is under strong light irradiation, and then the actual fatigue state is determined according to the at least one independent variable feature and the corresponding weight.
According to the method and the device for determining the actual fatigue state of the driver, on the basis of collecting the physiological data and the behavior data of the driver, the current working condition and/or the current environment of the vehicle are further combined, the actual fatigue state of the driver is determined, the detection accuracy is effectively improved, and different requirements of the driver in actual use are met.
As one example, status alerting the driver based on actual fatigue status includes: judging whether the actual fatigue state meets a preset reminding condition or not; if the actual fatigue state meets the preset reminding condition, the target reminding type and the reminding signal are matched according to the actual fatigue state, and the vehicle is controlled to carry out state reminding based on the reminding signal according to the target reminding type.
It can be understood that the preset reminding condition can be understood as a certain condition which is preset and finished and can control the vehicle to remind the driver, for example, the driver is in a moderate fatigue driving state.
The preset reminding condition is that the fatigue level of the driver is equal to or higher than the moderate fatigue, and then when the actual fatigue state of the driver reaches the moderate fatigue, the driver can be judged to meet a certain reminding condition, at the moment, the target reminding type and the reminding signal can be matched according to the actual fatigue state of the driver, and the vehicle is controlled to carry out state reminding based on the reminding signal.
For example, the actual fatigue state of the driver is mild fatigue, the matched target reminding type is a voice alarm type, and the reminding signal is a broadcasting reminding, so that the vehicle can be controlled to broadcast through the loudspeaker in the vehicle by voice: "the driver is your good, detects that you have continuously driven for more than 3 hours at present, and continued driving may cause fatigue driving, please note rest-! "; for another example, the driver is in a moderate fatigue state, the matched target reminding type is an action type, the reminding signal is seat adjustment, and the more comfortable seat angle to which the driver is adjusted at daily can be adjusted, for example, the backrest is slightly lifted by 15 degrees, so that the driver is reminded.
According to the method and the device, the reminding conditions of certain reminding can be preset, so that when the actual fatigue state meets certain conditions, the driver is reminded of the state through the generated reminding type and the reminding signal, the driver is reminded at the key moment, the possibility of safety accident occurrence is avoided, the experience of the driver is improved, and the use requirement of the driver in an actual scene is met.
As one example, the preset alert condition is that the fatigue level of the actual fatigue state is greater than the preset level.
It can be understood that the preset level can be understood as a certain fatigue state level which is preset and needs to be reminded.
For example, the preset level is light fatigue, and when the driver is in a fatigue state and the fatigue level is moderate fatigue or heavy fatigue, the fatigue level is larger than the preset level, that is, a certain reminding condition is met, and a certain state reminding can be performed on the driver.
According to the embodiment of the application, certain reminding conditions and fatigue levels can be preset, so that certain reminding conditions are judged to be met when the actual fatigue level is greater than the certain fatigue level, a driver is reminded, driving safety is guaranteed, experience of the driver is improved, and customer viscosity is guaranteed.
According to the human factor intelligent cabin driver fatigue detection method provided by the embodiment of the application, physiological data and behavior data of a driver within a certain period of time can be collected and preprocessed, at least one independent variable characteristic is extracted and input into the fatigue state recognition model constructed in advance, so that the actual fatigue state of the driver is recognized, the vehicle is controlled to carry out relevant state reminding, the accuracy of fatigue state recognition of the driver is improved, the reliability of recognition results is improved, the experience of the driver and the viscosity of a client are enhanced, and different requirements of the driver under different scenes are met. Therefore, the problems that in the related art, the driving state of a driver is detected according to the facial expression and road information of the driver, so that the fatigue detection mode and the recognition result are single, the actual driving state of the driver cannot be accurately recognized, the accuracy and the reliability of the detection result are reduced, the application range is narrow, the actual detection requirement of the driver cannot be met, the use experience of the driver is reduced and the like are solved.
Next, a human factor intelligent cabin driver fatigue detection device according to an embodiment of the present application will be described with reference to the accompanying drawings.
Fig. 3 is a block schematic diagram of a human factor intelligent cockpit driver fatigue detection apparatus according to an embodiment of the present application.
As shown in fig. 3, the human-caused intelligent cockpit driver fatigue detection apparatus 10 includes: the device comprises an acquisition module 100, an extraction module 200 and a detection module 300.
The acquisition module 100 is used for acquiring physiological data and behavior data of a driver.
The extraction module 200 is used for extracting at least one independent variable characteristic of the driver according to the physiological data and the behavior data.
The detection module 300 is configured to identify an actual fatigue state of the driver according to at least one argument feature, and control the vehicle to perform a state alert to the driver based on the actual fatigue state.
Optionally, in one embodiment of the present application, the acquisition module 100 includes: a first acquisition unit and a second acquisition unit.
The first acquisition unit is used for acquiring at least one change data of heart rate, brain electrical signals and electromyographic signals of a driver in a preset duration and taking the change data as physiological data.
The second acquisition unit is used for acquiring at least one of change data of head gestures, blink frequency, eye opening and closing states and facial expressions of the driver in a preset time period as behavior data.
Optionally, in one embodiment of the present application, the extraction module 200 includes: a first processing unit and an extraction unit.
The first processing unit is used for preprocessing the physiological data and the behavior data to obtain processed acquisition data.
And the extraction unit is used for extracting at least one energy ratio index from the acquired data as at least one independent variable characteristic.
Optionally, in one embodiment of the present application, the detection module 300 includes: and a second processing unit.
The second processing unit is used for inputting at least one independent variable characteristic into a pre-constructed fatigue state identification model and outputting an actual fatigue state, wherein the fatigue state identification model is jointly constructed by KSS scale scores before and after driving and a plurality of fatigue states.
Optionally, in one embodiment of the present application, the detection module 300 includes: an acquisition unit, a generation unit and a determination unit.
The acquisition unit is used for acquiring the current working condition and/or the current environment of the vehicle.
And the generating unit is used for generating the weight of each independent variable characteristic in at least one independent variable characteristic according to the current working condition and/or the current environment.
And the determining unit is used for determining the actual fatigue state according to at least one independent variable characteristic and the corresponding weight.
Optionally, in one embodiment of the present application, the detection module 300 includes: a judging unit and a control unit.
The judging unit is used for judging whether the actual fatigue state meets the preset reminding condition.
And the control unit is used for matching the target reminding type and the reminding signal according to the actual fatigue state when the actual fatigue state meets the preset reminding condition, and controlling the vehicle to carry out state reminding based on the reminding signal according to the target reminding type.
Optionally, in one embodiment of the present application, the human-induced intelligent cockpit driver fatigue detection apparatus further includes: and setting a module.
The setting module is used for presetting a reminding condition that the fatigue grade of the actual fatigue state is larger than a preset grade.
It should be noted that the foregoing explanation of the embodiment of the method for detecting fatigue of a human-derived intelligent cockpit driver is also applicable to the device for detecting fatigue of a human-derived intelligent cockpit driver of this embodiment, and will not be repeated here.
According to the human factor intelligent cabin driver fatigue detection device provided by the embodiment of the application, physiological data and behavior data of a driver in a certain time period can be collected and preprocessed, at least one independent variable characteristic is extracted and input into the fatigue state recognition model constructed in advance, so that the actual fatigue state of the driver is recognized, the vehicle is controlled to carry out relevant state reminding, the accuracy of fatigue state recognition of the driver is improved, the reliability of recognition results is improved, the experience of the driver and the viscosity of a client are enhanced, and different requirements of the driver under different scenes are met. From this, solved among the correlation technique and detected driver's driving state according to driver's facial expression and road information alone, lead to tired detection mode and recognition result comparatively singleness, unable accurate discernment driver's actual driving state reduces the accuracy and the reliability of detection result, application scope is narrower, can't satisfy driver's actual detection demand, reduces the problem such as driver's use experience.
The embodiment of the application also provides the artificial intelligence cabin, which comprises the artificial intelligence cabin driver fatigue detection device. The human-based intelligent cabin can collect and preprocess physiological data and behavior data of a driver for a certain period of time, at least one independent variable characteristic is extracted and input into a fatigue state recognition model constructed in advance, so that the actual fatigue state of the driver is recognized, a vehicle is controlled to carry out relevant state reminding, the accuracy of fatigue state recognition of the driver is improved, the reliability of recognition results is improved, the experience of the driver and the viscosity of a client are enhanced, and different requirements of the driver in different scenes are met. The method solves the problems that in the related art, the driving state of a driver is detected independently according to the facial expression and road information of the driver, so that the fatigue detection mode and the recognition result are single, the actual driving state of the driver cannot be accurately recognized, the accuracy and the reliability of the detection result are reduced, the application range is narrow, the actual detection requirement of the driver cannot be met, the use experience of the driver is reduced and the like.
Fig. 4 is a schematic structural diagram of a vehicle according to an embodiment of the present application. The vehicle may include:
Memory 401, processor 402, and a computer program stored on memory 401 and executable on processor 402.
The processor 402 implements the human factor intelligent cockpit driver fatigue detection method provided in the above embodiment when executing a program.
Further, the vehicle further includes:
a communication interface 403 for communication between the memory 401 and the processor 402.
A memory 401 for storing a computer program executable on the processor 402.
Memory 401 may comprise high-speed RAM memory or may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
If the memory 401, the processor 402, and the communication interface 403 are implemented independently, the communication interface 403, the memory 401, and the processor 402 may be connected to each other by a bus and perform communication with each other. The bus may be an industry standard architecture (Industry Standard Architecture, abbreviated ISA) bus, an external device interconnect (Peripheral ComponentInterconnect, abbreviated PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one thick line is shown in fig. 4, but not only one bus or one type of bus.
Alternatively, in a specific implementation, if the memory 401, the processor 402, and the communication interface 403 are integrated on a chip, the memory 401, the processor 402, and the communication interface 403 may complete communication with each other through internal interfaces.
The processor 402 may be a central processing unit (Central Processing Unit, abbreviated as CPU), or an application specific integrated circuit (Application Specific Integrated Circuit, abbreviated as ASIC), or one or more integrated circuits configured to implement embodiments of the present application.
The embodiment of the application also provides a computer readable storage medium, on which a computer program is stored, which when being executed by a processor, realizes the human factor intelligent cabin driver fatigue detection method.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or N embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "N" is at least two, such as two, three, etc., unless explicitly defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and additional implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order from that shown or discussed, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or N wires, a portable computer cartridge (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the N steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, may be implemented in a combination of any one or more of the following techniques, which are well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product.
The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like. Although embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives, and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.
Claims (11)
1. The human-caused intelligent cabin driver fatigue detection method is characterized by comprising the following steps of:
collecting physiological data and behavior data of a driver;
extracting at least one argument feature of said driver from said physiological data and said behavioral data; and
and identifying the actual fatigue state of the driver according to the at least one independent variable characteristic, and controlling the vehicle to carry out state reminding on the driver based on the actual fatigue state.
2. The human-caused intelligent cockpit driver fatigue detection method according to claim 1, wherein the collecting physiological data and behavioral data of the driver includes:
collecting at least one change data of heart rate, brain electrical signals and electromyographic signals of the driver in a preset time period as the physiological data;
And collecting at least one change data of the head gesture, blink frequency, eye opening and closing state and facial expression of the driver in a preset time period as the behavior data.
3. The human-caused intelligent cockpit driver fatigue detection method according to claim 1, wherein the extracting at least one independent variable feature of the driver according to the physiological data and the behavioral data comprises:
preprocessing the physiological data and the behavior data to obtain processed acquisition data;
and extracting at least one energy ratio index from the acquired data as the at least one independent variable characteristic.
4. The human-derived intelligent cockpit driver fatigue detection method of claim 1 wherein said identifying an actual fatigue state of the driver from the at least one argument feature comprises:
inputting the at least one independent variable characteristic into a pre-constructed fatigue state identification model, and outputting the actual fatigue state, wherein the fatigue state identification model is jointly constructed by KSS scale scores before and after driving and multiple fatigue states.
5. The method of claim 1, wherein said identifying an actual fatigue state of the driver from the at least one argument feature comprises:
Acquiring the current working condition and/or the current environment of the vehicle;
generating the weight of each independent variable characteristic in the at least one independent variable characteristic according to the current working condition and/or the current environment;
and determining the actual fatigue state according to the at least one independent variable characteristic and the corresponding weight.
6. The method of claim 1, wherein the alerting the driver of the status based on the actual fatigue status comprises:
judging whether the actual fatigue state meets a preset reminding condition or not;
and if the actual fatigue state meets the preset reminding condition, matching a target reminding type and a reminding signal according to the actual fatigue state, and controlling the vehicle to carry out state reminding based on the reminding signal according to the target reminding type.
7. The method of claim 6, wherein the predetermined alert condition is that the actual fatigue state has a fatigue level greater than a predetermined level.
8. Human factor intelligence cockpit driver fatigue detection device, its characterized in that includes:
the acquisition module is used for acquiring physiological data and behavior data of the driver;
An extraction module for extracting at least one argument feature of the driver from the physiological data and the behavioral data; and
and the detection module is used for identifying the actual fatigue state of the driver according to the at least one independent variable characteristic and controlling the vehicle to remind the driver of the state based on the actual fatigue state.
9. An artificial intelligence cabin, characterized by comprising: the human factor intelligent cockpit driver fatigue detection device of claim 8.
10. A vehicle, characterized by comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the human-induced intelligent cockpit driver fatigue detection method of any of claims 1-7.
11. A computer readable storage medium having stored thereon a computer program, characterized in that the program is executed by a processor for implementing the human-induced intelligent cockpit driver fatigue detection method according to any of claims 1-7.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311865457.4A CN117770821A (en) | 2023-12-29 | 2023-12-29 | Human-caused intelligent cabin driver fatigue detection method, device, vehicle and medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311865457.4A CN117770821A (en) | 2023-12-29 | 2023-12-29 | Human-caused intelligent cabin driver fatigue detection method, device, vehicle and medium |
Publications (1)
Publication Number | Publication Date |
---|---|
CN117770821A true CN117770821A (en) | 2024-03-29 |
Family
ID=90398276
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202311865457.4A Pending CN117770821A (en) | 2023-12-29 | 2023-12-29 | Human-caused intelligent cabin driver fatigue detection method, device, vehicle and medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN117770821A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN118277835A (en) * | 2024-05-29 | 2024-07-02 | 惠州华阳通用电子有限公司 | Driving fatigue recognition method and system based on large model |
-
2023
- 2023-12-29 CN CN202311865457.4A patent/CN117770821A/en active Pending
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN118277835A (en) * | 2024-05-29 | 2024-07-02 | 惠州华阳通用电子有限公司 | Driving fatigue recognition method and system based on large model |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Halim et al. | On identification of driving-induced stress using electroencephalogram signals: A framework based on wearable safety-critical scheme and machine learning | |
WO2021174618A1 (en) | Training method for electroencephalography mode classification model, classification method and system | |
Wang et al. | A novel real-time driving fatigue detection system based on wireless dry EEG | |
Huo et al. | Driving fatigue detection with fusion of EEG and forehead EOG | |
Gurudath et al. | Drowsy driving detection by EEG analysis using wavelet transform and K-means clustering | |
CN109976525B (en) | User interface interaction method and device and computer equipment | |
Huang et al. | Recognition of driver’s mental workload based on physiological signals, a comparative study | |
Sahayadhas et al. | Physiological signal based detection of driver hypovigilance using higher order spectra | |
Sengupta et al. | A multimodal system for assessing alertness levels due to cognitive loading | |
El-Nabi et al. | Machine learning and deep learning techniques for driver fatigue and drowsiness detection: a review | |
US10877444B1 (en) | System and method for biofeedback including relevance assessment | |
Guettas et al. | Driver state monitoring system: A review | |
Katsis et al. | Emotion recognition in car industry | |
CN117770821A (en) | Human-caused intelligent cabin driver fatigue detection method, device, vehicle and medium | |
Arefnezhad et al. | Driver drowsiness classification using data fusion of vehicle-based measures and ECG signals | |
Allam et al. | Customized deep learning algorithm for drowsiness detection using single-channel EEG signal | |
Li et al. | Driver distraction from the EEG perspective: A review | |
Jiao et al. | Detecting slow eye movements with bimodal-LSTM for recognizing drivers’ sleep onset period | |
Esteves et al. | AUTOMOTIVE: a case study on AUTOmatic multiMOdal drowsiness detecTIon for smart VEhicles | |
Pan et al. | A vigilance estimation method for high-speed rail drivers using physiological signals with a two-level fusion framework | |
Baccour et al. | Comparative analysis of vehicle-based and driver-based features for driver drowsiness monitoring by support vector machines | |
Thorslund | Electrooculogram analysis and development of a system for defining stages of drowsiness | |
Bulygin et al. | Image-Based Fatigue Detection of Vehicle Driver: State-of-the-Art and Reference Model | |
CN113598773A (en) | Data processing device and method for evaluating user discomfort | |
Caryn et al. | Driver drowsiness detection based on drivers’ physical behaviours: a systematic literature review |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |