CN119027923A

CN119027923A - A driver attention monitoring method and system based on artificial intelligence

Info

Publication number: CN119027923A
Application number: CN202411482971.4A
Authority: CN
Inventors: 胡强; 任浩; 宁平; 刘云剑; 黄飞
Original assignee: Changsha Chaochuang Electronic Technology Co ltd
Current assignee: Changsha Chaochuang Electronic Technology Co ltd
Priority date: 2024-10-23
Filing date: 2024-10-23
Publication date: 2024-11-26
Anticipated expiration: 2044-10-23
Also published as: CN119027923B

Abstract

The invention discloses a driver attention monitoring method and system based on artificial intelligence, wherein the method comprises the following steps: extracting a direction histogram characteristic of the front image of the driver and classifying the front image of the driver by a support vector machine to obtain a face area image of the driver; detecting an eye region and a mouth region in a face region image of a driver, and outputting center position coordinates of a left eye, a right eye and a mouth; calculating a horizontal rotation angle, a pitching angle and a rolling angle of the head of the driver according to the central position coordinates; constructing an attention area classification model based on the horizontal rotation angle, the pitching angle and the rolling angle of the head of the driver to obtain an attention area of the driver; the horizontal rotation angle, the pitching angle, the rolling angle and the attention area of the head of the driver are processed by using the artificial intelligent driven long-short-time memory network, the distraction is monitored, and the monitoring result is output. The invention can more accurately identify the distraction behavior by analyzing the continuous change of the head gesture of the driver.

Description

Driver attention monitoring method and system based on artificial intelligence

Technical Field

The invention relates to the technical field of driving safety, in particular to a driver attention monitoring method and system based on artificial intelligence.

Background

Along with the rapid development of intelligent driving technology and the continuous increase of the quantity of automobile conservation, the driver state monitoring technology has important significance in improving driving safety and reducing the occurrence rate of traffic accidents, and particularly in long-time driving and complex road environments, potential dangerous behaviors can be timely found and early-warned through real-time monitoring of the attention of a driver, so that driving safety is ensured.

Currently, a common driver attention monitoring method mainly relies on continuous shooting of facial expressions or eye states of a driver by a camera, and analysis of shot data is performed by using a rule-based or traditional machine learning method. However, most monitoring methods are based on static images, and only the attention state of the driver at a certain moment can be detected, but the attention change of the driver in a time sequence is not involved, misjudgment or missed judgment is easy to occur, and the problem of persistent distraction cannot be accurately identified.

In addition, many existing monitoring methods only focus on the facial or eye area of the driver, and the external appearance features are various when the driver is not focused, so that more information needs to be introduced to improve the accuracy of the monitoring result. For example, the posture change information of the head of the driver, including but not limited to, horizontal rotation, pitch, roll, etc., can provide an important basis for determining whether the driver is focused.

Therefore, establishing a set of monitoring methods capable of identifying the attention state of the driver more accurately is one of the important research points in the current intelligent driving field.

Disclosure of Invention

In view of the above, the present invention provides an artificial intelligence-based driver attention monitoring method and system, aiming to more accurately recognize distraction by analyzing continuous changes in driver head pose.

In order to achieve the above object, the invention discloses a driver attention monitoring method based on artificial intelligence, which comprises the following steps:

s1: extracting a direction histogram characteristic of the collected front image of the driver and classifying the collected front image of the driver by a support vector machine to obtain circumscribed rectangular coordinates of a face area of the driver, and obtaining an image of the face area of the driver according to the circumscribed rectangular coordinates;

s2: extracting a face outline in a face area image of a driver through edge detection, positioning and detecting an eye area and a mouth area in the face area image of the driver by using a template matching algorithm, and outputting central position coordinates of a left eye, a right eye and a mouth;

S3: calculating the horizontal rotation angle, the pitching angle and the side tilting angle of the head of the driver by utilizing the geometric relationship according to the central position coordinates of the left eye, the right eye and the mouth;

s4: constructing an attention area classification model based on the horizontal rotation angle, the pitching angle and the rolling angle of the head of the driver to obtain an attention area of the driver; comprising the following steps:

constructing an attention area classification model, and obtaining the attention area of the driver, wherein the attention area classification model maps the horizontal rotation angle, the pitching angle and the rolling angle of the head of the driver to different attention areas respectively The mapping formula is as follows:；

in the formula, AndFront area thresholds for the horizontal rotation angle and the pitch angle, respectively; Is a lateral gaze threshold; is the lower gaze threshold; Is the upper gaze threshold; the attention areas are respectively represented as front, left, right, lower, upper and other areas;

s5: the horizontal rotation angle, the pitching angle and the rolling angle of the head of the driver and the attention area of the driver are processed by using the artificial intelligent driven long-short-time memory network, the distraction is monitored, and the monitoring result is output.

Preferably, the step S1 includes the steps of:

s11: extracting the direction histogram characteristics of the collected front image of the driver:

for the acquired front image of the driver Calculating the horizontal gradient of each pixelAnd a vertical gradient；

Based on horizontal gradientsAnd a vertical gradientCalculating gradient magnitudeAnd gradient angleThe calculation formula is as follows:

；

in the formula, Is the pixel location; And Respectively, front images of driversAt the pixel positionA gradient value of a gradient in the horizontal direction and a gradient value of a gradient in the vertical direction; is the amplitude of the gradient At the pixel positionGradient magnitude at; is the gradient angle At the pixel positionAngle value at the gradient angle represents the angle of the gradient direction relative to the horizontal direction, ranging from；

Image the front of driverDividing into a plurality of non-overlapping image blocks, the size of the image blocks beingWithin each image block, the gradient angle of each pixel is calculatedIs assigned to a corresponding direction interval in which,For pixel locations in an image block, the direction interval will beThe angular range is divided intoA plurality of equally spaced directional intervals; when (when)After the distribution is completedCorresponding toThe direction histogram of each image block is obtained by summing, and the calculation formula is as follows:

；

in the formula, Is thatDimension vector, representing the firstDirection histogram of individual image blockValues of dimensions;， Is the total number of image blocks; ； Is the first A set of pixel locations contained in the image blocks; Is the first A plurality of direction intervals; is the gradient angle At the pixel positionAn angle value at; Judging Whether or not the value of (2) is at the firstIn the direction intervals, if yes, the direction interval is 1, otherwise, the direction interval is 0; is the amplitude of the gradient At the pixel positionGradient magnitude at;

S12: facial region classification using support vectors:

Inputting the direction histogram feature of each image block into a pre-trained support vector machine, wherein the support vector machine is used for classifying the face area of the corresponding image block, and if the image block is classified as the face area, recording the pixel position of the upper left corner and the pixel position of the lower right corner of the image block;

After finishing the face region classification of all the image blocks, selecting a pixel position with the minimum horizontal direction coordinate and the minimum vertical direction coordinate as the upper left corner coordinate of the circumscribed rectangle of the face region of the driver, and selecting a pixel position with the maximum horizontal direction coordinate and the maximum vertical direction coordinate as the lower right corner coordinate of the circumscribed rectangle of the face region of the driver in all the image blocks classified as the face region;

Clipping the driver front image based on the upper left and lower right coordinates of the driver's face region An image of the driver's face area is obtained.

Preferably, the step S2 includes the steps of:

S21: and (3) carrying out Canny edge detection:

Extracting an edge image of the driver face region image obtained in the step S1 by applying a Canny edge detection algorithm Namely, the facial outline;

S22: detecting an eye area by using an eye shape template, and outputting center position coordinates of a left eye and a right eye in an edge image;

s23: and detecting a mouth region by using a mouth shape template, and outputting the central position coordinate of the mouth in the edge image.

Further preferably, the step S22 includes the steps of:

S221: eye shape template matching:

Matching by using an eye shape template, wherein the eye shape template is an image obtained by averaging a pre-collected image of the left eye and the right eye of a driver after being processed by a Canny edge detection algorithm, and the edge image is calculated by a sliding window method Each pixel position of (a)Eye match score of (2)The calculation formula is as follows:

；

in the formula, Is thatThe elements of the set are selected to be,Form for eyesA set of pixel locations contained in the image; Is an edge image At the pixel positionPixel values at; form for eyes At the pixel positionPixel values at;

s222: determining an eye area:

by maximizing edge images Eye matching score in top left quarter image block range, edge image is determinedCenter position coordinates of middle left eye; By maximizing edge imagesEye matching score in upper right quarter image block range, and edge image is determinedCenter position coordinates of middle right eye。

Further preferably, the step S23 includes the steps of:

S231: mouth shape template matching:

matching is carried out by using a mouth shape template, wherein the mouth shape template is an image obtained by processing a pre-collected driver mouth image through a Canny edge detection algorithm, and an edge image is calculated by a sliding window method Each pixel position of (a)Mouth match score of (a)The calculation formula is as follows:

；

in the formula, Is thatThe elements of the set are selected to be,Template with mouth shapeA set of pixel locations contained in the image; Is an edge image At the pixel positionPixel values at; Template with mouth shape At the pixel positionPixel values at;

S232: determining a mouth area:

by maximizing edge images Mouth matching score in the range of one-half image block below, and determining edge imageCenter position coordinates of middle mouth。

Preferably, the step S3 includes the steps of:

s31: calculating a horizontal rotation angle:

Calculating the horizontal rotation angle of the head of the driver according to the positions of the left eye and the right eye in the horizontal direction The calculation formula is as follows:

；

in the formula, A focal length of a camera for taking a front image of the driver;

s32: calculating a pitching angle:

calculating the pitching angle of the head of the driver according to the positions of the eyes and the mouth in the vertical direction The calculation formula is as follows:

；

s33: calculating the roll angle:

calculating the roll angle of the head of the driver based on the vertical and horizontal positions of the eyes The calculation formula is as follows:

。

preferably, the step S5 includes the steps of:

s51: constructing an input sequence of a long-short-time memory network:

Using the horizontal rotation angle, pitch angle and roll angle of the head of the driver obtained in step S3 and the attention area obtained in step S4 as input data of a long-short time memory network, the input feature vector of each time step of the long-short time memory network is The eigenvector formula is as follows:

；

in the formula, For the time steps, each time step will take a front image of the driver,AndRespectively the firstThe horizontal rotation angle, the pitching angle, the rolling angle and the attention area corresponding to the front images of the driver shot in each time step;

Length-based memory network The feature vector sequence contained in the time window of (2) is taken as input, the feature vector sequence formula is as follows:

；

in the formula, Is the firstA sequence of input feature vectors for each time step;

S52: building a long-time and short-time memory network:

Will be After being input into a long-short-time memory network, the obtained output is in a hidden stateAnd conceal the state through a full connection layer and activation functionMapping into two classification outputs;

S53: training long and short term memory network:

training a long-time memory network by using a two-class cross entropy loss function, wherein a training formula is as follows:

；

in the formula, The number of training samples;； Is the first Training sample at the firstThe real labels of the time steps are 1 or 0, wherein 1 represents distraction, 0 represents concentration and is obtained by manual labeling; output of the network for long-short time memory Training sample at the firstA driver distraction probability for each time step;

Training parameters of a long-short-time memory network by using a random gradient descent method to minimize a loss function, so as to obtain a trained long-short-time memory network;

S54: long and short time memory network with training completion:

Judging real-time data by using a trained long and short time memory network, and outputting each time step If it is determined that the attention is distracted over a plurality of consecutive time steps, it is determined that the driver is currently in a distracted state.

The invention also discloses a driver attention monitoring system based on artificial intelligence, which comprises:

face region extraction module: extracting a direction histogram characteristic of the collected front image of the driver and classifying the collected front image of the driver by a support vector machine to obtain circumscribed rectangular coordinates of a face area of the driver, and obtaining an image of the face area of the driver according to the circumscribed rectangular coordinates;

The eye and mouth positioning and monitoring module: extracting a face outline in a face area image of a driver through edge detection, positioning and detecting an eye area and a mouth area in the face area image of the driver by using a template matching algorithm, and outputting central position coordinates of a left eye, a right eye and a mouth;

A head pose estimation module: calculating the horizontal rotation angle, the pitching angle and the side tilting angle of the head of the driver by utilizing the geometric relationship according to the central position coordinates of the left eye, the right eye and the mouth;

Attention area extraction module: constructing an attention area classification model based on the horizontal rotation angle, the pitching angle and the rolling angle of the head of the driver to obtain an attention area of the driver;

And a monitoring and judging module: the method comprises the steps of processing a horizontal rotation angle, a pitching angle and a rolling angle of a head of a driver and an attention area of the driver by using an artificial intelligent driven long-short-time memory network, monitoring distraction, and outputting a monitoring result;

The driver attention monitoring method based on artificial intelligence is realized through the system.

The beneficial effects are that:

According to the invention, by combining with a plurality of advanced image processing technologies such as direction histogram feature extraction, support vector machine classification, template matching and the like, the facial features of the driver can be accurately extracted, the eyes and mouth areas can be accurately identified, and on the basis, the head posture change condition of the driver can be further obtained by calculating the geometric relationship, so that the system can not only identify whether the driver looks ahead, but also judge the distraction state of the driver by analyzing the rotation angle, the pitching angle and the side tilting angle of the head, and the multidimensional analysis greatly improves the attention monitoring precision of the driver and reduces misjudgment.

According to the invention, the time sequence information-based deep analysis is performed by utilizing the artificial intelligent driven long-short-time memory network, so that not only is the state of the driver at a single moment judged, but also the change of the head gesture and the attention area of the driver in a period of time is continuously analyzed, the change of the attention of the driver in a time sequence is captured, and the continuous distraction behavior is accurately identified, thereby effectively reducing false alarm caused by single-frame image analysis and improving the monitoring effect of the system in long-time driving.

According to the invention, through multidimensional analysis of the head gesture of the driver, false alarm caused by natural head movement or simple gesture adjustment of the driver is effectively avoided, and in addition, short non-attention-dispersing behaviors such as rearview mirrors or side-looking traffic conditions can be identified by using time series analysis based on long and short time memories, so that the system is prevented from being excessively sensitive; the method can still keep a stable monitoring effect under the complex illumination condition and under the condition of certain shielding, so that the driver monitoring system is more practical and reliable in practical application and is suitable for complex driving environments.

Drawings

Fig. 1 is a flow chart of a driver attention monitoring method based on artificial intelligence in embodiment 1 of the invention.

Detailed Description

The invention is further described below with reference to the accompanying drawings, without limiting the invention in any way, and any alterations or substitutions based on the teachings of the invention are intended to fall within the scope of the invention.

Example 1

As shown in fig. 1, a driver attention monitoring method based on artificial intelligence includes the following steps:

S1: extracting direction histogram features of the collected front face image of the driver and classifying the collected front face image of the driver by a support vector machine to obtain circumscribed rectangular coordinates of a face area of the driver, and obtaining an image of the face area of the driver according to the circumscribed rectangular coordinates:

for the acquired front image of the driver Calculating the horizontal gradient of each pixelAnd a vertical gradientThe calculation formula is as follows:

；

in the formula, For the pixel location(s),AndRespectively a horizontal direction coordinate and a vertical direction coordinate of the pixel position; And Respectively, front images of driversAt the pixel positionA gradient value of a gradient in the horizontal direction and a gradient value of a gradient in the vertical direction;、、 And Respectively, front images of driversAt the pixel position、、AndPixel values at;

based on horizontal gradients And a vertical gradientCalculating gradient magnitudeAnd gradient angleThe calculation formula is as follows:

；

in the formula, Is the amplitude of the gradientAt the pixel positionGradient magnitude at; is the gradient angle At the pixel positionAngle value at the gradient angle represents the angle of the gradient direction relative to the horizontal direction, ranging fromIgnoring the direction;

image the front of driver Dividing into a plurality of non-overlapping image blocks, the size of the image blocks beingWithin each image block, the gradient angle of each pixel is calculatedIs assigned to a corresponding direction interval in which,For pixel locations in an image block, the direction interval will beThe angular range is divided intoA plurality of equally spaced directional intervals; when (when)After the distribution is completedCorresponding toThe direction histogram of each image block is obtained by summing, and the calculation formula is as follows:

；

S12: facial region classification using support vectors:

The direction histogram features can capture local gradient directions and edge information in the image, are particularly suitable for target detection of a face and the like with obvious edge structures, and can further improve the accuracy and the robustness of face region detection by combining the classification capability of a support vector machine; therefore, in step S1, the face region can be effectively and accurately identified from the front image of the driver by combining the direction histogram feature extraction with the support vector machine classification.

S2: extracting a face outline in a face area image of a driver through edge detection, positioning and detecting an eye area and a mouth area in the face area image of the driver by using a template matching algorithm, and outputting central position coordinates of a left eye, a right eye and the mouth:

S21: and (3) carrying out Canny edge detection:

S221: eye shape template matching:

；

in the formula, Is thatThe elements of the set are selected to be,Form for eyesA set of pixel locations contained in the image; And Eye shape templates respectivelyA horizontal direction coordinate and a vertical direction coordinate of the middle pixel position; Is an edge image At the pixel positionPixel values at; form for eyes At the pixel positionPixel values at;

s222: determining an eye area:

S23: detecting a mouth region by using a mouth shape template, and outputting center position coordinates of a mouth in an edge image:

S231: mouth shape template matching:

；

in the formula, Is thatThe elements of the set are selected to be,Template with mouth shapeA set of pixel locations contained in the image,AndRespectively mouth-shaped templatesA horizontal direction coordinate and a vertical direction coordinate of the middle pixel position; Is an edge image At the pixel positionPixel values at; Template with mouth shape At the pixel positionPixel values at;

S232: determining a mouth area:

The Canny edge detection is very sensitive to edge changes in the image, edge contour information of the face of the driver can be effectively extracted, the central positions of eyes and the mouth can be accurately positioned by means of prior information through template matching, the accurate positioning of key features of the face is ensured, and the possibility of false detection is reduced; therefore, step S2, by combining Canny edge detection with template matching, can accurately identify the eyes and mouth, which are key areas in the driver' S face image.

S3: calculating the horizontal rotation angle, the pitching angle and the side tilting angle of the head of the driver by using the geometrical relationship according to the central position coordinates of the left eye, the right eye and the mouth:

s31: calculating a horizontal rotation angle:

；

s32: calculating a pitching angle:

；

s33: calculating the roll angle:

。

And step S3, the horizontal rotation angle, the pitching angle and the rolling angle of the head are calculated based on the geometric relation between the left eye, the right eye and the mouth, so that a complex three-dimensional reconstruction or deep learning model is avoided, the head posture of the driver can be judged in real time with lower calculation complexity, and the method is suitable for a driver monitoring system with high real-time requirements.

S4: constructing an attention area classification model based on the horizontal rotation angle, the pitching angle and the rolling angle of the head of the driver to obtain the attention area of the driver:

in the formula, AndThe front area threshold values of the horizontal rotation angle and the pitch angle are respectively, in this embodiment；The lateral gaze threshold, in this embodiment；For the lower gaze threshold, in this embodiment；For the upper gaze threshold, in this embodiment；The attention areas are respectively represented as front, left, right, lower, upper and other areas.

And S4, deducing the attention area by combining the horizontal rotation, pitching and rolling gesture changes of the head of the driver, wherein compared with the analysis of a single angle, the method can effectively capture the complex head gesture changes based on multiple dimensions, avoid missing or misjudging the attention state of the driver, and can quickly calculate and output the attention area of the driver due to the fact that the method mainly depends on the geometric calculation of the head angle, thereby being very suitable for being applied to a real-time driving monitoring system.

S5: the horizontal rotation angle, the pitching angle and the rolling angle of the head of the driver and the attention area of the driver are processed by using the artificial intelligent driven long-short-time memory network, the distraction is monitored, and the monitoring result is output:

s51: constructing an input sequence of a long-short-time memory network:

；

S52: building a long-time and short-time memory network:

Will be After being input into a long-short-time memory network, the obtained output is in a hidden stateAnd conceal the state through a full connection layer and activation functionMapping into two kinds of output, the mapping formula is as follows:

；

in the formula, In this embodiment, a long-short-time memory network with a layer number of 3 is used, and the dimension of the hidden state is 128; And The weight and the bias value of the full connection layer are respectively; to activate the function, a sigmoid function is used in this embodiment; Is the first A driver distraction probability for each time step;

S53: training long and short term memory network:

；

S54: long and short time memory network with training completion:

Judging real-time data by using a trained long and short time memory network, and outputting each time step If it is determined that the attention is distracted in a plurality of consecutive time steps, it is determined that the driver is currently in a distraction state; in this embodiment, the determination of the attention situation over 5 consecutive time steps is used.

The long-short-time memory network can process the data of the head gesture and the attention area of the driver in real time and generate a real-time judgment result of the attention state of the driver; the long-short-term memory network has the advantage that the long-short-term memory network can memorize past input data and make current judgment by utilizing the information, and is particularly suitable for processing time series data, so that the system can not only identify the attention state under a single frame image, but also identify the attention change in a continuous time period, and particularly identify more accurately aiming at short distraction behaviors and continuous distraction behaviors. Therefore, in the step S5, the time sequence information of the head gesture and the attention area of the driver is processed by using a long-short-time memory network, and the state change of the driver in the driving process is effectively captured by continuous time step monitoring, so that the dynamic attention state monitoring of the driver is realized.

Example 2

An artificial intelligence based driver attention monitoring system comprising the following five modules:

The driver's attention monitoring method based on artificial intelligence in embodiment 1 is implemented by this system.

It should be noted that, the foregoing reference numerals of the embodiments of the present invention are merely for describing the embodiments, and do not represent the advantages and disadvantages of the embodiments. And the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that comprises the element.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.

The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.

Claims

1. A driver attention monitoring method based on artificial intelligence, characterized in that it includes the following steps:

S1: extracting directional histogram features and performing support vector machine classification on the collected front image of the driver to obtain the coordinates of the circumscribed rectangle of the driver's facial area, and obtaining the driver's facial area image according to the circumscribed rectangle coordinates;

S2: extract the facial contour in the driver's facial area image by edge detection, locate and detect the eye area and mouth area in the driver's facial area image by using the template matching algorithm, and output the center position coordinates of the left eye, right eye and mouth;

S3: Calculate the horizontal rotation angle, pitch angle and roll angle of the driver's head using geometric relationships according to the center position coordinates of the left eye, right eye and mouth;

S4: construct an attention area classification model based on the horizontal rotation angle, pitch angle and roll angle of the driver's head to obtain the driver's attention area; including:

Construct an attention area classification model to obtain the driver's attention area. The attention area classification model maps the horizontal rotation angle, pitch angle and roll angle of the driver's head to different attention areas respectively. , the mapping formula is as follows: ;

In the formula, and are the front area thresholds for horizontal rotation angle and pitch angle respectively; is the lateral gaze threshold; is the lower fixation threshold; is the upper fixation threshold; They represent the attention areas as front, left, right, bottom, top and other areas respectively;

S5: Uses an AI-driven long short-term memory network to process the horizontal rotation angle, pitch angle, and roll angle of the driver's head and the driver's attention area, monitors distraction, and outputs monitoring results.

2. The driver attention monitoring method based on artificial intelligence according to claim 1, characterized in that the step S1 comprises the following steps:

S11: Extract the direction histogram features of the collected driver's front image:

The collected frontal image of the driver Calculate the horizontal gradient of each pixel and vertical gradient ;

Based on the horizontal gradient and vertical gradient Calculate the gradient magnitude and gradient angle , the calculation formula is as follows:

;

In the formula, is the pixel position; and The front image of the driver At pixel position The gradient value of the horizontal gradient and the gradient value of the vertical gradient; is the gradient amplitude At pixel position The gradient amplitude at ; is the gradient angle At pixel position The angle value at the gradient angle represents the angle of the gradient direction relative to the horizontal direction, and the range is ;

Driver front view Divide into multiple non-overlapping image blocks, the size of the image block is , in each image block, the gradient angle of each pixel is Assigned to the corresponding direction interval, is the pixel position in the image block, and the direction interval is Angle range is divided into equally spaced direction intervals; when After the allocation is completed, Corresponding The sum is performed to obtain the direction histogram of each image block. The calculation formula is as follows:

;

In the formula, for dimensional vector, indicating the The orientation histogram of the image block The value of the dimension; , is the total number of image blocks; ; For the The set of pixel positions contained in an image block; For the Direction intervals; is the gradient angle At pixel position The angle value at judge Is the value of If it is in the direction interval, it is 1, otherwise it is 0; is the gradient amplitude At pixel position The gradient amplitude at ;

S12: Facial region classification using support vectors:

Input the directional histogram features of each image block to a pre-trained support vector machine to classify the corresponding image block into a facial region. If the image block is classified as having a facial region, the upper left corner pixel position and the lower right corner pixel position of the image block are recorded.

After completing the facial region classification for all image blocks, among all image blocks classified as having facial regions, the pixel position where the upper left corner horizontal coordinate and the vertical coordinate simultaneously reach the minimum is selected as the upper left corner coordinate of the circumscribed rectangle of the driver's facial region, and the pixel position where the lower right corner horizontal coordinate and the vertical coordinate simultaneously reach the maximum is selected as the lower right corner coordinate of the circumscribed rectangle of the driver's facial region;

Crop the driver's front image based on the upper left corner coordinates and lower right corner coordinates of the driver's facial area , get the driver's facial area image.

3. The driver attention monitoring method based on artificial intelligence according to claim 1, characterized in that the step S2 comprises the following steps:

S21: Perform Canny edge detection:

Apply the Canny edge detection algorithm to extract the edge image of the driver's facial area image obtained in step S1 , which is the facial contour;

S22: Use the eye shape template to perform eye region detection, and output the center position coordinates of the left eye and the right eye in the edge image;

S23: Use the mouth shape template to detect the mouth area and output the center position coordinates of the mouth in the edge image.

4. The driver attention monitoring method based on artificial intelligence according to claim 3, characterized in that the step S22 comprises the following steps:

S221: Eye shape template matching:

The eye shape template is used for matching. The eye shape template is an image obtained by averaging the pre-collected left and right eye images of the driver after being processed by the Canny edge detection algorithm. The edge image is calculated by the sliding window method. Each pixel position Eye Matching Score , the calculation formula is as follows:

;

In the formula, for The elements in the collection, Template for eye shape The set of pixel positions contained in ; For edge images At pixel position The pixel value at ; Template for eye shape At pixel position The pixel value at ;

S222: Determine the eye area:

By maximizing the edge image The eye matching score within the upper left quarter of the image block determines the edge image The center position coordinates of the left eye ; By maximizing the edge image The eye matching score within the upper right quarter of the image block determines the edge image The center position coordinates of the right eye .

5. The driver attention monitoring method based on artificial intelligence according to claim 4, characterized in that the step S23 comprises the following steps:

S231: Mouth shape template matching:

The mouth shape template is used for matching. The mouth shape template is an image of the driver's mouth collected in advance and processed by the Canny edge detection algorithm. The edge image is calculated by the sliding window method. Each pixel position Mouth matching score , the calculation formula is as follows:

;

In the formula, for The elements in the collection, Mouth shape template The set of pixel positions contained in ; For edge images At pixel position The pixel value at ; Mouth shape template At pixel position The pixel value at ;

S232: Determine the mouth area:

By maximizing the edge image The mouth matching score within the lower half of the image block determines the edge image The center coordinates of the mouth .

6. The driver attention monitoring method based on artificial intelligence according to claim 5, characterized in that the step S3 comprises the following steps:

S31: Calculate the horizontal rotation angle:

Calculate the horizontal rotation angle of the driver's head based on the horizontal positions of the left and right eyes , the calculation formula is as follows:

;

In the formula, is the focal length of the camera that captures the frontal image of the driver;

S32: Calculate the pitch angle:

Calculate the pitch angle of the driver's head based on the vertical positions of the eyes and mouth , the calculation formula is as follows:

;

S33: Calculate the roll angle:

Calculate the tilt angle of the driver's head based on the vertical and horizontal positions of the eyes , the calculation formula is as follows:

.

7. The driver attention monitoring method based on artificial intelligence according to claim 6, characterized in that the step S5 comprises the following steps:

S51: Constructing the input sequence of the long short-term memory network:

The horizontal rotation angle, pitch angle and roll angle of the driver's head obtained in step S3 and the attention area obtained in step S4 are used as the input data of the long short-term memory network. The input feature vector of each time step of the long short-term memory network is , the eigenvector formula is as follows:

;

In the formula, is the time step, and the driver’s front image is captured once in each time step. and Respectively The horizontal rotation angle, pitch angle, roll angle and attention area corresponding to the front image of the driver taken at each time step;

The long short-term memory network is based on a The feature vector sequence contained in the time window is taken as input, and the feature vector sequence formula is as follows:

;

In the formula, For the The input feature vector sequence of time steps;

S52: Building a long short-term memory network:

Will After inputting into the long short-term memory network, the output is the hidden state , and transform the hidden state through a fully connected layer and activation function Mapped to binary classification output;

S53: Training Long Short-Term Memory Networks:

The binary cross entropy loss function is used to train the long short-term memory network. The training formula is as follows:

;

In the formula, is the number of training samples; ; For the The training samples are The true label of each time step is 1 or 0, where 1 represents distraction and 0 represents concentration, obtained by manual annotation; is the output of the long short-term memory network The training samples are The probability of driver distraction in each time step;

Use the stochastic gradient descent method to train the parameters of the long short-term memory network to minimize the loss function and obtain a trained long short-term memory network;

S54: Apply the trained LSTM network:

Use the trained long short-term memory network to judge the real-time data and output each time step If the driver is judged as distracted at multiple consecutive time steps, it is determined that the driver is currently in a distracted state.

8. A driver attention monitoring system based on artificial intelligence, characterized by comprising:

Facial region extraction module: extracts the direction histogram features and classifies the driver's frontal image using a support vector machine to obtain the coordinates of the circumscribed rectangle of the driver's facial region, and obtains the driver's facial region image based on the circumscribed rectangle coordinates;

Eye and mouth positioning monitoring module: extracts the facial contour in the driver's facial area image through edge detection, uses the template matching algorithm to locate and detect the eye area and mouth area in the driver's facial area image, and outputs the center position coordinates of the left eye, right eye and mouth;

Head posture estimation module: Based on the center position coordinates of the left eye, right eye and mouth, the horizontal rotation angle, pitch angle and roll angle of the driver's head are calculated using geometric relationships;

Attention area extraction module: constructs an attention area classification model based on the horizontal rotation angle, pitch angle and roll angle of the driver's head to obtain the driver's attention area;

Monitoring and judgment module: Uses an AI-driven long short-term memory network to process the horizontal rotation angle, pitch angle, and roll angle of the driver's head and the driver's attention area, monitors distraction, and outputs monitoring results;

To implement the artificial intelligence-based driver attention monitoring method as described in any one of claims 1-7.