JP5109750B2 - Driver state detection device, consciousness state detection method - Google Patents

Description

  The present invention relates to a driver state detection device and a consciousness state detection method for detecting a driver's consciousness state, and in particular, driving for detecting a driver's consciousness state based on a driver's eye state following a change in display information. The present invention relates to a person state detection device and a consciousness state detection method.

Sleepiness is cited as one of the factors that distracts the driver's attention while driving the vehicle, and a technique for detecting that the driver is sleepy has been proposed (see, for example, Patent Document 1). Patent Document 1 describes a dozing alarm device that adjusts photographing means from a face position and a focal length in a dozing alarm device that captures a driver's face image and estimates the driver's arousal level from the eye state. Yes. In addition, a technique for detecting a driver's tension state from changes in the pupil has been proposed (see, for example, Patent Document 2). Patent Document 2 describes a driver state detection device that determines a tension state when a change in pupil diameter is small when a preceding vehicle approaches, and accelerates a warning or braking timing.
Japanese Patent Laid-Open No. 11-488820 JP 2002-367100 A

  However, in the driver state detection device described in Patent Document 2, since the pupil diameter does not change, the driver's tension state is detected, but does the pupil diameter not change because the driver is gazing at something? Or, it is impossible to detect whether the pupil diameter does not change due to distraction. For this reason, there is a problem in that an unnecessary warning may be generated for a driver who is gazing forward or the like simply by detecting a change in pupil diameter and giving a warning.

  In view of the above problems, an object of the present invention is to provide a driver state detection device and a consciousness state detection method capable of accurately determining a driver's consciousness state.

In view of the above problems, the present invention provides a driver state detection device that detects a driver's consciousness state, a display unit that displays display information whose contents are variable, a photographing unit that captures a driver's face image, a face A face direction / gaze direction detection means for detecting at least one of a face direction or a gaze direction from an image, and a part or all of the screen of the display means arranged in the gaze direction detected by the face direction / gaze direction detection means Based on whether or not the state of the driver's eyes has changed following the change of part or all of the screen by the screen changing means (for example, the blur generation unit 19 and the visual stimulation part 32) to be changed and the screen changing means. Driver consciousness determination means for detecting the driver's consciousness state (for example, pupil tracking determination section 18 and stimulus tracking determination section 33), and pupil detection means for detecting the size of the driver's pupil from the face image. The screen The means blurs part or all of the outer edge of the display information indefinitely, and the driver consciousness determination means follows whether or not the size of the driver's pupil has changed following the blur of the display information. Based on the above, the driver's consciousness state is detected .

  According to the present invention, the screen of the driver's line-of-sight direction is changed and it is determined whether or not the user is gazing. It is possible to detect the state separately from the state in which the direction is actually gazing.

  It is possible to provide a driver state detection device and a consciousness state detection method that can accurately determine the driver's consciousness state.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the accompanying drawings.

  FIG. 1 is an example of a diagram for explaining an outline of determination as to whether or not the user is gazing. The driver state detection apparatus 100 according to the present embodiment intentionally blurs display information displayed on a navigation screen 12, a head-up display (hereinafter referred to as HUD), a meter display unit, and the like. In FIG. 1, the outer edge of a star-shaped figure is blurred. By the way, it is known that the size of the pupil of the human eye changes due to the adjustment of the brightness and darkness of the ambient light and the perspective. For this reason, when the display information that has been watched until then is blurred, the size of the pupil changes so as to unconsciously focus. Therefore, as shown in FIG. 1, the size of the pupil of the driver who is gazing at the blurred display information changes (is larger in the figure). The driver state detection device 100 detects the change in the size of the pupil from the face image, and pays attention to the display medium in the sight line direction at that time, for example, whether the front of the vehicle is being watched or not. It is determined whether or not.

  Conventionally, it has been possible to detect whether or not the line of sight is directed in the direction from the face direction, the line-of-sight direction, and their stopping time, but it is difficult to detect the state of consciousness such as arousal level and attention. In this embodiment, when the line of sight is stopped in the same direction, it is possible to distinguish the state where the attention is distracted from the state where the attention is actually gazing, and to appropriately determine the driver's consciousness state.

  FIG. 2 shows an example of a functional block diagram of the driver state detection device 100 of the present embodiment. The functional block diagram of FIG. 2 shows a driver monitor sensor 11 corresponding to a means for detecting pupils, a face direction / line-of-sight direction detection unit 16, a pupil detection unit 17, and a navigation screen 12 corresponding to blurring means, a meter The display unit 13, the HUD 14, the side mirror 15 (hereinafter sometimes collectively referred to as a display medium 20), and the blur generation unit 19. Further, the pupil tracking determination unit 18 determines whether or not the size of the pupil changes following the blur. Of these, the face direction / gaze direction detection unit 16, the pupil detection unit 17, the pupil tracking determination unit 18, and the blur generation unit 19 are executed by a CPU of a predetermined electronic control unit (ECU) or an ASIC (Application Realized by hardware such as Specific Integrated Circuit). Each function need not be realized by only one ECU, and may be realized by being distributed to a plurality of ECUs. For example, the blur generation unit 19 is provided in each ECU that controls the navigation screen 12 and the like.

(Pupil detection)
The driver monitor sensor 11 is a photographing unit that photographs a driver's face image disposed on, for example, a steering column. FIG. 3 shows an example of a front view of the driver monitor sensor 11. The driver monitor sensor 11 includes a camera 21 and an infrared projector 22, and the camera 21 is disposed on a vertical axis passing through the center of the steering wheel and photographs the driver's face from the front. The same number (three in the figure) of infrared projectors 22 are arranged on the left and right sides of the camera 21. The infrared projector 22 is an LED lamp that projects near-infrared light toward the driver's face, and facilitates shooting of the driver's face and detection of the pupil at night.

  The camera 21 is composed of a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like, and has sensitivity to infrared light irradiated by the infrared projector 22, and after the incident light is converted into an electrical signal, a predetermined value is obtained. Tone digital data (face image) is generated. Further, the light emitted from the infrared projector 22 enters the driver's pupil, and the luminance of the pupil portion becomes different from the luminance of the surrounding eyes, so that the position and size of the pupil are detected.

  Next, detection of the driver's face direction and line-of-sight direction will be described. Face images taken by the camera 21 are sequentially input to the face direction / gaze direction detection unit 16 and stored in the memory.

  First, the face direction / gaze direction detection unit 16 detects the approximate position of the face (face outline) from the face image. For example, pixel values (luminance) at the same pixel position are compared between sequentially input face images, and the difference between pixel values of pixels having pixel values different from a predetermined value or more is stored together with the pixel position. Then, if the difference in pixel values at the same pixel position is counted for a predetermined number of face images, the number of pixel positions counted in the contour portion increases. Although the face image shows the background, etc., the background is stationary, so the left and right edges of the face have the largest number of counts. It becomes the contour position of the face in the horizontal direction. In this way, the left and right ends of the face position are determined.

  Next, the face direction / gaze direction detection unit 16 detects a feature point of the face from the edge information of the face image. Based on the edge information, facial feature points (such as eyebrows, upper and lower eyelids, nostrils, mouth corners, and upper and lower lip borders) are detected as pixels having a large luminance change compared to skin. For example, if a known edge detection algorithm such as Sobel is used, the contour of each part of the face surrounded by two types of edge information of low brightness to high brightness and high brightness to low brightness can be obtained.

  The face direction / line-of-sight direction detection unit 16 detects the upper and lower contour positions and the center line of the face from the edge information. Since feature points such as the eyes and nose of a human face are arranged symmetrically, a vertical line in which the number of pieces of left and right edge information is almost equal is the center line of the face. For example, a virtual vertical line that divides the face image into left and right is set, and the number of left and right edge information is counted based on the vertical line. The difference in the number of left and right edge information is compared with a threshold, and the vertical line that is equal to or smaller than the threshold or the vertical line that minimizes the difference in the number of edge information is determined as the center line of the face.

  The face direction / line-of-sight direction detection unit 16 detects the driver's face direction based on the obtained center line. That is, based on the distance from the center line to the left or right contour position, the degree of face orientation in the left-right direction, which is zero degrees with respect to the front direction, is detected, for example, in increments of about 5 to 10 degrees.

  The detection of the gaze direction will be described. The line-of-sight direction is detected by, for example, a corneal reflection method. The corneal reflection method detects the line-of-sight direction by utilizing the fact that the virtual image of the light of the infrared projector 22 on the cornea moves in parallel with the eyeball movement due to the difference between the rotation centers of the cornea and the eyeball. Therefore, the line-of-sight direction is obtained by detecting the corneal reflection point 23 and the pupil center 24 from the face image.

  FIG. 4 is an example of a diagram schematically showing detection of the line-of-sight direction. The pupil tends to appear darker than the portion of the eye other than the pupil. For this reason, for example, when the blinking of the infrared projector 22 and the photographing are synchronized and the difference in pixel value between the face image when turned on and the face image when turned off is calculated, only the pupil reacts to the light and the pixel value changes. The pixel values other than the pupil become zero or a negligible small value due to the difference, and only the pupil increases the difference in the pixel value, so that the pupil can be easily detected. The size of the pupil varies in the range of about 2 to 6 mm in adults, but on average is about 4 mm. In the case of humans, since the pupil is circular, the center thereof is determined as the position of the pupil center 24.

  Similarly, the corneal reflection point 23 is a point with high brightness that is detected only when the infrared projector 22 is turned on, and thus is clearly detected from the difference in pixel values between the face image when turned on and the face image when turned off. .

  The position of the pupil center 24 and the position of the corneal reflection point 23 are positions in three-dimensional coordinates with the origin as a predetermined portion of the camera 21, for example. Since there is an individual difference in the actual human gaze direction with respect to the positions of the corneal reflection point 23 and the pupil center 24, coordinates in this coordinate system are used to calibrate the relationship between the position of the corneal reflection point 23 and the pupil center 24 and the gaze direction. The known point Q is taken to the driver and the positions of the corneal reflection point 23 and the pupil center 24 at that time are obtained. Since this position is a three-dimensional position, it is preferably detected including the distance in the front-rear direction of the vehicle from the seat position and the reclining angle, or more preferably detected directly as a three-dimensional position by a stereo camera.

  Therefore, the vector EO from the camera 21 to the pupil center 24 becomes clear, and the known coordinate Q and the vector EQ to the pupil center 24 become clear. Further, an angle α formed by the vector EO and the vector EQ is obtained.

  Next, a distance r between the corneal reflection point 23 and the pupil center 24 is obtained from the positions of the corneal reflection point 23 and the pupil center 24. This distance r is a distance on three-dimensional coordinates. Further, the angle α formed with the distance r has a linear relationship, and a relationship r = f (α) is obtained.

  When this relational expression is obtained, the line-of-sight vector when gazing at an unknown point can be calculated from α ′ by substituting r ′ obtained from the pupil center 24 and the corneal reflection point 23 at that time into the relational expression. . Α or α ′ is represented by two components in the horizontal direction and the elevation angle direction, respectively. If necessary, the final gaze direction is obtained by correcting the gaze direction based on the already detected face orientation.

  The pupil detection unit 17 detects the size of the pupil. As described above, the pupil is detected from the difference in pixel values between the face image when the infrared projector 22 is turned on and the face image when the infrared projector 22 is turned off. In this case, the pupil is circular, but the vertical direction of the pupil can vary depending on the blink of the driver and the opening of the eye, so the pupil detection unit 17 detects the horizontal length of the pupil as the size of the pupil. . The number of pixels in the pupil part may be converted to mm or the like, or the number of pixels may be handled as it is.

[Occurrence of blur]
Blurring can be generated in all display media 20 whose display information content is variable. For example, the navigation screen 12 displays icons representing the vehicle position, parking lot, etc., roads, operation menus, guide lines for parking assistance, messages, and the like. The meter display unit 13 or the HUD 14 displays a vehicle speed, an engine speed, a shift position, various warning marks, an odd / trip meter, an operating state of the in-vehicle device, and the like. For example, an arrow mark is displayed on the side mirror 15 as guidance information. The side mirror 15 has, for example, a structure having a display member such as a liquid crystal inside the half mirror, and when the display member does not display display information, a landscape behind is reflected on the half mirror. Therefore, the display information may be displayed not only on the side mirror 15 but also on a mirror such as a room mirror or a vanity mirror.

  The blur generation unit 19 generates blur in the display information. FIG. 5 is a diagram illustrating an example of blur. 5 (a) shows an icon indicating a parking lot displayed on the navigation screen 12, FIG. 5 (b) shows an own vehicle icon displayed on the navigation screen 12, and FIG. 5 (c) shows a meter display unit 13 or The vehicle speed displayed on the HUD 14 and the arrow displayed on the side mirror 15 are shown in FIG.

  The blur generation unit 19 performs blur processing on the original display information. Icon information that determines the shape and the like of the icon displayed on the navigation screen 12 is stored in advance as, for example, bitmap data. The blur processing can be performed by diffusing the outer portion of the bitmap data outward to reduce the dot density. For other display information stored in advance as an icon such as an arrow, blurring processing can be similarly performed.

  When numbers or characters such as a vehicle speed or a message are displayed information, there are cases where numbers or characters of a bitmap font are displayed and outline fonts are drawn and displayed. Among these, the numbers and characters of the bitmap font are determined in the same shape and size as the icons, so that the same blurring process as the icons can be performed. The outline font may be drawn once and converted into bitmap data and then subjected to the same processing.

  Further, even if a plurality of the same icons, characters or numerical values are displayed with their centers slightly shifted, the same display as in the blurring process can be obtained. In this case, blurring can be easily expressed by changing the color of an icon or the like that shifts the center (for example, changing the color of each icon displayed by shifting the center like a gradation) or changing the line width of a character.

  5A to 5D, some icons or characters of the display medium 20 are blurred. However, even if the entire display medium 20 (all icons and characters constituting the screen) is blurred. Good.

  Moreover, HUD14 is set as the structure which projects a display image on the semi-transmissive member provided in the window glass, and makes a driver | operator visually recognize this display image. For this reason, it is possible to blur by shifting the focus at the time of projection, and it is not necessary to perform blurring processing on the icon or the like, which is easy to realize. In this case, the entire display medium 20 is blurred.

  Then, the pupil tracking determination unit 18 detects the size of the pupil detected by the pupil detection unit 17 when the blur generation unit 19 blurs display information on the display member in the gaze direction detected by the face direction / gaze direction detection unit 16. It is determined whether or not the brightness has changed following the blur of the display information. When the pupil size changes following the tracking, it is determined that the driver is gazing at the direction.

[Procedure for determining whether or not you are gazing]
FIG. 6 is an example of a flowchart illustrating a procedure for determining whether or not the driver state detection device 100 is gazing at the display medium 20. The flowchart of FIG. 6 starts when the ignition is turned on.

  The driver monitor sensor 11 repeatedly captures a face image every predetermined cycle time (S10). The face direction / gaze direction detection unit 16 detects the face direction and the gaze direction from each face image (S20). Further, the pupil detection unit 17 detects the size of the pupil from the face image (S30).

  Next, the blur generation unit 19 generates blur on the display medium 20 in the viewing direction (S40). The blur generation process in step S40 may be performed, for example, when the same line-of-sight direction is detected continuously for a predetermined time (0.5 to several seconds).

  When a focus shift occurs, the pupil tracking determination unit 18 determines whether the size of the pupil has changed following the blur (S50). “Following” refers to focusing on the size of the pupil immediately after the occurrence of blurring. Further, the change in the size of the pupil may be increased or decreased. If a clear image cannot be obtained with respect to the blur, the human eye changes the pupil so as to become larger and also changes so as to become smaller. In step S50, the eye changes to either large or small. However, it is determined that the size of the pupil has changed. In order to detect more reliably, it may be determined that the size of the pupil has changed when the size of the pupil has changed to both large and small.

  When the size of the pupil changes following the blur (Yes in S50), the pupil follow-up determination unit 18 determines that the driver is gazing at the display medium 20 causing the blur (S60).

  By the way, when it is determined that the user is gazing, the gazing may be preferable or not. First, a case where gaze in a direction in which gaze is not preferable is detected will be described. For example, while driving straight ahead, gazing at the navigation screen 12, the meter display unit 13, and the side mirror 15 (the display medium 20 not in the front direction) for a predetermined time or more cannot be said to be an appropriate line-of-sight direction. The detection device 100 alerts the driver to return the line of sight to the front. The method of alerting is, for example, that the display medium being watched is known, so the driver can be surely alerted by displaying “Let's look at the front direction” on the display medium that has blurred. be able to. Note that a similar message may be output from a speaker by voice, or an alarm sound may be generated.

  It may be necessary for the driver to watch the display medium 20 other than the front direction while traveling. For example, there is a case where it is desired to gaze at the vehicle speed displayed on the meter display unit 13, and there is a case where it is desired to gaze at the side mirror 15 when the lane is changed. Therefore, the predetermined time until alerting the driver may be variable according to the display medium 20. For example, it is considered that the vehicle speed can be grasped in a short time, and the movement of the line of sight from the front direction may be small. Therefore, the predetermined time until alerting when the meter display unit 13 is watched can be shortened. Further, for example, in the case of the side mirror 15, it is not necessary to call attention when changing lanes, so that it can be set so as not to call attention from the time when the blinker is switched on to off. In addition, the navigation screen 12 can be set for a long period of time until the driver is alerted at the right or left turn point on the route to the destination. The predetermined time until alerting can be set short.

  A case where gaze is preferable will be described. The case where gaze is preferable is, for example, a case where the front direction is being gazeed while traveling straight ahead. When blur occurs in the HUD 14 arranged in the front direction and the size of the pupil changes, it is determined that the traveling direction is being watched. Note that since the image projected on the front window is a virtual image, gazing at the HUD 14 may be determined as gazing in the front direction.

  Returning to step S50, when the size of the pupil does not change following the blur (No in S50), the pupil following determination unit 18 determines that the driver is not gazing at the display medium that caused the blur (S70). ). Although it is not gazing, for example, it is not preferable that the line-of-sight direction is directed to the navigation screen 12, the meter display unit 13 or the side mirror 15 for a predetermined time or more during straight running. Alert the driver to return his gaze to the front. Since the driver is not gazing at the display medium 20, a message such as “Let's gaze at the front direction” may be output from the speaker by voice, or an alarm sound may be sounded.

  On the other hand, when the driver is not gazing at the front direction while traveling straight ahead, the driver's arousal level may be reduced. Therefore, in such a case, the driver state detection device 100 executes alerting control (buzzing of alarm sound, message output by voice, temperature adjustment, vibration generation, etc.) for promoting awakening.

  As for the timing of this alerting, when the line of sight is facing in the front direction while traveling straight ahead, the HUD 14 does not necessarily have to be watched. May output an unnecessary alarm.

  For this reason, for example, when it is determined that the front direction is not watched even if blur is generated several times, the driver's arousal level is determined to be drowsiness and the driver's arousal level is increased. It is preferable to adjust the timing of alerting such as alerting.

  Moreover, in HUD14 which emphasizes and displays a pedestrian like a night vision, for example, when it is determined that the gaze direction is directed to the front direction but is not gazing, the driver is alerted. It is possible to call attention to pedestrians at night.

  Since the driver state detection device 100 of the present embodiment determines not only the gaze direction or the dwell time in the gaze direction but also whether the driver is gazing at the gaze direction, the state where the attention is distracted, The state of consciousness of the driver can be detected by distinguishing from the state in which the direction is actually watched.

  In this embodiment, a driver state detection apparatus 100 that detects a decrease in the degree of gaze concentration in the line-of-sight direction will be described. FIG. 7 shows an example of the stimulus display 31 displayed on the display medium 20 by the driver state detection device 100. FIG. 7 shows the vehicle speed displayed on the meter display unit 13 or the HUD 14 and is displayed as “40 km / h”. When it is determined that the driver is pointing the line of sight toward the meter display unit 13 or the HUD 14, the driver state detection device 100 displays the stimulus display 31 around the driver's visual field. In the figure, a circular light spot blinks in the upper right of “40 km / h”. When the light spot is turned on or blinked, and the driver follows the direction of the line of sight in the direction of the light spot, the driver state detection device 100 determines that the driver's concentration has decreased.

  For example, when the stimulus display 31 is displayed on the HUD 14 and the driver turns the line of sight toward the stimulus display 31, it is determined that the driver's concentration on the gaze in the front direction is reduced. Further, for example, when the stimulus display 31 is displayed on the navigation screen 12 and the driver turns the line of sight toward the stimulus display 31, the driver is looking at the navigation screen 12, that is, the driver's view of the navigation screen 12. It is determined that the degree of concentration is decreasing.

  It can be presumed that the degree of concentration is in a state in which the driver's arousal level is reduced or the consciousness is decreasing although it is not drowsiness. Therefore, similarly to the first embodiment, the driver state detection device 100 according to the present embodiment can detect the driver's consciousness state by determining whether or not the sight line direction is concentrated.

FIG. 8 shows an example of a functional block diagram of the driver state detection device 100 of the present embodiment. In FIG. 8, the same components as those in FIG. In this embodiment, a visual stimulation unit 32 and a stimulation follow-up determination unit 33 are provided. As shown in FIG. 7, the visual stimulation unit 32 displays display information that stimulates the driver's vision around the visual field in the driver's line-of-sight direction. The display information that stimulates vision has the following modes.
I) The stimulus display 31 is displayed in addition to the vehicle status notification and display information for driving support.
II) Increase the brightness of some of the display information for vehicle status notifications and driving assistance.
III) A part of display information for notification of vehicle status and driving support is transformed into a stimulus display 31.
IV) Processes I to III are performed not on the display medium 20 in the line-of-sight direction but on the display medium 20 adjacent to the display medium 20 in the line-of-sight direction.

  In addition, the visual field periphery should just be inside the driver | operator's maximum visual field except the center part of a visual field. Since the size and the arrangement direction seen from the driver are different for each display medium 20, the periphery of the visual field is not uniquely determined for the same driver. For example, in the navigation screen 12, the lower part of the screen is the periphery of the visual field, and in the meter display unit 13 and the HUD 14, the vicinity of the visual field is the vicinity of the visual field.

  FIG. 9A shows an example of the stimulus display 31 displayed in addition to the display information for vehicle status notification and driving support (I). For example, a stimulus display 31 “sunny and cloudy” is displayed around the vehicle speed “40 km / h” displayed on the HUD 14 and the meter display unit 13. A meaningful stimulus display 31 such as a character can be said to be more stimulating than a simple light spot as shown in FIG. It can be detected. Moreover, news may be used as the stimulus display 31, for example, only one kanji character may be used as the stimulus display 31, and an icon that makes an image of animals and plants, industrial products, anime characters, etc. may be used as the stimulus display 31. .

  FIG. 9B shows an example of the stimulus display 31 formed by increasing the luminance of a part of display information for vehicle status notification and driving support (II). In FIG. 9B, the brightness of the building and the section on the lower left of the navigation screen 12 is greatly displayed. The driver usually travels with the traveling direction upward, for example, and gazes in the direction to pass, so the lower part of the navigation screen 12 is around the visual field as shown in FIG. 9B. When the gaze direction of the driver changes following the visual stimulus given by increasing the luminance around the visual field, it is determined that the degree of gaze concentration on the navigation screen 12 is low.

  FIG. 9C shows an example of the stimulus display 31 in which a part of the display information is deformed and displayed (III). Similarly to FIG. 9B, a stimulus display 31 “sunny and cloudy” is displayed on a building and a section around the visual field. Not only characters but also icons may be displayed.

  Further, as shown in FIGS. 9A to 9C, the stimulus display 31 is not displayed on the display medium 20 in the line-of-sight direction, but is displayed on the display medium 20 adjacent to the display medium 20 in the line-of-sight direction. May be. For example, the HUD 14 and the meter display unit 13 and the meter display unit 13 and the navigation screen 12 may be arranged adjacent to each other. When the entire surface of the meter display unit 13 is composed of liquid crystal or the like, the meter display unit 13 is disposed directly below the HUD 14. Therefore, for example, in the state where the line-of-sight direction is in the front direction, not only the HUD 14 but also the meter display unit 13 is included in the periphery of the visual field, and whether or not the front direction is watched by displaying the stimulus display 31 on the meter display unit 13. Can be determined.

  Returning to FIG. 8, when the visual stimulus unit 32 displays the stimulus display 31 on the display medium 20 in the line-of-sight direction detected by the face direction / line-of-sight direction detection unit 16, the stimulus follow-up determination unit 33 It is determined whether or not the line-of-sight direction detected by the direction detection unit 16 has changed following. When the line-of-sight direction changes following, it is determined that the driver's gaze concentration is decreasing.

  FIG. 10 is an example of a flowchart illustrating a procedure in which the driver state detection device 100 determines the degree of gaze concentration of the display medium 20. The flowchart in FIG. 10 starts when the ignition is turned on, for example.

  The driver monitor sensor 11 repeatedly captures a face image every predetermined cycle time (S10). The face direction / gaze direction detection unit 16 detects the face direction and the gaze direction from each face image (S20). When the line-of-sight direction is determined, the visual stimulation unit 32 estimates the visual field periphery in accordance with the line-of-sight direction and the display medium 20 in the line-of-sight direction (S130). As described above, for example, in the case of the navigation screen 12, the periphery of the visual field is the lower part of the screen. A relationship between the line-of-sight direction and the periphery of the visual field may be determined and stored in advance.

  Next, the visual stimulation unit 32 displays the stimulation display 31 around the visual field of the display medium 20 in the line-of-sight direction (S140). The display of the stimulus display 31 in step S140 may be performed, for example, when the same line-of-sight direction is detected continuously for a predetermined time (0.5 to several seconds).

  When the stimulus display 31 is displayed, the stimulus follow-up determination unit 33 determines whether the line-of-sight direction has changed following the stimulus display 31 (S150). “Following” refers to focusing on the line of sight immediately after displaying the stimulus display 31. The change in the line-of-sight direction is a change in the line-of-sight direction toward the stimulus display 31.

  When the line-of-sight direction changes following the display of the stimulus display 31 (Yes in S150), the stimulus follow-up determination unit 33 determines that the driver's gaze concentration is decreasing with respect to the display medium 20 in the line-of-sight direction. (S160).

  When the degree of concentration in the line of sight decreases, the driver's arousal level may decrease. Therefore, the driver state detection device 100 outputs a message such as “Let's watch the front”, “Let's take a break”, “Let's ventilate” from the speaker or sound an alarm sound. To do.

  Although the gaze direction is directed to the front direction, if it is determined that the concentration of gaze is reduced, it is considered that the degree of arousal is particularly low, so the driver can be warned early. On the other hand, if it is determined that the concentration of gaze on the display medium 20 is preferably not necessarily watched, such as the navigation screen 12, the driver is warned about once every three times. Therefore, it is preferable to change the frequency of warning to the driver.

  When the line-of-sight direction does not change following the display of the stimulus display 31 (No in S150), the stimulus follow-up determination unit 33 determines that the driver's gaze concentration does not decrease with respect to the display medium 20 in the line-of-sight direction. To do. In this case, since it may be determined that the driver's arousal level is high, the process is terminated as it is.

  The driver state detection device 100 according to the present embodiment determines not only the line-of-sight direction or the stop time in the line-of-sight direction but also whether or not the driver is looking at the line-of-sight direction. Can be detected.

It is an example of the figure explaining the outline of determination of whether it is gazing. It is an example of the functional block diagram of a driver | operator state detection apparatus. It is an example of the front view of a driver monitor sensor. It is an example of the figure which shows the detection of a gaze direction typically. It is a figure which shows an example of a blur. It is an example of the flowchart figure which shows the procedure which determines whether a driver | operator state detection apparatus is gazing at a display medium. It is a figure which shows an example of the display for a stimulus which a driver | operator state detection apparatus displays on a display medium. It is an example of the functional block diagram of a driver | operator state detection apparatus. It is a figure which shows an example of the display for stimulation displayed other than the display information for notification of a vehicle condition or driving assistance. FIG. 10 is an example of a flowchart illustrating a procedure for determining a concentration level of gaze of a display medium by a driver state detection device.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Driver monitor sensor 16 Face direction / gaze direction detection part 17 Pupil detection part 19 Blur generation part 20 Display medium 32 Visual stimulus part 100 Driver state detection apparatus

Claims (4)

In the driver state detection device that detects the driver's consciousness state,
Display means for displaying display information whose contents are variable;
Photographing means for photographing a driver's face image;
A face direction / gaze direction detecting means for detecting at least one of the face direction and the gaze direction from the face image;
A screen changing unit arranged in the line-of-sight direction detected by the face direction / line-of-sight direction detecting unit, and changing a part or all of the screen generated by the display unit;
A driver awareness determining means for detecting a driver's consciousness state based on whether or not the driver's eye state has changed following the change of part or all of the screen by the screen changing means;
Pupil detection means for detecting the size of the driver's pupil from the face image;
The screen changing means blurs the outer edge of part or all of the display information unclearly,
The driver awareness determination means detects the driver's awareness state based on whether or not the size of the pupil of the driver has changed following the blur of the display information .
A driver state detection device characterized by the above. The screen changing means estimates the driver's visual field periphery based on the line-of-sight direction, forms a stimulus display that stimulates vision on a part of the screen included in the visual field periphery,
The driver awareness determination means, line-of-sight direction of the driver following the formation of the display the stimulus is based on whether a change is detected consciousness state of the driver,
The driver state detection device according to claim 1. The screen changing means displays the stimulus display while displaying the display information.
The driver state detection device according to claim 2 . Display means for displaying display information whose contents are variable;
In a consciousness state detection method of a driver state detection device comprising a photographing means for photographing a driver's face image,
A face direction / line-of-sight direction detecting unit detecting at least one of the face direction and the line-of-sight direction from the face image;
A screen changing step for changing part or all of the screen of the display means, the screen changing means being arranged in the line-of-sight direction detected by the face direction / line-of-sight direction detecting means;
The driver awareness determination means, based on whether to follow a part or all screen changes according to the change screen means the state of the driver's eyes is changed, the driver awareness for detecting the state of consciousness of the driver A determination step;
The pupil detecting means detects the size of the driver's pupil from the face image; and
The screen changing means blurs a part or all of the outer edge of the display information unclearly in the screen changing step,
The driver awareness determination means detects a driver awareness state based on whether or not the size of the driver's pupil has changed following the blur of the display information in the driver awareness determination step. ,
A consciousness state detection method characterized by the above.