CN103677270A - Human-computer interaction method based on eye movement tracking - Google Patents

Abstract

The invention provides a human-computer interaction method based on eye movement tracking, wherein five infrared source sets which face a computer operator and a camera with a built-in infrared filter are used, four infrared source sets are arranged at four corners of a computer display screen, and the other infrared source set is arranged surrounding a lens of the camera; the camera is arranged below the computer display screen and connected to a computer, wherein the lens of the camera faces the face of the operator. The human-computer interaction method comprises the steps of calibration, image capturing and detection, blink judgment, fixation point judgment and interaction instruction output. The human-computer interaction method based on eye movement tracking has the advantages that calibration is simple and portability is good; the number of users who the method is applicable to is increased due to detection of dark pupils; interference caused by physiological human eye tremors is overcome through smoothing; system accuracy is still high even if the head of the operator moves within a wide range; only one camera is used, so that cost is lower; processing speed is quite high, and real-time human-computer interaction can be realized.

Description

A kind of man-machine interaction method based on eye-tracking

Technical field

The present invention relates to computer vision control technology field, is specifically a kind of man-machine interaction method based on eye-tracking.

Background technology

In man-machine interaction, eye-tracking is being played the part of a very important role, and eye-tracking can be taken as the interface that connects people and computing equipment, and compared with mouse and keyboard, eye-tracking offers alternately a kind of more naturally mode of people and carries out man-machine interaction.The method that estimation eye movement realizes direction also has a variety of: reflection method, potential electronics skin, and contact lenses etc., these methods are classified as again the method for intrusive mood or non-intrusion type, the method of non-intrusion type more has superiority, more comfortable because the method for non-intrusion type uses.But in eye-tracking research, still also have many open questions.Its precision problem the most generally, the restriction of head movement, robustness, and the easness of demarcating.

Summary of the invention

The object of this invention is to provide a kind of man-machine interaction method based on eye-tracking, this method makes operator's head still can keep higher degree of accuracy when moving in a big way.

The technical scheme that realizes the object of the invention is as follows: a kind of man-machine interaction method based on eye-tracking, the camera that comprises five groups of infrared light supplies towards computer operation person and a built-in infrared optical filter, wherein four groups of infrared light supplies are arranged on four corners of computer display, and another group infrared light supply is arranged on around camera lens; Camera is arranged on computer display below and is connected to computing machine, and camera lens is towards operator's face; Comprise

The step of demarcating, comprises

101: face-image when computing machine is caught operator and watched attentively the infrared light supply in the arbitrary corner of display screen by camera, image is carried out to eye detection, spot detection and pupil detection, obtain pupil center, camera lens corresponding spot center and spot center corresponding to this group infrared light supply of infrared light supply around, calculate corresponding calibration coefficient;

102: according to aforesaid operations, corresponding calibration coefficient when calculating operation person watches the infrared light supply in other corner of display screen attentively respectively; Catch the step with detected image, comprise

201: computing machine continues to catch with frame frequency F the face-image that operator watches display screen by camera;

202: each two field picture is carried out to eye detection, spot detection, pupil detection and fixation point and estimate;

The step that judgement is blinked, comprises

301: the time period T that any time t is starting point is take in judgement _lin the L frame consecutive image of catching, L=T _l* F; If there is and only has a time period T in L two field picture _kin the continuous image of K frame of catching hot spot do not detected, T _k<T _l, be set as operator and blink once; If there are two time period T in L two field picture _k1and T _k2in K1 and the continuous image of K2 frame of catching separately hot spot do not detected, T _k1<T _l, T _k2<T _land T _k1+ T _k2<T _l, and other image detection between two time periods consecutive image of catching is separately to hot spot, is set as operator and blinks twice;

The step that judgement fixation point changes:

401: establish fixation point in that two field picture that any time t the catches horizontal ordinate g on screen _xordinate g _y;

402: the time period T that moment t is starting point is take in judgement _rin R frame consecutive image, R=T _r* F, if the fixation point in R two field picture all rests on horizontal ordinate g _xordinate g _yfor the center of circle, in the circle that radius is r, be set as fixation point and stop;

403: the time period T that moment t is starting point is take in judgement _din D frame consecutive image, D=T _d* F, if the horizontal ordinate of the fixation point in D two field picture and ordinate dullness reduce, and total decrease surpasses respectively horizontal ordinate variable quantity X and ordinate variable quantity Y, is set as fixation point and moves to upper left side;

404: the time period T that moment t is starting point is take in judgement _uin U frame consecutive image, U=T _u* F, if the horizontal ordinate of the fixation point in U two field picture and ordinate monotone increasing, and total recruitment surpasses respectively horizontal ordinate variable quantity X and ordinate variable quantity Y, is set as fixation point and moves to lower right;

The step of output interactive instruction, comprises

501: as operator blinks once, export the instruction that left mouse button is clicked;

502: as operator blinks twice, export the instruction that left mouse button is double-clicked;

503: as fixation point stops, export the instruction that right mouse button is clicked;

504: as fixation point moves to upper left side, export the instruction that mouse roller scrolls up;

505: as fixation point moves to lower right, export the instruction that mouse roller rolls downwards.

Further,

Described eye detection comprises: the face-image of catching is carried out to binaryzation, obtain black white image; Black white image is carried out to profile and search, determine that the minimum rectangle frame of the profile border parcel finding is human eye rectangle frame;

Described spot detection comprises: according to human eye rectangle frame intercepting face-image, obtain human eye rectangle frame image; Human eye rectangle frame image is carried out to binaryzation, obtain binary image; Binary image is removed to noise; Search five white portions of area maximum in the binary image of removing after noise as five groups of hot spots that infrared light supply is corresponding; The barycenter of determining five hot spots is spot center; Determine the one-to-one relationship of five hot spots and five groups of infrared light supplies;

Described pupil detection comprises: take respectively the mean value of horizontal ordinate of five spot center and initialization abscissa value and the initialization ordinate value that the mean value of ordinate is pupil center; Set pupil rectangle frame, take the initialization abscissa value of pupil center and the center that initialization ordinate value is pupil rectangle frame; Pupil rectangle frame is carried out to vertical and horizontal projecting integral, obtain only comprising in pupil rectangle frame the little rectangle frame of pupil; From little rectangle frame, search pupil boundary and center thereof;

Described fixation point is estimated to comprise: according to five spot center and calibration coefficient, calculate respectively spot center that the infrared light supply in four corners of display screen is corresponding four virtual projection points on eyes cornea; According to pupil center and four virtual projection points, and the length of display screen and width, estimate fixation point.

Further, described time period T _l, time period T _r, time period T _dwith time period T _ube at the same time section.

Further, described frame frequency F was 15 frame/seconds; Described time period T _l, time period T _r, time period T _dwith time period T _uduration be 2 seconds; Described time period T _k, time period T _k1with time period T _k2all be greater than 0.5 second; Described radius r is

described horizontal ordinate variable quantity X and ordinate variable quantity Y are respectively

with

wherein w is display screen width, and h is display screen height.

Beneficial effect of the present invention is, 1, demarcate simply, and transplantability is good; 2, by dark pupil is detected, avoided different human eyes, the bad user of bright pupil effect, has increased applicable user; 3,, by smoothing processing, overcome the interference that the vibration of people's physiology of eye brings; 4, operator's head mobile system degree of accuracy in is in a big way still higher; 5, use single camera, compare with using a plurality of cameras, cost is relatively low; 6, processing speed is very fast, and system responses is quicker, in time, can meet real-time man-machine interaction.

Accompanying drawing explanation

Fig. 1 is the schematic diagram of infrared light supply, camera and the face-image of catching, human eye rectangle frame, human eye rectangle frame image;

Fig. 2 is the schematic diagram that in human eye rectangle frame image, hot spot distributes;

Fig. 3, Fig. 4 and Fig. 5 are the principle schematic that fixation point is estimated;

Fig. 6 is the schematic diagram that fixation point stops;

Fig. 7 is the schematic diagram that fixation point moves to upper left side;

Fig. 8 is the schematic diagram that fixation point moves to lower right.

Embodiment

As shown in Figure 1, a kind of man-machine interaction method based on eye-tracking, the camera that comprises five groups of infrared light supplies towards computer operation person and a built-in infrared optical filter, wherein four groups of infrared light supply LED1, LED2, LED3 and LED4 are arranged on four corners of computer display, and another group infrared light supply LED5 is arranged on around camera lens 3; Camera is arranged on computer display below and is connected to computing machine, and camera lens 3 is towards operator's face.

The method of eye detection is: infrared light supply reflects and produces the pul speck of admiring in human eye, in the grey face-image 1 of catching with the camera of optical filter, the pul being positioned on eye cornea is admired speck for the brightest, the position of searching human eye can change into searches the admire position of speck of the brightest pul, the position of searching left eye or right eye can change into search abscissa value minimum or the admire position of speck of maximum pul.Pass through binaryzation, obtain pul and admire speck region for white, all the other positions are the black white image of black, this black white image is carried out to profile and search (border of searching monochrome pixels), and the minimum rectangle frame of definite profile border parcel finding is human eye rectangle frame 2 as the region at human eye place.

The method of spot detection is:

Step1: use human eye rectangle frame 2 intercepting face-images, obtain human eye rectangle frame image 2 ';

Step2: human eye rectangle frame image is cut apart by setting threshold, obtained a binary image;

Step3: process binary image and morphological operation to remove noise;

In binary image, infrared lamp reflection produces the hot spot that five brightness values are 1 (pul admire speck), and different hot spots forms different white portions.But, due to the existence of noise, and the external environment illumination condition of different brightness, also may produce brightness value and be 1 other white portion.In addition, improper due to selected threshold, the single hot spot that infrared lamp reflection produces also may be divided into a plurality of fritters, form polylith white portion, by first corroding again expansion form, learn operation (< < study Opencv (Chinese edition) > >) and eliminate the interference that noise that part area is less brings here.

Step4: group areas;

After morphological operation, now in binary image, be only left the relatively large white portion of some areas.We find, noise place white portion area is less than the white portion area at hot spot place, therefore only need in binary image, find out five white portions that white portion is hot spot place of area maximum.By the cvFindContours function of increasing income in the Opencv of computer vision storehouse, search the profile of every white portion in binary image, recycling cvContourArea function calculates respectively the area that every white portion comprises (pixel and).Each region area is sorted according to descending, and the region of choosing five area maximums after sequence is spot area.

Step5: the center of calculating every spot area;

By calculating above, five hot spots have all been detected, and obtain the barycenter of each spot area as spot center.

Step6: the one-to-one relationship of determining five hot spots and five groups of infrared light supplies;

Relative position relation by five groups of infrared light supplies can be known, ordinate value maximum be LED5, abscissa value minimum be LED1 and LED3, abscissa value maximum be LED2 and LED4, and the ordinate value of LED1 is less than the ordinate value of LED3, the ordinate value of LED2 is less than the ordinate value of LED4.According to this relation, compare the coordinate figure of the spot center of five hot spots, can determine the one-to-one relationship of five hot spots and five groups of infrared light supplies.As Fig. 2, in human eye rectangle frame image, hot spot LED1 ', LED2 ', LED3 ', LED4 ' and LED5 ' are corresponding one by one with infrared light supply LED1, LED2, LED3, LED4 and LED5.

The method of pupil detection is:

Step1: initialization pupil region center;

Before obtained the center of five hot spots.In fact, the position of five hot spots and pupil is separated by very near, therefore we use respectively the horizontal ordinate of five spot center and ordinate mean value as the initialization coordinate figure of pupil center, using this centre coordinate as the center of pupil rectangle frame, and the large I of this rectangle frame is set (general length and width are 160*120 pixel) according to actual conditions.

Step2: iterative projection integration

Above-mentioned pupil rectangle frame is carried out to vertical and horizontal projecting integral, because it is dark that want in other regions of the relative around eyes of pupil, by vertical and horizontal projection integration, each row pixel value of every a line and minimum coordinate are all found out, by the projecting integral of iteration, we can access the little rectangle frame (length and width are 80*60 pixel) that only comprises pupil in pupil rectangle frame.By Opencv binaryzation function cvThreshold, profile is searched function cvFindContours again, and Hough transformation is searched circular function cvHoughCircles, can find pupil boundary and center thereof.

The method of demarcating is:

Face-image when computing machine is caught operator and watched attentively the infrared light supply in the arbitrary corner of display screen by camera, image is carried out to eye detection, spot detection and pupil detection, obtain pupil center, camera lens corresponding spot center and spot center corresponding to this group infrared light supply of infrared light supply LED5 around, calculate corresponding calibration coefficient.

Suppose that operator watches LED1 attentively, computing machine is caught its face-image by camera.By eye detection, spot detection and pupil detection, obtain the u of pupil center _p, camera lens spot center u corresponding to infrared light supply LED5 around _cthe spot center u corresponding with infrared light supply LED1 _r1.Because that operator watches attentively is LED1, the corresponding virtual projection point of hot spot that LED1 produces should with the center superposition of pupil, by this constraint, we can calculate this coefficient in order to lower equation:

${\mathrm{\&alpha;}}_1=\frac{d(u_p,u_c)}{d(u_{r1},u_c)}$

In above formula, α ₁the calibration coefficient of operator while watching LED1 attentively, d (x ₁, x ₂) be an x ₁, x ₂between Euclidean distance.When user watches respectively LED2, LED3 and LED4 attentively, according to said method, can calculate corresponding calibration coefficient α ₂, α ₃and α ₄.

The method that fixation point is estimated is:

By the unchangeability of double ratio in projection process, estimate the position of eye gaze point.In Fig. 3, supposing has a tangent virtual section X with it at anterior corneal surface, some v ₁, v ₂, v ₃, v ₄on this virtual section, be four groups of infrared light supply LED1 of corner screen, LED2, LED3 and LED4 are at the subpoint in this virtual section, and we are referred to as virtual projection point these four points, and their projection centre is cornea ball centre J.Suppose that these virtual projection points are roughly coplanar, the quadrilateral that these four virtual point are linked to be is so exactly the projection of screen display, and these virtual projections Dian He P of pupil center is projected into five some u on camera image plane T _v1, u _v2, u _v3, u _v4and u _p, Z is camera optical axis center.Thereby, from screen display to the plane of delineation, there are two projection conversions here, if this virtual projection point is roughly coplanar, fixation point g point coordinate just can be estimated to obtain by calculating with Projective invariance.

Specific as follows: as shown in Figure 4, u _v1, u _v2, u _v3, u _v4the virtual projection point on cornea, u _pbe pupil center in image, c is u _v1, u _v2, u _v3, u _v4the point of crossing of the quadrilateral summit line forming.Wherein, u _v1=u _c+ α ₁(u _r1-u _c), u _v2=u _c+ α ₂(u _r2-u _c), u _v3=u _c+ α ₃(u _r3-u _c), u _v4=u _c+ α ₄(u _r4-u _c), u in formula _ccamera lens spot center corresponding to infrared light supply LED5 around, u _r1the spot center that infrared light supply LED1 is corresponding, u _r2the spot center that infrared light supply LED2 is corresponding, u _r3the spot center that infrared light supply LED3 is corresponding, u _r4it is the spot center that infrared light supply LED4 is corresponding.

U _m1straight line and u for end point a and c place _v1, u _v2the intersection point of place straight line, u _m2for end point a and u _pthe straight line at place and u _v1, u _v2the intersection point of place straight line, u _m3for end point b and u _pthe straight line at place and u _v2, u _v4the intersection point of place straight line, u _m4straight line and u for end point b and c point place _v2, u _v4the intersection point of place straight line.

Suppose $u_{\mathrm{vi}}=(x_i^v,y_i^v)(i=\mathrm{1,2,3,4}),u_{\mathrm{mi}}=(x_i^m,y_i^m)(i=\mathrm{1,2,3,4}),$ Straight line

the double ratio of upper four points is:

${\mathrm{CR}}_{\mathrm{image}}^x=\frac{(x_1^v{y}_1^m-x_1^m{y}_1^v)(x_2^m{y}_2^v-x_2^v{y}_2^m)}{(x_1^v{y}_2^m-x_2^m{y}_1^v)(x_2^m{y}_2^v-x_2^v{y}_1^m)}$

Same, the double ratio of screen display, as shown in Figure 5, adopts following equation to calculate:

${\mathrm{CR}}_{\mathrm{image}}^x=\frac{(w-\frac{w}{2}){\hat{x}}_g}{(w-{\hat{x}}_g)\frac{\mathrm{<figure-callout\; id="2"\; label="w"\; filenames="HDA0000437760490000011.png"\; state="\{\{state\}\}">w</figure-callout>}}{2}}=\frac{{\hat{x}}_g}{w-{\hat{x}}_g}$

W is the width of screen display,

it is the x coordinate of estimation point g.

According to the cross ratio invariability character of projector space, those double ratios equate.So

${\hat{x}}_g=\frac{w\&CenterDot;{\mathrm{CR}}_{\mathrm{image}}^x}{1+{\mathrm{CR}}_{\mathrm{image}}^x}$

The y coordinate that g is ordered can be estimated equally in this way, and image double ratio is:

${\mathrm{CR}}_{\mathrm{image}}^y=\frac{(x_2^v{y}_3^m-x_3^m{y}_2^v)(x_4^m{y}_3^v-x_3^v{y}_4^m)}{(x_2^v{y}_4^m-x_4^m{y}_2^v)(x_3^m{y}_3^v-x_3^v{y}_3^m)}$

The double ratio of screen display is:

${\mathrm{CR}}_{\mathrm{image}}^y=\frac{(h-\frac{h}{2}){\hat{y}}_g}{(h-{\hat{y}}_g)\frac{\mathrm{<figure-callout\; id="2"\; label="h"\; filenames="HDA0000437760490000011.png"\; state="\{\{state\}\}">h</figure-callout>}}{2}}=\frac{{\hat{y}}_g}{h-{\hat{y}}_g}$

H is the height of screen display,

it is the y coordinate of estimation point g.And

${\hat{y}}_g=\frac{h\&CenterDot;{\mathrm{CR}}_{\mathrm{image}}^y}{1+{\mathrm{CR}}_{\mathrm{image}}^y}$

The implementing procedure of the man-machine interaction method based on eye-tracking is: the step of first carry out demarcating, and by human eye detection, spot detection, pupil detection, and user watches respectively four corners of screen attentively and demarcates, and obtains calibration coefficient α ₁, α ₂, α ₃, α ₄.After demarcation completes, carry out and catch and detect, be specially: computing machine continues to catch with frame frequency F the face-image that operator watches display screen by camera; Each two field picture is carried out to eye detection, spot detection, pupil detection and fixation point to be estimated.The method that eye detection, spot detection, pupil detection and fixation point are estimated is as aforementioned.According to the result detecting and fixation point is estimated, the judgement that blink judgement and fixation point change.

The method of judgement nictation is: the time period T that any time t is starting point is take in judgement _lin the L frame consecutive image of catching, L=T _l* F; If there is and only has a time period T in L two field picture _kin the continuous image of K frame of catching hot spot do not detected, T _k<T _l, be set as operator and blink once; If there are two time period T in L two field picture _k1and T _k2in K1 and the continuous image of K2 frame of catching separately hot spot do not detected, T _k1<T _l, T _k2<T _land T _k1+ T _k2<T _l, and other image detection between two time periods consecutive image of catching is separately to hot spot, is set as operator and blinks twice.

The method that judgement fixation point changes is: as shown in Figure 6, establish fixation point in that two field picture that any time t the catches horizontal ordinate g on screen _xordinate g _y; The time period T that moment t is starting point is take in judgement _rin R frame consecutive image, R=T _r* F, if the fixation point in R two field picture all rests on horizontal ordinate g _xordinate g _yfor the center of circle, in the circle that radius is r, be set as fixation point and stop.As shown in Figure 7, the time period T that moment t is starting point is take in judgement _din D frame consecutive image, D=T _d* F, if the horizontal ordinate of the fixation point in D two field picture and ordinate dullness reduce, and total decrease surpasses respectively horizontal ordinate variable quantity X and ordinate variable quantity Y, is set as fixation point and moves to upper left side.As shown in Figure 8, the time period T that moment t is starting point is take in judgement _uin U frame consecutive image, U=T _u* F, if the horizontal ordinate of the fixation point in U two field picture and ordinate monotone increasing, and total recruitment surpasses respectively horizontal ordinate variable quantity X and ordinate variable quantity Y, is set as fixation point and moves to lower right.

The method of output interactive instruction is: according to judged result, as operator blinks once, export the instruction that left mouse button is clicked; As operator blinks twice, export the instruction that left mouse button is double-clicked; As fixation point stops, export the instruction that right mouse button is clicked; As fixation point moves to upper left side, export the instruction that mouse roller scrolls up; As fixation point moves to lower right, export the instruction that mouse roller rolls downwards.

In actual judgement, for simplifying the operation, can take out one group of continuous image since any time, this group image is blinked successively and judged and fixation point variation judgement.That is to say, the frame number of the consecutive image that blink judgement and fixation point change to judge is identical, i.e. L=R=D=U, and the time period of catching this framing is also identical, i.e. time period T _l, time period T _r, time period T _dwith time period T _ube at the same time section.If meet nictation or fixation point in this group image, change one of five kinds of situations of judgement, operational order corresponding to instant output just, usings the next frame image of current frame image simultaneously as the start frame of next group consecutive image; If all do not met, do not export any instruction, again get one group of continuous image and judge next time, the start frame that the second frame of above one group of consecutive image is this group consecutive image.

For example, computing machine is caught after operator's face-image with the frame frequency of 15 frame/seconds continuously by camera, carry out eye detection, spot detection, pupil detection and fixation point and estimate, from any frame, start to get one group of 30 continuous two field picture afterwards, the duration of this 30 two field picture is 2 seconds.Suppose in this 30 two field picture, the 3rd frame to the 10 frames all do not detect hot spot, and the time that respective operations person closes one's eyes is greater than 0.5 second, and other image all detects hot spot, can regard as operator and blink once, correspondingly export the instruction that left mouse button is clicked.Suppose in this 30 two field picture, the 3rd frame to the 10 frames and the 20th frame to the 27 frames all do not detect hot spot, and other image all detects hot spot, can regard as operator and blink twice, correspondingly export the instruction that left mouse button is double-clicked.Suppose that, in this 30 two field picture, the fixation point of the first frame is horizontal ordinate g _xordinate g _y, the fixation point of other all frames is with horizontal ordinate g _xordinate g _yfor the center of circle, within the circle that radius is r, can regard as fixation point and stop, correspondingly export the instruction that right mouse button is clicked.Radius r can be set as 1/20th of display screen width w.Suppose that, in this 30 two field picture, the horizontal ordinate of fixation point and ordinate dullness reduce, and the total decrease of horizontal ordinate surpasses

the total decrease of ordinate surpasses

can regard as fixation point and move to upper left side, correspondingly export the instruction that mouse roller scrolls up.Suppose in this 30 two field picture, the horizontal ordinate of fixation point and ordinate are dull to be increased, and the total increase of horizontal ordinate surpasses the total increase of ordinate surpasses

can regard as fixation point and move to lower right, correspondingly export the instruction that mouse roller rolls downwards.Here h is the height of display screen.According to the instruction of output, just can carry out some shirtsleeve operations, such as controlling ppt, webpage rolling etc., thereby reach the object of man-machine interaction.

Claims (5)

1. the man-machine interaction method based on eye-tracking, the camera that comprises five groups of infrared light supplies towards computer operation person and a built-in infrared optical filter, wherein four groups of infrared light supplies (LED1, LED2, LED3, LED4) are arranged on four corners of computer display, and another group infrared light supply (LED5) is arranged on around camera lens; Camera is arranged on computer display below and is connected to computing machine, and camera lens is towards operator's face; It is characterized in that, comprise

The step of demarcating, comprises

101: face-image when computing machine is caught operator and watched attentively the infrared light supply in the arbitrary corner of display screen by camera, image is carried out to eye detection, spot detection and pupil detection, obtain spot center and the spot center corresponding to this group infrared light supply of pupil center, camera lens infrared light supply (LED5) correspondence around, calculate corresponding calibration coefficient;

102: according to aforesaid operations, corresponding calibration coefficient when calculating operation person watches the infrared light supply in other corner of display screen attentively respectively;

Catch the step with detected image, comprise

201: computing machine continues to catch with frame frequency F the face-image that operator watches display screen by camera;

202: each two field picture is carried out to eye detection, spot detection, pupil detection and fixation point and estimate;

The step that judgement is blinked, comprises

301: the time period T that any time t is starting point is take in judgement _lin the L frame consecutive image of catching, L=T _l* F; If there is and only has a time period T in L two field picture _kin the continuous image of K frame of catching hot spot do not detected, T _k<T _l, be set as operator and blink once; If there are two time period T in L two field picture _k1and T _k2in K1 and the continuous image of K2 frame of catching separately hot spot do not detected, T _k1<T _l, T _k2<T _land T _k1+ T _k2<T _l, and other image detection between two time periods consecutive image of catching is separately to hot spot, is set as operator and blinks twice;

The step that judgement fixation point changes:

401: establish fixation point in that two field picture that any time t the catches horizontal ordinate g on screen _xordinate g _y;

402: the time period T that moment t is starting point is take in judgement _rin R frame consecutive image, R=T _r* F, if the fixation point in R two field picture all rests on horizontal ordinate g _xordinate g _yfor the center of circle, in the circle that radius is r, be set as fixation point and stop;

403: the time period T that moment t is starting point is take in judgement _din D frame consecutive image, D=T _d* F, if the horizontal ordinate of the fixation point in D two field picture and ordinate dullness reduce, and total decrease surpasses respectively horizontal ordinate variable quantity X and ordinate variable quantity Y, is set as fixation point and moves to upper left side;

404: the time period T that moment t is starting point is take in judgement _uin U frame consecutive image, U=T _u* F, if the horizontal ordinate of the fixation point in U two field picture and ordinate monotone increasing, and total recruitment surpasses respectively horizontal ordinate variable quantity X and ordinate variable quantity Y, is set as fixation point and moves to lower right;

The step of output interactive instruction, comprises

501: as operator blinks once, export the instruction that left mouse button is clicked;

502: as operator blinks twice, export the instruction that left mouse button is double-clicked;

503: as fixation point stops, export the instruction that right mouse button is clicked;

504: as fixation point moves to upper left side, export the instruction that mouse roller scrolls up;

505: as fixation point moves to lower right, export the instruction that mouse roller rolls downwards.

2. man-machine interaction method as claimed in claim 1, is characterized in that,

Described eye detection comprises: the face-image of catching is carried out to binaryzation, obtain black white image; Black white image is carried out to profile and search, determine that the minimum rectangle frame of the profile border parcel finding is human eye rectangle frame;

Described spot detection comprises: according to human eye rectangle frame intercepting face-image, obtain human eye rectangle frame image; Human eye rectangle frame image is carried out to binaryzation, obtain binary image; Binary image is removed to noise; Search five white portions of area maximum in the binary image of removing after noise as five groups of hot spots that infrared light supply is corresponding; The barycenter of determining five hot spots is spot center; Determine the one-to-one relationship of five hot spots and five groups of infrared light supplies;

Described pupil detection comprises: take respectively the mean value of horizontal ordinate of five spot center and initialization abscissa value and the initialization ordinate value that the mean value of ordinate is pupil center; Set pupil rectangle frame, take the initialization abscissa value of pupil center and the center that initialization ordinate value is pupil rectangle frame; Pupil rectangle frame is carried out to vertical and horizontal projecting integral, obtain only comprising in pupil rectangle frame the little rectangle frame of pupil; From little rectangle frame, search pupil boundary and center thereof;

Described fixation point is estimated to comprise: according to five spot center and calibration coefficient, calculate respectively spot center that the infrared light supply in four corners of display screen is corresponding four virtual projection points on eyes cornea; According to pupil center and four virtual projection points, and the length of display screen and width, estimate fixation point.

3. man-machine interaction method as claimed in claim 1, is characterized in that, described time period T _l, time period T _r, time period T _dwith time period T _ube at the same time section.

4. man-machine interaction method as claimed in claim 2, is characterized in that, described time period T _l, time period T _r, time period T _dwith time period T _ube at the same time section.

5. the man-machine interaction method of any one as described in claim 1 to 4, is characterized in that, described frame frequency F was 15 frame/seconds; Described time period T _l, time period T _r, time period T _dwith time period T _uduration be 2 seconds; Described time period T _k, time period T _k1with time period T _k2all be greater than 0.5 second; Described radius r is

described horizontal ordinate variable quantity X and ordinate variable quantity Y are respectively

with

wherein w is display screen width, and h is display screen height.

Images