DE102014008852B4 - Calibration of a vehicle eye tracking system - Google Patents

Abstract

The invention relates to a method for adapting an eye tracking system (12) of a motor vehicle (10) to a user (34) by means of an adaptation device (26) for remote calibration, wherein an eye position assumed by the user (34) when viewing a predetermined image sequence (40) is detected and a user-specific calibration profile (20) for the eye tracking system (12) is generated from the detected eye position. The invention is based on the object of adapting the eye tracking system (12) to a specific user (34). For this purpose, the eye position is detected by a detection device (32) of the adaptation device (26) that is different from the eye tracking system (12), and a calibration device (38) of the adaptation device (26) generates a preliminary calibration profile (50) for the detection device (32). A transformation device (52, 52') then converts the preliminary calibration profile (50) into the calibration profile (20) for the eye tracking system (12) using a predetermined mapping rule (54). The adaptation device (26) has the form of glasses, whereby the user (34) is given an operating task to carry out when ordering the motor vehicle in order to direct the attention of the user (34) to certain graphic objects.

Description

The invention relates to a method for adapting an eye tracking system of a motor vehicle to a user. In other words, the eye tracking system is calibrated for the user. In this case, the position of the user's eyes when viewing an image sequence is recorded. A user-specific calibration profile for the eye tracking system is then generated from the recorded eye position. Another term for eye tracking is gaze direction detection.

Calibrating an eye tracking system is, for example, WO 2013/059940 A1 described. The user then uses the eye tracking system while an image sequence is shown, where it is known where the user should look. The calibration profile for the user can then be created based on the determined eye position and the target eye position intended for the image sequence.

Such a self-calibration of an eye tracker is also possible in connection with data glasses from the EN 10 2009 049 073 A1 This combines the calibration of an eye tracking system with the calibration of a display device for a virtual reality effect.

From the US 2012/0235902 A1 Data glasses with self-calibration for gesture recognition are known. The self-calibration can be triggered via a smartphone or other remote control.

The eye tracking systems known from the state of the art have the disadvantage that they have to be used to calibrate them.

In connection with motor vehicles, the use of eye tracking systems is also known in order to be able to activate or operate different functions in the motor vehicle using only one glance based on the direction in which a user is looking. For example, in a motor vehicle infotainment system, a menu entry can be selected from an operating menu based on the direction in which the user is looking.

Since today's eye tracking systems require calibration when used for the first time by a specific user, the user must sit in the vehicle and perform a specific operating task, such as focusing on points on a display device. This can be very cumbersome for a user in connection with a new car purchase, especially if the eye tracking system is also intended to support driving tasks, i.e. provide driver assistance. If the user has not yet calibrated the eye tracking system, this can lead to unwanted distraction during the first few trips.

In the WO 2014/020323 A1 A calibration device for a computer system is described, by means of which cursor control is possible both by means of a computer mouse and by means of a viewing direction control. For calibration, the user must move a symbol with the computer mouse and follow the symbol with his eyes. To record the eye movement, the calibration device comprises a camera that is arranged below a monitor.

The WO 2012/154418 A1 describes a gaze tracking technique using a head-mounted gaze tracking device that communicates with a server. The server receives scene images from the head-mounted gaze tracking device that captures external scenes seen by a user wearing the head-mounted device.

The invention is based on the object of adapting an eye tracking system of a motor vehicle to a specific user.

The problem is solved by the subject matter of the independent patent claims. Advantageous developments of the invention result from the features of the dependent patent claims.

The method according to the invention is based on the calibration process described at the beginning, in which an eye position assumed by the user when viewing a predetermined image sequence is recorded and a user-specific calibration profile for the eye tracking system is generated from the recorded eye position. With this calibration profile, the eye tracking system can be adapted to the user in such a way that an eye position recorded by the eye tracking system is subsequently correctly converted in a known manner into the coordinate of a point on a display device that the user is actually viewing.

In the method according to the invention, the eye position is not recorded with the eye tracking system for the described calibration process, but by a recording device that is different from the eye tracking system. In other words, the user does not have to sit in the vehicle. Using the data that describe the recorded eye position, a calibration device then generates a preliminary calibration profile for this recording device. In other words, a different eye tracking system can be used and calibrated in a manner known per se. This gives the preliminary calibration profile. According to the invention, the preliminary calibration profile is then converted into the calibration profile for the eye tracking system by a transformation device. To do this, the transformation device uses a predetermined mapping rule which describes how calibration data of the preliminary calibration profile are to be converted so that calibration data for the calibration profile of the eye tracking system are obtained.

The adaptation device is in the form of glasses, whereby the user is asked to perform a short operating task when ordering the motor vehicle in order to direct the user's attention to certain graphic objects.

The invention has the advantage that the calibration profile for the eye tracking system of the motor vehicle can be generated before the user uses the motor vehicle for the first time. The calibration profile can simply be saved in the eye tracking system of the motor vehicle. When the user then gets into the motor vehicle for the first time and uses its eye tracking system, the system is already adapted to the user, i.e. calibrated for the specific user.

If the motor vehicle has two eye tracking systems, the provisional calibration profile can of course also be generated with one of the eye tracking systems and then the calibration profile for the other eye tracking system can be generated using the transformation device. However, an adaptation device different from the motor vehicle is preferably provided, which has both a display device for displaying the image sequence, for example a monitor, and the detection device for detecting the eye position as well as the calibration device for generating the provisional calibration profile. The adaptation device is therefore preferably a vehicle-external, self-calibrating eye tracking system.

There are now two different developments of the invention, which also lead to two different devices.

In one embodiment, the transformation device can also be provided by the vehicle-external adaptation device. In other words, the finished calibration profile for the eye tracking system of the motor vehicle is already provided by the vehicle-external calibration device.

In this context, the invention also provides such an external vehicle adaptation device for an eye tracking system of a motor vehicle, which has the described display device for the image sequence, the detection device for detecting the eye position and the calibration device for self-calibration. The adaptation device according to the invention is characterized by its own transformation device, which is designed to convert the preliminary calibration profile into a calibration profile for the eye tracking system using a predetermined mapping rule. Such an adaptation device has the advantage that it can provide a calibration profile for an eye tracking system that can be used immediately in a motor vehicle without further adaptation steps.

The adaptation device is in the form of glasses. In addition, the detection device of the adaptation device has sensors that are designed to capture data for creating the preliminary calibration profile, while the user is given an operating task when ordering the motor vehicle.

According to another embodiment of the invention, however, only the provisional calibration profile is provided by the adaptation device. Here, it is now provided that the provisional calibration profile is converted by the eye tracking system of the motor vehicle itself using the mapping rule. The transformation device is therefore provided by the eye tracking system of the motor vehicle itself.

In this context, an eye tracking system or an eye tracking device for a motor vehicle can be provided, which has: a receiving device for receiving a preliminary calibration profile for a detection device different from the eye tracking system. The eye tracking system of the motor vehicle can therefore receive the preliminary calibration profile. Furthermore, the eye tracking system has a transformation device which is designed to convert this received preliminary calibration profile into a calibration profile for the eye tracking system using a predetermined mapping rule. This results in the advantage that the user can use a generally valid standard calibration profile, namely the preliminary calibration profile, and can transmit this to the eye tracking system, which then converts the standard calibration profile into a calibration profile which can be used by the eye tracking system. The user can therefore calibrate several of the eye tracking systems with a single preliminary calibration profile. If necessary, these convert the preliminary calibration profile into a specific Calibration profile for the respective eye tracking system.

In order to simplify the calibration of the motor vehicle eye tracking system even further, one embodiment provides that the preliminary calibration profile or the calibration profile for the eye tracking system is transmitted to the motor vehicle via a communication connection. The communication connection can be a radio connection, such as can be established using WLAN (Wireless Local Area Network) or mobile radio (GSM, UMTS, LTE). A wired communication connection can also be provided, such as can be established based on the Ethernet standard.

The mapping rule already described for converting the preliminary calibration profile into the vehicle calibration profile is designed to convert profile data of the preliminary calibration profile into profile data that is suitable for calibrating the eye tracking system of the motor vehicle. For this purpose, the mapping rule can comprise a geometric transformation of a geometry of the detection device to a geometry of the eye tracking system. This then advantageously allows a relative position of the detection system, for example a camera, to be adapted or converted or transformed to the display device to be viewed by the user, for example a screen. In other words, for example, a coordinate system X of the vehicle-external adaptation device is adapted to a coordinate system Y of the eye tracking system of the motor vehicle.

In connection with the eye tracking system, which is provided in the motor vehicle and has its own transformation device, a further development provides for sharpening of the profile data. This takes place during operation of the eye tracking system, i.e. after it has been calibrated or adjusted using the described calibration profile and while the user is already operating at least one function of the motor vehicle using the eye tracking system. During operation, position data of a user's eye position is recorded by a recording device of the eye tracking system, for example the eye position is determined by a camera. Based on the position data, a correction device detects a miscalibration. For example, on the basis of a statistical analysis, it can be determined that with the eye tracking system, the gaze direction detection always first determines coordinates on the display device that point next to a certain icon or other graphic object and that the user then always first takes a corrective step towards the graphic object by changing the eye position. If, for example, the first coordinates always point to the left of a graphical object and then within a predetermined time, for example within one or two seconds, a correction to the right of the graphical object occurs, it can be assumed that there is a miscalibration, which the user compensates for by deliberately looking next to it.

The correction device then changes the calibration profile for the eye tracking system depending on the detected miscalibration in order to reduce the miscalibration. For example, an offset is introduced for the detection of the eye position so that in the example described, the first glance at the graphic object is correctly detected by the eye tracking system.

Finally, a motor vehicle can also be provided which has an embodiment of the eye tracking system. The motor vehicle is preferably designed as a motor vehicle, in particular as a passenger car.

An embodiment of the invention is described below. The single figure (Fig.) shows a schematic representation of an embodiment of the adaptation device according to the invention.

The exemplary embodiment explained below is a preferred embodiment of the invention. In the exemplary embodiment, however, the components of the embodiment described each represent individual features of the invention that are to be considered independently of one another, which also develop the invention independently of one another and are therefore to be considered as part of the invention individually or in a combination other than that shown. Furthermore, the described embodiment can also be supplemented by other features of the invention that have already been described.

The figure shows a motor vehicle 10, which can be a motor vehicle, in particular a passenger car. The motor vehicle 10 can have an eye tracking system 12 designed in a manner known per se, by means of which, for example, an infotainment system 14 of the motor vehicle 10 can be operated in a direction of gaze-controlled manner. For example, a menu 16 with menu entries 18 can be displayed on a screen and a user can select one of them by looking at one of the menu entries 18. The eye tracking system 12 then records the user's eye position and recognizes based on the eye position which coordinate on the screen, in particular which menu entry 18, is currently being looked at. It can be provided that based on the eye position a gaze direction vector is generated which describes where the user is currently looking.

In order to ensure that the coordinates are correctly calculated based on, for example, camera image data from a camera of the eye tracking system 12, or generally from a device for detecting the eye position, a calibration profile 20 for the current user is stored in the eye tracking system 12. The calibration profile 20 can have an assignment rule for the detected eye position to coordinates on the screen of the infotainment system 14 or an assignment rule for the detected eye position to a gaze direction vector.

In the case of the motor vehicle 10, it is not necessary to get into the motor vehicle 10 and use the eye tracking system 12 to create the calibration profile 20. The calibration profile 20 can be received in the motor vehicle 10 from outside the motor vehicle 10 via a receiving device 22 of the motor vehicle 10. For this purpose, a radio connection 24 can be established to an adaptation device 26 external to the vehicle by the receiving device 22, for example a mobile radio module or a WLAN module. The radio connection 24 can also be established by the adaptation device 26 via a transmitting device 28 of the adaptation device 26. The communication connection can also be established via the Internet.

To generate the calibration profile 20, the adaptation device 26 can have its own eye tracking system 30 with a detection device 32 for detecting an eye position of a user 34 and with a display device 36 for displaying image information, in particular an image sequence. The eye tracking system 30 can be self-calibrating, i.e. it can have its own calibration device 38. The calibration device 38 can control the display device 36 and thereby display an image sequence 40. The image sequence 40 can, for example, comprise a graphic object 42, for example a cross or a point or even a figure, which can move along a predetermined movement path 44 on the display device 36. The user 34 must follow the graphic object 42 with his eyes. The eye position of the user 34 is detected by the detection device 32 and position data 46 is transmitted to the calibration device 38. From the image sequence, it is known where the graphic object is located, i.e. the coordinates of the graphic object are known. The recorded eye position when looking at the object and the coordinates that the object has are also known.

The calibration device 38 can then generate a preliminary calibration profile 50 from the position data 48 in a manner known per se, by means of which the eye tracking system 30 can be calibrated.

The preliminary calibration profile 50 is used in the adaptation device 26 to generate the calibration profile 20 for the eye tracking system 12 of the motor vehicle 10. For this purpose, the adaptation device 26 has a transformation device 52 which has stored a mapping rule 54 which specifies how calibration data for the calibration profile 20 of the eye tracking system 12 can be calculated from calibration data of the preliminary calibration profile 50 for the eye tracking system 30. Such a mapping rule 54 can be determined by simple tests and/or geometric calculations and/or depending on the geometry of the adaptation device 26 and the geometry of the eye tracking system 12. In particular, the relative position of the detection device 32 to the display device 36 on the one hand and the relative position of the eye tracking system 12 to the display device to be viewed in the motor vehicle 10 on the other hand are used as a basis.

The calibration profile 20 can then be transmitted from the transformation device 52 to the transmitting device 28, which can transmit the calibration profile 20 via the radio connection 24 to the receiver 22 so that the calibration profile 20 can then be stored in the eye tracking system 12.

It can also be provided that the transformation device 52 is not part of the adaptation device 26, but part of the eye tracking system 12, which is illustrated in the figure by a transformation device 52' in the eye tracking system 12. The calibration device 38 can then transmit the provisional calibration profile 50 instead of the calibration profile 20 to the receiver 22 by means of the transmission device 28 via the radio connection 24. The receiver 22 then transmits the received provisional user profile 50 to the transformation device 52'. In this case, the adaptation device 26 can then be, for example, an eye tracking system with self-calibration known per se from the prior art. The transformation is then carried out by the eye tracking system 12 of the motor vehicle.

After the user 34 has used the adjustment device 26, he can enter the Get into the motor vehicle 10 and use the eye tracking system 12 without a prior calibration phase in the motor vehicle 10.

The adaptation device 26 can be set up as a stationary device, for example at a motor vehicle dealer. The adaptation device 26 enables remote calibration. The device is in the form of glasses. When ordering his vehicle, the user 34 is given a short operating task to perform, which can also have entertaining, funny content in order to draw the user's 34 attention particularly intensively to certain graphic objects. For example, the user 34 can have the vehicle he has ordered displayed as a 3D model so that he can move this model with his eyes, open the doors of the model by controlling his eyes and/or operate other functions. Without the user 34 noticing, the sensors of the recording device can record the parameters/data for creating the user profile, in particular the position data 48.

When the motor vehicle 10 is delivered, the user 34 no longer needs calibration, since the eye tracking system 12 has already saved its parameters or at least some of them. It may be possible or possible to sharpen the calibration in the motor vehicle 10. However, this runs in the background without the operating function having to be interrupted and therefore does not bother the user 34. The user 34 can therefore drive off directly with the motor vehicle 10 and use the eye tracking system 12.

Overall, the example shows how the invention can provide remote calibration for an eye tracking system 12.

Claims (6)

Method for adapting an eye tracking system (12) of a motor vehicle (10) to a user (34) by means of an adaptation device (26) for remote calibration, wherein an eye position assumed by the user (34) when viewing a predetermined image sequence (40) is detected and a user-specific calibration profile (20) for the eye tracking system (12) is generated from the detected eye position, wherein the eye position is detected by a detection device (32) of the adaptation device (26) that is different from the eye tracking system (12), a calibration device (38) of the adaptation device (26) generates a preliminary calibration profile (50) for the detection device (32) and a transformation device (52, 52') converts the preliminary calibration profile (50) into the calibration profile (20) for the eye tracking system (12), characterized in that the adaptation device (26) has the shape of glasses, wherein the user (34) is given an operating task to perform when ordering the motor vehicle in order to draw the attention of the user (34) to certain graphic objects. Procedure according to Claim 1 , wherein the transformation device (52) is provided by the adaptation device (26). Procedure according to Claim 1 , wherein the transformation device (52') is provided by the eye tracking system (12). Method according to one of the preceding claims, wherein the preliminary calibration profile (50) for the detection device (32) or the calibration profile (20) for the eye tracking system (12) are transmitted to the motor vehicle (10) via a communication connection (24). Method according to one of the preceding claims, wherein the mapping rule (54) comprises a geometric transformation of a geometry of the detection device (52) to a geometry of the eye tracking system (12). Adaptation device (26) for remote calibration of an eye tracking system (12) of a motor vehicle (10), with a display device (36) for displaying an image sequence (40), a detection device (32) for detecting an eye position of a user (34) when displaying the image sequence (40), a calibration device (38) for generating a preliminary calibration profile (50) for the detection device (32), wherein the adaptation device (26) has a transformation device (50) which is designed to convert the preliminary calibration profile (50) into a calibration profile (20) for the eye tracking system (12) by means of a predetermined mapping rule (54), characterized in that the adaptation device (26) has the shape of glasses, and the detection device (32) has sensors which are designed to capture data for creating the preliminary calibration profile (50) while the user (34) is ordering the motor vehicle (10) is given an operating task.

Images