research-article

A novel approach to single camera, glint-free 3D eye model fitting including corneal refraction

Authors:

Moritz Kassner,

Andreas BullingAuthors Info & Claims

ETRA '18: Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications

Article No.: 9, Pages 1 - 9

https://doi.org/10.1145/3204493.3204525

Published: 14 June 2018 Publication History

Abstract

Model-based methods for glint-free gaze estimation typically infer eye pose using pupil contours extracted from eye images. Existing methods, however, either ignore or require complex hardware setups to deal with refraction effects occurring at the corneal interfaces. In this work we provide a detailed analysis of the effects of refraction in glint-free gaze estimation using a single near-eye camera, based on the method presented by [Świrski and Dodgson 2013]. We demonstrate systematic deviations in inferred eyeball positions and gaze directions with respect to synthetic ground-truth data and show that ignoring corneal refraction can result in angular errors of several degrees. Furthermore, we quantify gaze direction dependent errors in pupil radius estimates. We propose a novel approach to account for corneal refraction in 3D eye model fitting and by analyzing synthetic and real images show that our new method successfully captures refraction effects and helps to overcome the shortcomings of the state of the art approach.

References

[1]

S. Agarwal, K. Mierle, and Others. 2010. Ceres Solver. http://ceres-solver.org. (2010).

[2]

A. D. Barsingerhorn, F. N. Boonstra, and H. H. L. M. Goossens. 2017. Optics of the Human Cornea Influence the Accuracy of Stereo Eye-Tracking Methods: a Simulation Study. Biomedical Optics Express 8, 2 (2017), 712--725.

[3]

I. Bekerman, P. Gottlieb, and M. Vaiman. 2014. Variations in Eyeball Diameters of the Healthy Adults. Journal of Ophthalmology 2014, 3 (2014).

[4]

J. Chen and Q. Ji. 2008. 3D Gaze Estimation With a Single Camera Without IR Illumination. In ICPR 2008. 19th International Conference on Pattern Recognition. IEEE, 1--4.

[5]

J. Chen, Y. Tong, W. Gray, and Q. Ji. 2008. A Robust 3D Eye Gaze Tracking System Using Noise Reduction. In Proceedings of the 2008 Symposium on Eye Tracking Research & Applications (ETRA '08). ACM, New York, NY, USA, 189--196.

Digital Library

[6]

C. Fedtke, F. Mann, and A. Ho. 2010. The Entrance Pupil of the Human Eye: a Three-Dimensional Model as a Function of Viewing Angle. Optics Express 18 (2010), 1--13.

[7]

W. Fuhl, T. Santini, G. Kasneci, W. Rosenstiel, and E. Kasneci. 2017. PupilNet v2.0: Convolutional Neural Networks for CPU Based Real Time Robust Pupil Detection. CoRR (2017). https://arxiv.org/abs/1711.00112

[8]

W. Fuhl, M. Tonsen, A. Bulling, and E. Kasneci. 2016. Pupil detection for head-mounted eye tracking in the wild: An evaluation of the state of the art. Springer Machine Vision and Applications 27, 8 (2016), 1275--1288.

Digital Library

[9]

B. Gagl, S. Hawelka, and F. Hutzler. 2011. Systematic Influence of Gaze Position on Pupil Size Measurement: Analysis and Correction. Behavior Research Methods 43, 4 (2011), 1171--1181.

[10]

H. Gross (Ed.). 2008. Handbook of Optical Systems: Vol. 4 Survey of Optical Instruments. Wiley-VCH Verlag GmbH+Co. KGaA, Weinheim.

[11]

E. D. Guestrin and M. Eizenman. 2006. General Theory of Remote Gaze Estimation Using the Pupil Center and Corneal Reflections. IEEE Transactions on Biomedical Engineering 53, 6 (2006), 1124--1133.

[12]

D. W. Hansen and Q. Ji. 2010. In the Eye of the Beholder: A Survey of Models for Eyes and Gaze. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 3 (2010), 478--500.

Digital Library

[13]

C. Hennessey, B. Noureddin, and P. Lawrence. 2006. A Single Camera Eye-Gaze Tracking System With Free Head Motion. In Proceedings of the 2006 Symposium on Eye Tracking Research & Applications (ETRA '06). ACM, 87--94.

Digital Library

[14]

Y. Itoh and G. Klinker. 2014. Interaction-Free Calibration for Optical See-Through Head-Mounted Displays based on 3D Eye Localization. In 2014 IEEE Symposium on 3D User Interfaces (3DUI). 1--8.

[15]

A.-H. Javadi, Z. Hakimi, M. Barati, V. Walsh, and L. Tcheang. 2015. SET: a Pupil Detection Method Using Sinusoidal Approximation. Frontiers in Neuroengineering 8 (2015).

[16]

M. Kassner, W. Patera, and A. Bulling. 2014. Pupil: an Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-Based Interaction. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication. ACM, 1151--1160.

Digital Library

[17]

C.-C. Lai, S.-W. Shih, and Y.-P. Hung. 2015. Hybrid Method for 3-D Gaze Tracking Using Glint and Contour Features. IEEE Transactions on Circuits and Systems for Video Technology 25, 1 (2015), 24--37.

Digital Library

[18]

Y. Le Grand. 1957. Light, Color and Vision. Wiley, New York, NY, USA.

[19]

M. Mansouryar, J. Steil, Y. Sugano, and A. Bulling. 2016. 3D Gaze Estimation from 2D Pupil Positions on Monocular Head-Mounted Eye Trackers. In Proc. of the 9th ACM International Symposium on Eye Tracking Research & Applications (ETRA). 197--200.

Digital Library

[20]

K. P. Mashige. 2013. A Review of Corneal Diameter, Curvature, and Thickness Values and Influencing Factors. The South African Optometrist (2013), 185--194.

[21]

A. Mayberry, P. Hu, B. Marlin, C. Salthouse, and D. Ganesan. 2014. iShadow: Design of a Wearable, Real-time Mobile Gaze Tracker. In Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services. 82--94.

Digital Library

[22]

T. Ohno, N. Mukawa, and A. Yoshikawa. 2002. FreeGaze: A Gaze Tracking System for Everyday Gaze Interaction. Proceedings of the 2002 Symposium on Eye Tracking Research & Applications (ETRA '02) (2002), 1--8.

Digital Library

[23]

R. Safaee-Rad, I. Tchoukanov, K. C. Smith, and B. Banhabib. 1992. Three-Dimensional Location Estimation of Circular Features for Machine Vision. IEEE Transactions on Robotics and Automation 8, 5 (1992), 624--640.

[24]

T. Santini, W. Fuhl, and E. Kasneci. 2018. PuRe: Robust Pupil Detection for Real-Time Pervasive Eye Tracking. Computer Vision and Image Understanding (2018).

[25]

S. W. Shih and J. Liu. 2004. A Novel Approach to 3-D Gaze Tracking Using Stereo Cameras. IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) 34, 1 (2004), 234--245.

Digital Library

[26]

S. Sirois and J. Brisson. 2014. Pupillometry. Wiley Interdisciplinary Reviews: Cognitive Science 5, 6 (2014), 679--692.

[27]

Y. Sugano and A. Bulling. 2015. Self-calibrating Head-mounted Eye Trackers Using Egocentric Visual Saliency. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST). 363--372.

Digital Library

[28]

L. Świrski, A. Bulling, and N. Dodgson. 2012. Robust Real-Time Pupil Tracking in Highly Off-Axis Images. In Proceedings of the Symposium on Eye Tracking Research & Applications (ETRA '12). 173--176.

Digital Library

[29]

L. Świrski and N. A. Dodgson. 2013. A Fully-Automatic, Temporal Approach to Single Camera, Glint-Free 3D Eye Model Fitting. In Proceedings of ECEM 2013. http://www.cl.cam.ac.uk/research/rainbow/projects/eyemodelfit/

[30]

M. Tonsen, J. Steil, Y. Sugano, and A. Bulling. 2017. InvisibleEye: Mobile Eye Tracking Using Multiple Low-Resolution Cameras and Learning-Based Gaze Estimation. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) (2017).

Digital Library

[31]

M. Tonsen, X. Zhang, Y. Sugano, and A. Bulling. 2016. Labelled pupils in the wild: A dataset for studying pupil detection in unconstrained environments. In Proc. of the 9th ACM International Symposium on Eye Tracking Research & Applications (ETRA). 139--142.

Digital Library

[32]

A. Tsukada and T. Kanade. 2012. Automatic Acquisition of a 3D Eye Model for a Wearable First-Person Vision Device. In Proceedings of the Symposium on Eye Tracking Research & Applications (ETRA '12).

Digital Library

[33]

A. Tsukada, M. Shino, M. Devyver, and T. Kanade. 2011. Illumination-Free Gaze Estimation Method for First-Person Vision Wearable Device. IEEE International Conference on Computer Vision Workshops (Nov. 2011), 2085--2091.

[34]

A. Villanueva and R. Cabeza. 2008. Evaluation of Corneal Refraction in a Model of a Gaze Tracking System. IEEE Transactions on Biomedical Engineering 55, 12 (2008), 2812--2822.

[35]

J.-G. Wang, E. Sung, and R. Venkateswarlu. 2003. Eye Gaze Estimation from a Single Image of One Eye. Proceedings of the Ninth IEEE Internatinal Conference on Computer Vision (ICCV) (2003), 1--8.

Digital Library

Cited By

Niehorster DNyström MHessels RAndersson RBenjamins JHansen DHooge I(2025)The fundamentals of eye tracking part 4: Tools for conducting an eye tracking studyBehavior Research Methods10.3758/s13428-024-02529-757:1Online publication date: 6-Jan-2025
https://doi.org/10.3758/s13428-024-02529-7
Chugh SYe JFu YEizenman M(2024)CSA-CNN: A Contrastive Self-Attention Neural Network for Pupil Segmentation in Eye Gaze TrackingProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3653351(1-7)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3653351
Mikawa YFukiage T(2024)Low-Latency Ocular Parallax Rendering and Investigation of Its Effect on Depth Perception in Virtual RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337207830:5(2228-2238)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3372078
Show More Cited By

Index Terms

A novel approach to single camera, glint-free 3D eye model fitting including corneal refraction
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Tracking
  2. Modeling and simulation
    1. Model development and analysis

Recommendations

A fast approach to refraction-aware eye-model fitting and gaze prediction
ETRA '19: Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications

By temporally integrating information about pupil contours extracted from eye images, model-based methods for glint-free gaze estimation can mitigate pupil detection noise. However, current approaches require time-consuming iterative solving of a ...
Rendering refraction and reflection of eyeglasses for synthetic eye tracker images
ETRA '16: Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications

While for the evaluation of robustness of eye tracking algorithms the use of real-world data is essential, there are many applications where simulated, synthetic eye images are of advantage. They can generate labelled ground-truth data for appearance ...
Automatic acquisition of a 3D eye model for a wearable first-person vision device
ETRA '12: Proceedings of the Symposium on Eye Tracking Research and Applications

A wearable gaze tracking device can work with users in daily-life. For long time of use, a non-active method that does not employ an infrared illumination system is desirable from safety standpoint. It is well known that the eye model constraints ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ETRA '18: Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications

June 2018

595 pages

ISBN:9781450357067

DOI:10.1145/3204493

Conference Chairs:
Bonita Sharif
Youngstown State University
,
Krzysztof Krejtz
University of Social Sciences and Humanities, Warsaw, Poland

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ETRA '18

Sponsor:

ETRA '18: 2018 Symposium on Eye Tracking Research and Applications

June 14 - 17, 2018

Warsaw, Poland

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
689
Total Downloads

Downloads (Last 12 months)80
Downloads (Last 6 weeks)8

Reflects downloads up to 11 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Niehorster DNyström MHessels RAndersson RBenjamins JHansen DHooge I(2025)The fundamentals of eye tracking part 4: Tools for conducting an eye tracking studyBehavior Research Methods10.3758/s13428-024-02529-757:1Online publication date: 6-Jan-2025
https://doi.org/10.3758/s13428-024-02529-7
Chugh SYe JFu YEizenman M(2024)CSA-CNN: A Contrastive Self-Attention Neural Network for Pupil Segmentation in Eye Gaze TrackingProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3653351(1-7)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3653351
Mikawa YFukiage T(2024)Low-Latency Ocular Parallax Rendering and Investigation of Its Effect on Depth Perception in Virtual RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337207830:5(2228-2238)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3372078
Velisar AShanidze N(2023)Noise estimation for head-mounted 3D binocular eye tracking using Pupil Core eye-tracking gogglesBehavior Research Methods10.3758/s13428-023-02150-056:1(53-79)Online publication date: 27-Jun-2023
https://doi.org/10.3758/s13428-023-02150-0
Wagner PHo AKim J(2023)Objective Quantification and Topographic Dioptric Demand of Near-WorkTranslational Vision Science & Technology10.1167/tvst.12.2.2812:2(28)Online publication date: 17-Feb-2023
https://doi.org/10.1167/tvst.12.2.28
Chen SLai YXia SRosin PGao L(2023)3D Face Reconstruction and Gaze Tracking in the HMD for Virtual InteractionIEEE Transactions on Multimedia10.1109/TMM.2022.315682025(3166-3179)Online publication date: 2023
https://doi.org/10.1109/TMM.2022.3156820
Meyer JGering SKasneci E(2023)Static Laser Feedback Interferometry-Based Gaze Estimation for Wearable GlassesIEEE Sensors Journal10.1109/JSEN.2023.325071423:7(7558-7569)Online publication date: 1-Apr-2023
https://doi.org/10.1109/JSEN.2023.3250714
Sun YZhou ZDiels CAsadipour A(2023)DeepMetricEye: Metric Depth Estimation in Periocular VR Imagery2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00058(434-443)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00058
Popovic NChristodoulou DPaudel DWang XVan Gool L(2023)Model-aware 3D Eye Gaze from Weak and Few-shot Supervisions2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct60411.2023.00161(746-751)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR-Adjunct60411.2023.00161
Plopski AIenaga NSugimoto M(2023)Tracking Systems: Calibration, Hardware, and PeripheralsSpringer Handbook of Augmented Reality10.1007/978-3-030-67822-7_9(211-238)Online publication date: 1-Jan-2023
https://doi.org/10.1007/978-3-030-67822-7_9
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten