Abstract
3D eye gaze estimation has emerged as an interesting and challenging task in recent years. As an attractive alternative to appearance-based models, 3D model-based gaze estimation methods are powerful because a general prior of eye anatomy or geometry has been integrated into the 3D model hence they adapt well under various head poses and illumination conditions. We present a method for constructing an anatomically accurate 3D deformable eye model from the IR images of eyes and demonstrate its application to 3D gaze estimation. The 3D eye model consists of a deformable basis capable of representing individual real-world eyeballs, corneas, irises and kappa angles. To validate the model’s accuracy, we combine it with a 3D face model (without eyeball) and perform image-based fitting to obtain eye basis coefficients The fitted eyeball is then used to compute 3D gaze direction. Evaluation results on multiple datasets show that the proposed method generalizes well across datasets and is robust under various head poses.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bas, A., Smith, W.A.: What does 2D geometric information really tell us about 3D face shape? Int. J. Comput. Vis. 127(10), 1455–1473 (2019)
Bulat, A., Tzimiropoulos, G.: How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230,000 3D facial landmarks). In: International Conference on Computer Vision (2017)
Cao, C., Weng, Y., Zhou, S., Tong, Y., Zhou, K.: Facewarehouse: a 3D facial expression database for visual computing. IEEE Trans. Vis. Comput. Graph. 20(3), 413–425 (2013)
Chen, J., Tong, Y., Gray, W., Ji, Q.: A robust 3D eye gaze tracking system using noise reduction. In: Proceedings of the 2008 Symposium on Eye Tracking Research & Applications, pp. 189–196 (2008)
Chen, Q., Wu, H., Wada, T.: Camera calibration with two arbitrary coplanar circles. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3023, pp. 521–532. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24672-5_41
Cheng, Y., Huang, S., Wang, F., Qian, C., Lu, F.: A coarse-to-fine adaptive network for appearance-based gaze estimation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 10623–10630 (2020)
Fuhl, W.: From perception to action using observed actions to learn gestures. User Model. User-Adapt. Interact. 31(1), 105–120 (2021)
Fuhl, W., Santini, T., Kasneci, E.: Fast camera focus estimation for gaze-based focus control. arXiv preprint arXiv:1711.03306 (2017)
Fuhl, W., et al.: Non-intrusive practitioner pupil detection for unmodified microscope oculars. Comput. Biol. Med. 79, 36–44 (2016)
Funes Mora, K.A., Monay, F., Odobez, J.M.: Eyediap: a database for the development and evaluation of gaze estimation algorithms from RGB and RGB-d cameras. In: Proceedings of the Symposium on Eye Tracking Research and Applications, pp. 255–258 (2014)
Gerig, T., et al.: Morphable face models-an open framework. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 75–82. IEEE (2018)
Glass2, T.P.: Tobii pro eye tracker data quality report (2017). https://www.tobiipro.com/siteassets/tobii-pro/accuracy-and-precision-tests/tobii-pro-glasses-2-accuracy-and-precision-test-report.pdf
Hennessey, C., Noureddin, B., Lawrence, P.: A single camera eye-gaze tracking system with free head motion. In: Proceedings of the 2006 Symposium on Eye Tracking Research & Applications, pp. 87–94 (2006)
Hutchinson, T.E., White, K.P., Martin, W.N., Reichert, K.C., Frey, L.A.: Human-computer interaction using eye-gaze input. IEEE Trans. Syst. Man Cybernet. 19(6), 1527–1534 (1989)
Kohlbecher, S., Bardinst, S., Bartl, K., Schneider, E., Poitschke, T., Ablassmeier, M.: Calibration-free eye tracking by reconstruction of the pupil ellipse in 3d space. In: Proceedings of the 2008 Symposium on Eye Tracking Research & Applications, pp. 135–138 (2008)
Lai, C.C., Shih, S.W., Hung, Y.P.: Hybrid method for 3-D gaze tracking using glint and contour features. IEEE Trans. Circ. Syst. Video Technol. 25(1), 24–37 (2014)
Li, R., et al.: Learning formation of physically-based face attributes (2020)
Li, T., Bolkart, T., Black, M.J., Li, H., Romero, J.: Learning a model of facial shape and expression from 4D scans. ACM Trans. Graph. 36(6), 1–17 (2017)
Liu, Y., Liu, R., Wang, H., Lu, F.: Generalizing gaze estimation with outlier-guided collaborative adaptation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3835–3844 (2021)
Lugaresi, C., et al.: Mediapipe: a framework for building perception pipelines. arXiv preprint arXiv:1906.08172 (2019)
Palmero, C., Selva, J., Bagheri, M.A., Escalera, S.: Recurrent CNN for 3D gaze estimation using appearance and shape cues. arXiv preprint arXiv:1805.03064 (2018)
Paysan, P., Knothe, R., Amberg, B., Romdhani, S., Vetter, T.: A 3D face model for pose and illumination invariant face recognition. In: 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, pp. 296–301. IEEE (2009)
Ploumpis, S., et al.: Towards a complete 3D morphable model of the human head. IEEE Trans. Pattern Anal. Mach. Intell. 43(11), 4142–4160 (2020)
Smith, B.A., Yin, Q., Feiner, S.K., Nayar, S.K.: Gaze locking: passive eye contact detection for human-object interaction. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, pp. 271–280 (2013)
Song, G., Cai, J., Cham, T.J., Zheng, J., Zhang, J., Fuchs, H.: Real-time 3D face-eye performance capture of a person wearing VR headset. In: Proceedings of the 26th ACM International Conference on Multimedia, pp. 923–931 (2018)
Swirski, L., Dodgson, N.: A fully-automatic, temporal approach to single camera, glint-free 3D eye model fitting. In: Proceeding of the PETMEI, pp. 1–11 (2013)
Tsukada, A., Kanade, T.: Automatic acquisition of a 3D eye model for a wearable first-person vision device. In: Proceedings of the Symposium on Eye Tracking Research and Applications, pp. 213–216 (2012)
Vicente, F., Huang, Z., Xiong, X., De la Torre, F., Zhang, W., Levi, D.: Driver gaze tracking and eyes off the road detection system. IEEE Trans. Intell. Transp. Syst. 16(4), 2014–2027 (2015)
Wang, K., Ji, Q.: Real time eye gaze tracking with kinect. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 2752–2757. IEEE (2016)
Wang, K., Ji, Q.: Real time eye gaze tracking with 3D deformable eye-face model. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1003–1011 (2017)
Wang, Y., et al.: Contrastive regression for domain adaptation on gaze estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19376–19385 (2022)
Wood, E., Baltrušaitis, T., Morency, L.-P., Robinson, P., Bulling, A.: A 3D morphable eye region model for gaze estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 297–313. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_18
Xiong, X., Liu, Z., Cai, Q., Zhang, Z.: Eye gaze tracking using an RGBD camera: a comparison with a RGB solution. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, pp. 1113–1121 (2014)
Zhou, X., Cai, H., Li, Y., Liu, H.: Two-eye model-based gaze estimation from a kinect sensor. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 1646–1653. IEEE (2017)
Zhu, H., et al.: Facescape: 3D facial dataset and benchmark for single-view 3D face reconstruction. arXiv preprint arXiv:2111.01082 (2021)
Acknowledgment
The work described in this paper is supported in part by the U.S. National Science Foundation award CNS 1629856.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Switzerland AG
About this paper
Cite this paper
Kuang, C., Kephart, J.O., Ji, Q. (2023). Towards an Accurate 3D Deformable Eye Model for Gaze Estimation. In: Rousseau, JJ., Kapralos, B. (eds) Pattern Recognition, Computer Vision, and Image Processing. ICPR 2022 International Workshops and Challenges. ICPR 2022. Lecture Notes in Computer Science, vol 13643. Springer, Cham. https://doi.org/10.1007/978-3-031-37660-3_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-37660-3_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-37659-7
Online ISBN: 978-3-031-37660-3
eBook Packages: Computer ScienceComputer Science (R0)