extended-abstract

"The story of life is quicker than the blink of an eye": using corneal imaging for life logging

Authors:

Christian Lander,

Antonio Krüger,

Markus LöchtefeldAuthors Info & Claims

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct

Pages 1686 - 1695

https://doi.org/10.1145/2968219.2968337

Published: 12 September 2016 Publication History

Abstract

The focus of the human visual attention often reflects his activity and environment -- or his context -- in the most coherent way. Our eyes literally reflect our daily life. The cornea, which is encasing the iris and pupil, is protected by tear fluid, and thus is a highly reflective surface. In this paper we present a first exploration of using corneal imaging to extract information for life logging. Our system is comprised of a head-mounted eye-camera (capturing corneal reflections) connected to a wearable computer, to record humans' daily activities. We conducted a live logging experiment, where we collected data in uninstrumented environments. Based on the analysis of the recordings we demonstrate what kind of information can be extracted out of the corneal image reflections and derive suitable applications use cases.

References

[1]

Michael Backes, Tongbo Chen, Markus Dürmuth, Hendrik P.A. Lensch, and Martin Welk. 2009. Tempest in a teapot: Compromising reflections revisited. In IEEE Symp. on Security and Privacy 2009.

Digital Library

[2]

Tadas BaltrušKaitis, Peter Robinson, and LouisPhilippe Morency. OpenFace: an open source facial behavior analysis toolkit. In IEEE Winter Conference on Applications of Computer Vision, 2016

[3]

Ronald N. Bracewell. The Fourier Transform and its applications. McGraw-Hill Series in Electrical Engineering, Networks and Systems, New York: McGraw-Hill,| c1986, 2nd rev. ed. 1 (1986).

[4]

Andreas Bulling and Hans Gellersen. 2010. Toward Mobile Eye-Based Human-Computer Interaction. IEEE Pervasive Computing 9, 4 (2010), 8--12.

Digital Library

[5]

Andreas Bulling, Jamie A. Ward, Hans Gellersen, and Gerhard Tröster. 2011. Eye Movement Analysis for Activity Recognition Using Electrooculography. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 4 (April 2011), 741--753

Digital Library

[6]

Andreas Bulling and Daniel Roggen. 2011. Recognition of Visual Memory Recall Processes Using Eye Movement Analysis. In Proc. UbiComp 2011. 455--464.

Digital Library

[7]

Daniel Buschek, Michael Spitzer, and Florian Alt. 2015. Video-Recording Your Life: User Perception and Experiences. In Proc CHI EA 2015 (CHI EA '15). ACM, New York, NY, USA, 2223--2228.

Digital Library

[8]

Pierre Chandon, J. Wesley Hutchinson, Eric T. Bradlow, Scott H. Young. 2009. Does In-Store Marketing Work? Effects of the Number and Position of Shelf Facings on Brand Attention and Evaluation at the Point of Purchase. Journal of Marketing 73, 6 (2009), 1--17.

[9]

Andrew T. Duchowski. 2007. Eye Tracking Methodology: Theory and Practice. Springer-Verlag New York, Inc., Secaucus, NJ, USA.

Digital Library

[10]

Kendall Goodrich. 2010. What's up? Exploring upper and lower visual field advertising effects. Journal of Advertising Research (2010).

[11]

Yi-Ching Lee, John D. Lee, and Linda Ng Boyle. 2007. Visual attention in driving: The effects of cognitive load and visual disruption. Human Factors 49, 4 (2007), 721--733.

[12]

Sabrina Hoppe, Tobias Loetscher, Stephanie Morey, and Andreas Bulling. 2015. Recognition of Curiosity Using Eye Movement Analysis. In Proc. UbiComp 2015. 185--188.

Digital Library

[13]

Päivi Majaranta and Kari-Jouko Räihä. 2002. Twenty years of eye typing: systems and design issues. In Proc ETRA 2002. ACM, 15--22.

Digital Library

[14]

J. Müller, D. Wilmsmann, J. Exeler, M. Buzeck, A. Schmidt, T. Jay, and A. Krüger. 2009. Display Blindness: The Effect of Expectations on Attention Towards Digital Signage. In Proc Pervasive 2009 (Pervasive '09).

Digital Library

[15]

Atsushi Nakazawa, Christian Nitschke, and Toyoaki Nishida. 2015. Non-calibrated and real-time human view estimation using a mobile corneal imaging camera. In IEEE ICMEW 2015. IEEE, 1--6.

[16]

David H. Nguyen, Gabriela Marcu, Gillian R. Hayes, Khai N. Truong, James Scott, Marc Langheinrich, Christof Roduner. Encountering SenseCam: Personal Recording Technologies in Everyday Life. Proceedings of UbiComp 2009, Orlando, USA.

Digital Library

[17]

Christian Nitschke, Atsushi Nakazawa, and Haruo Takemura. Corneal imaging revisited: An overview of corneal reflection analysis and applications. IMT 8, 2 (2013), 389--406.

[18]

Christina Nitschke, Atsushi Nakazawa, and Toyoaki Nishida. 2013. I See What You See: Point of Gaze Estimation from Corneal Images. In Proc. ACPR 2013. 298--304.

Digital Library

[19]

Ronald A. Rensink, J. Kevin O'Regan and James J. Clark. 1997. To see or not to see: The need for attention to perceive changes in scenes. Psychological science 8, 5 (1997), 368--373.

[20]

Robert J.K. Jacob. 1990. What you look at is what you get: eye movement-based interaction techniques. In Proc CHI 1990. ACM, 11--18.

Digital Library

[21]

Dirk Schnieders, D., Xingdou Fu and K.-Y. K. Wong. Reconstruction of display and eyes from a single image. In CVPR, IEEE Computer Society (2010), 1442--1449.

[22]

Linda E. Sibert, and Robert J.K. Jacob. Evaluation of eye gaze interaction. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, ACM (2000), 281--288.

Digital Library

[23]

Julian Steil and Andreas Bulling. 2015. Discovery of Everyday Human Activities From Long-term Visual Behaviour Using Topic Models. In Proc. UbiComp 2015. 75--85.

Digital Library

[24]

Sophie Stellmach and Raimund Dachselt. 2012. Look & Touch: Gaze-supported Target Acquisition. In Proc CHI 2012 (CHI '12). ACM, New York, NY, USA, 2981--2990.

Digital Library

[25]

Kentaro Takemura, Tomohisa Yamakawa, Jun Takamatsu and Tsukasa Ogasawara. (2014). Estimation of a focused object using a corneal surface image for eye-based interaction. Journal Of Eye Movement Research, 7(3).

[26]

Bernd Tessendorf, Andreas Bulling, Daniel Roggen, Thomas Stiefmeier, Manuela Feilner, Peter Derleth and Gerhard Tröster. 2011. Recognition of Hearing Needs From Body and Eye Movements to Improve Hearing Instruments. In Proc. Pervasive 2011. 314--331.

Digital Library

[27]

Roel Vertegaal. 2003. Attentive user interfaces. In Communications of the ACM. 30--33.

Cited By

Mohzary MAlmalki KChoi BSong S(2023)Apple in My Eyes (AIME): Liveness Detection for Mobile Security Using Corneal Specular ReflectionsIEEE Internet of Things Journal10.1109/JIOT.2022.321591610:4(3356-3367)Online publication date: 15-Feb-2023
https://doi.org/10.1109/JIOT.2022.3215916
Kaluarachchi TSiriwardhana SWen ENanayakkara S(2023)A Corneal Surface Reflections-Based Intelligent System for Lifelogging ApplicationsInternational Journal of Human–Computer Interaction10.1080/10447318.2022.216324039:9(1963-1980)Online publication date: 4-Jan-2023
https://doi.org/10.1080/10447318.2022.2163240
Zhang XIkematsu KKato KSugiura Y(2022)ReflecTouch: Detecting Grasp Posture of Smartphone Using Corneal Reflection ImagesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517440(1-8)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517440
Show More Cited By

Index Terms

"The story of life is quicker than the blink of an eye": using corneal imaging for life logging
1. Human-centered computing

Recommendations

Using corneal imaging for measuring a human's visual attention
UbiComp '17: Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers

The human's visual attention focus usually reflects the activity and environment engaged in or simply his context in the most coherent way. The cornea, which is encasing the iris and pupil, is protected by tear fluid, and thus is a highly reflective ...
In the blink of an eye: combining head motion and eye blink frequency for activity recognition with Google Glass
AH '14: Proceedings of the 5th Augmented Human International Conference

We demonstrate how information about eye blink frequency and head motion patterns derived from Google Glass sensors can be used to distinguish different types of high level activities. While it is well known that eye blink frequency is correlated with ...
Single camera remote eye gaze tracking under natural head movements
ICAT'06: Proceedings of the 16th international conference on Advances in Artificial Reality and Tele-Existence

Most available gaze tracking systems based on corneal reflection have low tolerance for head movements. In this paper, an eye-gaze tracking method, which can make this system be adaptive to external lighting changes and to the user’s head movements, is ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct

September 2016

1807 pages

ISBN:9781450344623

DOI:10.1145/2968219

General Chairs:
Paul Lukowicz
DFKI and University of Kaiserslautern
,
Antonio Krüger
DFKI and Saarland University
,
Program Chairs:
Andreas Bulling
Max Planck Institute for Informatics
,
Youn-Kyung Lim
KAIST
,
Shwetak N. Patel
University of Washington

Copyright © 2016 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 September 2016

Check for updates

Author Tags

Qualifiers

Extended-abstract

Conference

UbiComp '16

Sponsor:

UbiComp '16: The 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 12 - 16, 2016

Germany, Heidelberg

Acceptance Rates

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
213
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mohzary MAlmalki KChoi BSong S(2023)Apple in My Eyes (AIME): Liveness Detection for Mobile Security Using Corneal Specular ReflectionsIEEE Internet of Things Journal10.1109/JIOT.2022.321591610:4(3356-3367)Online publication date: 15-Feb-2023
https://doi.org/10.1109/JIOT.2022.3215916
Kaluarachchi TSiriwardhana SWen ENanayakkara S(2023)A Corneal Surface Reflections-Based Intelligent System for Lifelogging ApplicationsInternational Journal of Human–Computer Interaction10.1080/10447318.2022.216324039:9(1963-1980)Online publication date: 4-Jan-2023
https://doi.org/10.1080/10447318.2022.2163240
Zhang XIkematsu KKato KSugiura Y(2022)ReflecTouch: Detecting Grasp Posture of Smartphone Using Corneal Reflection ImagesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517440(1-8)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517440
Kaluarachchi TSapkota STaradel JThevenon AMatthies DNanayakkara S(2021)EyeKnowYou: A DIY Toolkit to Support Monitoring Cognitive Load and Actual Screen Time using a Head-Mounted WebcamAdjunct Publication of the 23rd International Conference on Mobile Human-Computer Interaction10.1145/3447527.3474850(1-8)Online publication date: 27-Sep-2021
https://dl.acm.org/doi/10.1145/3447527.3474850
Wu HFeng JTian XSun ELiu YDong BXu FZhong Sde Lara EMohomed INieh JBelding E(2020)EMOProceedings of the 18th International Conference on Mobile Systems, Applications, and Services10.1145/3386901.3388917(448-461)Online publication date: 15-Jun-2020
https://dl.acm.org/doi/10.1145/3386901.3388917
Santini TNiehorster DKasneci EKrejtz KSharif B(2019)Get a gripProceedings of the 11th ACM Symposium on Eye Tracking Research & Applications10.1145/3314111.3319835(1-10)Online publication date: 25-Jun-2019
https://dl.acm.org/doi/10.1145/3314111.3319835
Utsu TTakemura KKrejtz KSharif B(2019)Remote corneal imaging by integrating a 3D face model and an eyeball modelProceedings of the 11th ACM Symposium on Eye Tracking Research & Applications10.1145/3314111.3319817(1-5)Online publication date: 25-Jun-2019
https://dl.acm.org/doi/10.1145/3314111.3319817
Lander CKrüger A(2018)EyeSenseProceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers10.1145/3267305.3274121(980-987)Online publication date: 8-Oct-2018
https://dl.acm.org/doi/10.1145/3267305.3274121
Lander CSpeicher MKerber FKrüger AMandryk RHancock MPerry MCox A(2018)Towards Fixation Extraction in Corneal Imaging Based Eye Tracking DataExtended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3170427.3188597(1-6)Online publication date: 20-Apr-2018
https://dl.acm.org/doi/10.1145/3170427.3188597
Lander Clöchtefeld MKrüger A(2018)hEYEbridProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/31611661:4(1-29)Online publication date: 8-Jan-2018
https://dl.acm.org/doi/10.1145/3161166
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents