research-article

Exploiting thermal reflection for interactive systems

Authors:

Alireza Sahami Shirazi,

Yomna Abdelrahman,

Stefan Schneegass,

Mohammadreza Khalilbeigi,

Albrecht SchmidtAuthors Info & Claims

CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 3483 - 3492

https://doi.org/10.1145/2556288.2557208

Published: 26 April 2014 Publication History

Abstract

Thermal cameras have recently drawn the attention of HCI researchers as a new sensory system enabling novel interactive systems. They are robust to illumination changes and make it easy to separate human bodies from the image background. Far-infrared radiation, however, has another characteristic that distinguishes thermal cameras from their RGB or depth counterparts, namely thermal reflection. Common surfaces reflect thermal radiation differently than visual light and can be perfect thermal mirrors. In this paper, we show that through thermal reflection, thermal cameras can sense the space beyond their direct field-of-view. A thermal camera can sense areas besides and even behind its field-of-view through thermal reflection. We investigate how thermal reflection can increase the interaction space of projected surfaces using camera-projection systems. We moreover discuss the reflection characteristics of common surfaces in our vicinity in both the visual and thermal radiation bands. Using a proof-of-concept prototype, we demonstrate the increased interaction space for hand-held camera-projection system. Furthermore, we depict a number of promising application examples that can benefit from the thermal reflection characteristics of surfaces.

References

[1]

Beckmann, P., and Spizzichino, A. The scattering of electromagnetic waves from rough surfaces. Artech House Radar Library, 1987.

[2]

Bennett, H., and Porteus, J. Relation between surface roughness and specular reflectance at normal incidence. Journal of the Optical Society of America 51 (1961).

[3]

Blasko, G., Feiner, S., and Coriand, F. Exploring interaction with a simulated wrist-worn projection display. In Proceedings of the International Symposium on Wearable Computers (2005), 2--9.

Digital Library

[4]

Cao, X., and Balakrishnan, R. Interacting with dynamically defined information spaces using a handheld projector and a pen. In Proceedings of the Symposium on User interface software and technology (2006), 225--234.

Digital Library

[5]

Cao, X., Forlines, C., and Balakrishnan, R. Multi-user interaction using handheld projectors. In Proceedings of the Symposium on User interface software and technology (2007), 43--52.

Digital Library

[6]

Ciaramello, F. M., and Hemami, S. S. Real-time face and hand detection for videoconferencing on a mobile device. In Proceedings of the Workshop on Video Processing and Quality Metrics for Consumer Electronics (2009).

[7]

Cohen, C. J., Beach, G., and Foulk, G. A basic hand gesture control system for pc applications. In Proceedings of the Workshop on Applied Imagery Pattern Recognition (2001), 74--79.

Digital Library

[8]

Cowan, L. G., and Li, K. A. Shadowpuppets: supporting collocated interaction with mobile projector phones using hand shadows. In Proceedings of the Conference on Human Factors in Computing Systems (2011).

Digital Library

[9]

Duda, R. O., and Hart, P. E. Use of the hough transformation to detect lines and curves in pictures. Communications of the ACM 15, 1 (1972), 11--15.

Digital Library

[10]

Greaves, A., and Rukzio, E. Evaluation of picture browsing using a projector phone. In Proceedings of the Conference on Human computer interaction with mobile devices and services (2008), 351--354.

Digital Library

[11]

Harrison, C., Benko, H., and Wilson, A. D. Omnitouch: wearable multitouch interaction everywhere. In Proceedings of the Symposium on User interface software and technology (2011), 441--450.

Digital Library

[12]

Henze, N., Löcken, A., Boll, S., Hesselmann, T., and Pielot, M. Free-hand gestures for music playback: deriving gestures with a user-centred process. In Proceedings of the Conference on Mobile and Ubiquitous Multimedia (2010), 16:1--16:10.

Digital Library

[13]

Hilliges, O., Izadi, S., Wilson, A. D., Hodges, S., Garcia-Mendoza, A., and Butz, A. Interactions in the air: adding further depth to interactive tabletops. In Proceedings of the Symposium on User interface software and technology (2009), 139--148.

Digital Library

[14]

Iwai, D., and Sato, K. Heat sensation in image creation with thermal vision. In Proceedings of the Conference on Advances in computer entertainment technology (2005), 213--216.

Digital Library

[15]

Iwai, D., and Sato, K. Limpid desk: see-through access to disorderly desktop in projection-based mixed reality. In Proceedings of the Symposium on Virtual reality software and technology (2006), 112--115.

Digital Library

[16]

Iwai, Y., Watanabe, K., Yagi, Y., and Yachida, M. Gesture recognition by using colored gloves. In Proceedings of the Conference on Systems, Man, and Cybernetics (1996), 76--81.

[17]

Izadi, S., Hodges, S., Taylor, S., Rosenfeld, D., Villar, N., Butler, A., and Westhues, J. Going beyond the display: a surface technology with an electronically switchable diffuser. In Proceedings of the Symposium on User interface software and technology (2008), 269--278.

Digital Library

[18]

Kane, S. K., Avrahami, D., Wobbrock, J. O., Harrison, B., Rea, A. D., Philipose, M., and LaMarca, A. Bonfire: a nomadic system for hybrid laptop-tabletop interaction. In Proceedings of the 22nd annual ACM symposium on User interface software and technology (2009), 129--138.

Digital Library

[19]

Khan, M. M., Ingleby, M., and Ward, R. D. Automated facial expression classification and affect interpretation using infrared measurement of facial skin temperature variations. ACM Transactions on Autonomous Adaptive Systems 1, 1 (2006), 91--113.

Digital Library

[20]

Kim, D., Hilliges, O., Izadi, S., Butler, A. D., Chen, J., Oikonomidis, I., and Olivier, P. Digits: freehand 3d interactions anywhere using a wrist-worn gloveless sensor. In Proceedings of the Symposium on User interface software and technology (2012), 167--176.

Digital Library

[21]

Koike, H., Sato, Y., and Kobayashi, Y. Integrating paper and digital information on enhanceddesk: a method for realtime finger tracking on an augmented desk system. ACM Transactions on Computer-Human Interaction 8, 4 (2001), 307--322.

Digital Library

[22]

Kong, S. G., Heo, J., Boughorbel, F., Zheng, Y., Abidi, B. R., Koschan, A., Yi, M., and Abidi, M. A. Multiscale fusion of visible and thermal ir images for illumination-invariant face recognition. International Journal of Computer Vision 71, 2 (2007), 215--233.

Digital Library

[23]

Larson, E., Cohn, G., Gupta, S., Ren, X., Harrison, B., Fox, D., and Patel, S. Heatwave: thermal imaging for surface user interaction. In Proceedings of the Conference on Human Factors in Computing Systems (2011), 2565--2574.

Digital Library

[24]

Lyda, W., Zimmermann, J., Burla, A., Regin, J., Osten, W., Sawodny, O., and Westkämper, E. Sensor and actuator conditioning for multiscale measurement systems on example of confocal microscopy. In SPIE Europe Optical Metrology, vol. 7389 (2009).

[25]

Manresa, C., Varona, J., Mas, R., and Perales, F. Hand tracking and gesture recognition for human-computer interaction. Electronic letters on computer vision and image analysis 5, 3 (2005), 96--104.

[26]

Mistry, P., Maes, P., and Chang, L. Wuw - wear ur world: a wearable gestural interface. In Proceedings of the Conference on Human Factors in Computing Systems (extended abstracts) (2009), 4111--4116.

Digital Library

[27]

Murugappan, S., Vinayak, Elmqvist, N., and Ramani, K. Extended multitouch: recovering touch posture and differentiating users using a depth camera. In Proceedings of the Symposium on User interface software and technology (2012), 487--496.

Digital Library

[28]

Otsu, N. A threshold selection method from gray-level histograms. Automatica 11, 285--296 (1975), 23-27.

Digital Library

[29]

Pham, Q.-C., Gond, L., Begard, J., Allezard, N., and Sayd, P. Real-time posture analysis in a crowd using thermal imaging. In Proceedings of the Conference on Computer Vision and Pattern Recognition (2007), 1--8.

[30]

Puri, C., Olson, L., Pavlidis, I., Levine, J., and Starren, J. Stresscam: non-contact measurement of users' emotional states through thermal imaging. In Proceedings of the Conference on Human Factors in Computing Systems (extended abstracts) (2005), 1725--1728.

Digital Library

[31]

Raheja, J. L., Das, K., and Chaudhary, A. Fingertip detection: A fast method with natural hand. arXiv preprint arXiv:1212.0134 (2012).

[32]

Rukzio, E., Holleis, P., and Gellersen, H. Personal projectors for pervasive computing. IEEE Pervasive Computing 11, 2 (2012), 30--37.

Digital Library

[33]

Schmidt, D., Molyneaux, D., and Cao, X. Picontrol: using a handheld projector for direct control of physical devices through visible light. In Proceedings of the Symposium on User interface software and technology (2012), 379--388.

Digital Library

[34]

Stenger, B., Thayananthan, A., Torr, P. H., and Cipolla, R. Model-based hand tracking using a hierarchical bayesian filter. Transactions on Pattern Analysis and Machine Intelligence 28, 9 (2006), 1372--1384.

Digital Library

[35]

Tiziani, H., Haist, T., and Reuter, S. Optical inspection and characterization of microoptics using confocal microscopy. Optics and lasers in engineering 36, 5 (2001), 403--415.

[36]

Velten, A., Willwacher, T., Gupta, O., Veeraraghavan, A., Bawendi, M. G., and Raskar, R. Recovering three-dimensional shape around a corner using ultrafast time-of--ight imaging. Nature Communications 3 (2012).

[37]

Vollmer, M., Henke, S., Karstädt, D., Möllmann, K., and Pinno, F. Identification and suppression of thermal reflections in infrared thermal imaging. In Proceedings of Inframation, vol. 5 (2004), 287--298.

[38]

Wang, R. Y., and Popović, J. Real-time hand-tracking with a color glove. In ACM Transactions on Graphics, vol. 28 (2009), 63:1--63:8.

Digital Library

[39]

Willis, K. D., Poupyrev, I., and Shiratori, T. Motionbeam: a metaphor for character interaction with handheld projectors. In Proceedings of the Conference on Human Factors in Computing Systems (2011), 1031--1040.

Digital Library

[40]

Wilson, A. D. Touchlight: an imaging touch screen and display for gesture-based interaction. In Proceedings of the Conference on Multimodal interfaces (2004), 69--76.

Digital Library

[41]

Wilson, A. D. Playanywhere: a compact interactive tabletop projection-vision system. In Proceedings of the Symposium on User interface software and technology (2005), 83--92.

Digital Library

[42]

Wilson, A. D. Depth sensing video cameras for 3d tangible tabletop interaction. In Proceedings of Workshop on Horizontal Interactive Human-Computer Systems (2007).

[43]

Wilson, A. D. Using a depth camera as a touch sensor. In Proceedings of the Conference on Interactive Tabletops and Surfaces (2010), 69--72.

Digital Library

[44]

Wilson, A. D., and Benko, H. Combining multiple depth cameras and projectors for interactions on, above and between surfaces. In Proceedings of the Symposium on User interface software and technology (2010), 273--282.

Digital Library

[45]

Winkler, C., Pfeuffer, K., and Rukzio, E. Investigating mid-air pointing interaction for projector phones. In Proceedings of the Conference on Interactive tabletops and surfaces (2012), 85--94.

Digital Library

[46]

Winkler, C., Reinartz, C., Nowacka, D., and Rukzio, E. Interactive phone call: synchronous remote collaboration and projected interactive surfaces. In Proceedings of the Conference on Interactive Tabletops and Surfaces (2011), 61--70.

Digital Library

[47]

Wong, W. K., Lim, H. L., Loo, C. K., and Lim, W. S. Home alone faint detection surveillance system using thermal camera. In Proceedings of the Conference on Computer Research and Development (2010), 747--751.

Digital Library

[48]

Wong, W. K., Tan, P. N., Loo, C. K., and Lim, W. S. An effective surveillance system using thermal camera. In Proceedings of the Conference on Signal Acquisition and Processing (2009), 13--17.

Digital Library

Cited By

Jiang WWang CSarsenbayeva ZIrlitti AWei JKnibbe JDingler TGoncalves JKostakos V(2023)InfoPrintProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36109337:3(1-29)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610933
Alotaibi NWilliamson JKhamis M(2023)ThermoSecure: Investigating the Effectiveness of AI-Driven Thermal Attacks on Commonly Used Computer KeyboardsACM Transactions on Privacy and Security10.1145/356369326:2(1-24)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3563693
Saad AIzadi KAhmad Khan AKnierim PSchneegass SAlt FAbdelrahman Y(2023)HotFoot: Foot-Based User Identification Using Thermal ImagingProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580924(1-13)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580924
Show More Cited By

Index Terms

Exploiting thermal reflection for interactive systems
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Investigation of Material Properties for Thermal Imaging-Based Interaction
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Recent work demonstrated the exciting opportunities that thermal imaging offers for the development of interactive systems. It was shown that a thermal camera can sense when a user touches a surface, performs gestures in the camera's direct field of ...
Modelling the vertical heat exchanger in thermal basin
ICCSA'11: Proceedings of the 2011 international conference on Computational science and its applications - Volume Part IV

In geographical area characterize by specific geological conformations such as the Viterbo area which comprehend active volcanic basins, it is difficult to use conventional geothermal plants.

In fact the area presents at shallow depths thermal falde ...
Thermal cameras and applications: a survey

Thermal cameras are passive sensors that capture the infrared radiation emitted by all objects with a temperature above absolute zero. This type of camera was originally developed as a surveillance and night vision tool for the military, but recently ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2014

4206 pages

ISBN:9781450324731

DOI:10.1145/2556288

General Chairs:
Matt Jones
Swansea University, Wales, UK
,
Philippe Palanque
Université Paul Sabatier, France
,
Program Chairs:
Albrecht Schmidt
University of Stuttgart, Germany
,
Tovi Grossman
Autodesk Research, Canada

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 April 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '14

Sponsor:

SIGCHI

CHI '14: CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2014

Ontario, Toronto, Canada

Acceptance Rates

CHI '14 Paper Acceptance Rate 465 of 2,043 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
660
Total Downloads

Downloads (Last 12 months)47
Downloads (Last 6 weeks)3

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jiang WWang CSarsenbayeva ZIrlitti AWei JKnibbe JDingler TGoncalves JKostakos V(2023)InfoPrintProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36109337:3(1-29)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610933
Alotaibi NWilliamson JKhamis M(2023)ThermoSecure: Investigating the Effectiveness of AI-Driven Thermal Attacks on Commonly Used Computer KeyboardsACM Transactions on Privacy and Security10.1145/356369326:2(1-24)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3563693
Saad AIzadi KAhmad Khan AKnierim PSchneegass SAlt FAbdelrahman Y(2023)HotFoot: Foot-Based User Identification Using Thermal ImagingProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580924(1-13)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580924
Faltaous SWittpoth MAbdelrahman YSchneegass S(2022)HeatGoggles: Enabling Ubiquitous Touch Input through Head-Mounted Devices using Thermal ImagingProceedings of the 21st International Conference on Mobile and Ubiquitous Multimedia10.1145/3568444.3570597(254-256)Online publication date: 27-Nov-2022
https://dl.acm.org/doi/10.1145/3568444.3570597
Sahoo LMiazi NShehab MAlt FAbdelrahman Y(2022)You Know Too Much: Investigating Users’ Perceptions and Privacy Concerns Towards Thermal ImagingPrivacy Symposium 202210.1007/978-3-031-09901-4_11(207-229)Online publication date: 21-Jun-2022
https://doi.org/10.1007/978-3-031-09901-4_11
Alharbi RFeng CSen SJain JHester JAlshurafa N(2021)HeatSight: Wearable Low-power Omni Thermal SensingProceedings of the 2021 ACM International Symposium on Wearable Computers10.1145/3460421.3478811(108-112)Online publication date: 21-Sep-2021
https://dl.acm.org/doi/10.1145/3460421.3478811
Teyssier MKoelle MStrohmeier PFruchard BSteimle JKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Eyecam: Revealing Relations between Humans and Sensing Devices through an Anthropomorphic WebcamProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445491(1-13)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445491
Faltaous SLiebers JAbdelrahman YAlt FSchneegass S(2020)VPID: Towards Vein Pattern Identification Using Thermal Imagingi-com10.1515/icom-2019-000918:3(259-270)Online publication date: 14-Jan-2020
https://doi.org/10.1515/icom-2019-0009
Sorescu CMeena YSahoo DIqbal SMacLean KChevalier FMueller S(2020)PViMatAdjunct Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology10.1145/3379350.3416192(80-83)Online publication date: 20-Oct-2020
https://dl.acm.org/doi/10.1145/3379350.3416192
Zhang TZeng XZhang YSun KWang YChen YBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)ThermalRing: Gesture and Tag Inputs Enabled by a Thermal Imaging Smart RingProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376323(1-13)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376323
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents