research-article

Back-Mirror: back-of-device one-handed interaction on smartphones

Authors:

Pui Chung Wong,

Hongbo Fu,

Kening ZhuAuthors Info & Claims

SA '16: SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications

Article No.: 10, Pages 1 - 5

https://doi.org/10.1145/2999508.2999522

Published: 28 November 2016 Publication History

Get Access

Abstract

We present Back-Mirror, a low-cost camera-based approach for widening the interaction space on the back surface of a smartphone by using mirror reflection. Back-Mirror consists of two main parts: a smartphone accessory with a mirror that can reflect the back surface to the rear-facing camera of the phone, and a computer-vision algorithm for gesture recognition based on the visual pattern on the back surface. Our approach captures the finger position on the back surface, and tracks finger movement with higher resolution than the previous methods. We further designed a set of intuitive gestures that can be recognized by Back-Mirror, including swiping up, down, left and right, tapping left, middle, right, and holding gestures. Furthermore, we created applications of Back-of-device, such as game, media player, photo gallery, and unlock mechanism, allowing users to experience the use of Back-Mirror gestures in the real-life scenarios.

References

[1]

Chan, L., Chen, Y., Hsieh, C., Liang, R., and Chen, B. 2015. CyclopsRing: Enabling Whole-Hand and Context-Aware Interactions through a Fisheye Ring. In Proceedings of the 28th annual ACM symposium on User interface software and technology, UIST'15, 549-556.

Digital Library

Google Scholar

[2]

Chan, L., Hsieh, C., Chen, Y., Yang, S., Huang, D., Liang, R., and Chen, B. 2015. Cyclops: Wearable and Single-Piece Full-Body Gesture Input Devices. In Proceedings of the 2nd international conference on Embedded networked sensor systems, CHI'15, 311-311.

Digital Library

Google Scholar

[3]

Doogee Valencia DG800 - smartphone with back touch http://www.androidauthority.com/doogee-valencia-dg800-full-review-390264/

Google Scholar

[4]

Hough, P. Method and Means for Recognizing Complex Patterns, US Patent 3, 069, Ser.No.28,7156 Claims, 1962

Google Scholar

[5]

Karsten, S., and Kate, D. 2014. BackPat: One-Handed OffScreen Patting Gestures. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services, Mobile HCI'14, 77-80.

Digital Library

Google Scholar

[6]

Leagoo Alfa 2 - smartphone with back touch http://www.devicespecifications.com/en/model/6a0d395b

Google Scholar

[7]

Li, W. H. A., and Fu, H., 2013. BezelCursor: bezel-initiated cursor for one-handed target acquisition on mobile touch screens. In SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, ACM, 36

Digital Library

Google Scholar

[8]

LI, W. H. A., Fu, H., and Zhu, K. 2016. BezelCursor: Bezel-initiated cursor for one-handed target acquisition on mobile touch screens. In International Journal of Mobile Human Computer Interaction. Volume 8, Issue 1.

Digital Library

Google Scholar

[9]

Lv, Z., Halawani, A., Sikandar,M.,Rehman, S., and Li, H. 2013. Finger in Air: Touch-less Interaction on Smartphone. In Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia, MUM'13, Article No. 16.

Digital Library

Google Scholar

[10]

Mullern, F., Khalilbeigi, M., andZfuli, N.A. 2015. Study on Proximity-based Hand Input for One-handed Mobile Interaction. In Proceedings of the 3rd ACM Symposium on Spatial User Interaction, 53-56

Digital Library

Google Scholar

[11]

Osmo. Tangible Play. Award Winning Educational Game System for IPad. https://www.playosmo.com/en/

Google Scholar

[12]

Wang, J., Zhai, S., and Canny, J. 2006. Camera phone based motion sensing: interaction techniques, applications and performance study. In Proceedings of the 19th annual ACM symposium on User interface software and technology, UIST'06, 101-110

Digital Library

Google Scholar

[13]

Xiao, X., Han, T., and Wang, J. 2013. LensGesture: Augmenting Mobile Interactions with Back-of-Device Finger Gestures. In Proceedings of 15th ACM on International conference on multimodal interaction, ICMI'13, 287-294.

Digital Library

Google Scholar

Cited By

View all

Ye ZWang YChen Y(2024)CasePad: Privacy-preserving Finger Activity Sensing via Passive Acoustic Signals Enhanced by Mini-Structures in Smartphone Cases2024 33rd International Conference on Computer Communications and Networks (ICCCN)10.1109/ICCCN61486.2024.10637508(1-9)Online publication date: 29-Jul-2024
https://doi.org/10.1109/ICCCN61486.2024.10637508
Liang CWang XLi ZHsia CFan MYu CShi Y(2023)ShadowTouch: Enabling Free-Form Touch-Based Hand-to-Surface Interaction with Wrist-Mounted Illuminant by Shadow ProjectionProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606785(1-14)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606785
Ma JQian JZhou THuang J(2023)FocalPointProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808567:1(1-26)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3580856
Show More Cited By

Index Terms

Back-Mirror: back-of-device one-handed interaction on smartphones
1. Computing methodologies
  1. Computer graphics
    1. Image manipulation
      1. Computational photography

Recommendations

BackXPress: Using Back-of-Device Finger Pressure to Augment Touchscreen Input on Smartphones
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

When people hold their smartphone in landscape orientation, they use their thumbs for input on the frontal touchscreen, while their remaining fingers rest on the back of the device (BoD) to stabilize the grip. We present BackXPress, a new interaction ...
Back-Mirror: back-of-device one-handed interaction on smartphones
SA '16: SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications

Back-Mirror is a low-cost camera-based approach leveraging the usage of smartphone back surface by mirror reflection, and widening the interaction space on the back surface of phone. It consists of two main parts: a smartphone accessory with a small ...
Finger Placement and Hand Grasp during Smartphone Interaction
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems

Smartphones are currently the most successful mobile devices. Through their touchscreens, they combine input and output in a single interface. A body of work investigated interaction beyond direct touch. In particular, previous work proposed using the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SA '16: SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications

November 2016

76 pages

ISBN:9781450345514

DOI:10.1145/2999508

Conference Chairs:
Steven ZhiYing Zhou
National University of Singapore, Singapore
,
Börje Karlsson
Microsoft Research Asia, China

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 November 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SA '16

Sponsor:

SIGGRAPH

SA '16: SIGGRAPH Asia 2016

December 5 - 8, 2016

Macau

Acceptance Rates

Overall Acceptance Rate 178 of 869 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
447
Total Downloads

Downloads (Last 12 months)47
Downloads (Last 6 weeks)4

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Ye ZWang YChen Y(2024)CasePad: Privacy-preserving Finger Activity Sensing via Passive Acoustic Signals Enhanced by Mini-Structures in Smartphone Cases2024 33rd International Conference on Computer Communications and Networks (ICCCN)10.1109/ICCCN61486.2024.10637508(1-9)Online publication date: 29-Jul-2024
https://doi.org/10.1109/ICCCN61486.2024.10637508
Liang CWang XLi ZHsia CFan MYu CShi Y(2023)ShadowTouch: Enabling Free-Form Touch-Based Hand-to-Surface Interaction with Wrist-Mounted Illuminant by Shadow ProjectionProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606785(1-14)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606785
Ma JQian JZhou THuang J(2023)FocalPointProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808567:1(1-26)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3580856
Yeo HWu EKim DLee JKim HOh STakagi LWoo WKoike HQuigley A(2023)OmniSense: Exploring Novel Input Sensing and Interaction Techniques on Mobile Device with an Omni-Directional CameraProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580747(1-18)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580747
Visschedijk AKim HTejada CAshbrook D(2022)ClipWidgets: 3D-printed Modular Tangible UI Extensions for SmartphonesProceedings of the Sixteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3490149.3501314(1-11)Online publication date: 13-Feb-2022
https://dl.acm.org/doi/10.1145/3490149.3501314
Chen TZhu KYang M(2022)Deep-learning-based unobtrusive handedness prediction for one-handed smartphone interactionMultimedia Tools and Applications10.1007/s11042-021-11844-682:4(4941-4964)Online publication date: 18-Jan-2022
https://dl.acm.org/doi/10.1007/s11042-021-11844-6
Wang LZhang XJiang YZhang YXu CGao RZhang D(2021)Watching Your Phone's BackProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34635225:2(1-26)Online publication date: 24-Jun-2021
https://dl.acm.org/doi/10.1145/3463522
Lai JZhang DWang SKilic IZhou L(2019)ThumbStrokeACM Transactions on Management Information Systems10.1145/334385810:3(1-19)Online publication date: 7-Sep-2019
https://dl.acm.org/doi/10.1145/3343858
Yu CWei XVachher SQin YLiang CWeng YGu YShi YBrewster SFitzpatrick GCox AKostakos V(2019)HandSeeProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300935(1-13)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300935
Wang RYu CYang XHe WShi YBrewster SFitzpatrick GCox AKostakos V(2019)EarTouchProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300254(1-13)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300254
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

BackXPress: Using Back-of-Device Finger Pressure to Augment Touchscreen Input on Smartphones

Back-Mirror: back-of-device one-handed interaction on smartphones

Finger Placement and Hand Grasp during Smartphone Interaction