research-article

Reading between lines: high-rate, non-intrusive visual codes within regular videos via ImplicitCode

Authors:

Marco GruteserAuthors Info & Claims

UbiComp '15: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing

Pages 157 - 168

https://doi.org/10.1145/2750858.2805824

Published: 07 September 2015 Publication History

Abstract

Given the penetration of mobile devices equipped with cameras, there has been increasing interest in enabling user interaction via visual codes. Simple examples like QR Codes abound. Since many codes like QR Codes are visually intrusive, various mechanisms have been explored to design visual codes that can be hidden inside regular images or videos, though the capacity of these codes remains low to ensure invisibility. We argue, however, that high capacity while maintaining invisibility would enable a vast range of applications that embed rich contextual information in video screens.

To this end, we propose ImplicitCode, a high-rate visual codes that can be hidden inside regular videos. Our scheme combines existing techniques to achieve invisibility. However, we show that these techniques, when employed individually, are too constraining to deliver a high capacity. Experiment results show that ImplicitCode can deliver a significant capacity boost over two recent schemes, up to 12x that of HiLight [19] and 6x or 7x that of InFrame [32], while maintaining a similar or better level of invisibility.

References

[1]

ISO/IEC 18004:2006. Information technology. Automatic identification and data capture techniques. Bar code symbology. QR Code.

[2]

ISO/IEC 16022:2006 Information technology-Automatic identification and data capture techniques-Data Matrix bar code symbology specification.

[3]

http://www.digimarc.com/products/discover/id-manager.

[4]

Private communication with Grace Woo.

[5]

http://research.microsoft.com/en-us/projects/hccb/.

[6]

http://www.mathworks.com/matlabcentral/fileexchange/30790-image-pyramidgaussian-and-laplacian/content/.

[7]

http://www.elementaltechnologies.com/resources/4k-test-sequences.

[8]

M. Alghoniemy and A. Tewfik. Geometric invariance in image watermarking. Image Processing, IEEE Transactions on, 13(2): 145--153, Feb 2004.

Digital Library

[9]

C. M. Bishop. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2006.

Digital Library

[10]

A. Cheddad, J. Condell, K. Curran, and P. Mc Kevitt. Review: Digital Image Steganography: Survey and Analysis of Current Methods. Signal Process., 90(3): 727--752, Mar. 2010.

Digital Library

[11]

I. J. Cox, J. Kilian, F. T. Leighton, and T. Shamoon. Secure Spread Spectrum Watermarking for Multimedia. Trans. Img. Proc., 6(12): 1673--1687, Dec. 1997.

Digital Library

[12]

T. Hao, R. Zhou, and G. Xing. COBRA: Color Barcode Streaming for Smartphone Systems. In Proceedings of the 10th International Conference on Mobile Systems, Applications, and Services, MobiSys '12, pages 85--98. ACM, 2012.

Digital Library

[13]

F. Hartung and M. Kutter. Multimedia watermarking techniques. Proceedings of the IEEE, 87(7): 1079--1107, Jul 1999.

[14]

X. Hou and L. Zhang. Saliency detection: A spectral residual approach. In Computer Vision and Pattern Recognition, 2007. CVPR '07. IEEE Conference on, pages 1--8, June 2007.

[15]

W. Hu, H. Gu, and Q. Pu. LightSync: Unsynchronized Visual Communication over Screen-camera Links. In Proceedings of the 19th Annual International Conference on Mobile Computing & Networking, MobiCom '13, pages 15--26. ACM, 2013.

Digital Library

[16]

W. Hu, J. Mao, Z. Huang, Y. Xue, J. She, K. Bian, and G. Shen. Strata: Layered coding for scalable visual communication. In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking, MobiCom '14, pages 79--90. ACM, 2014.

Digital Library

[17]

L. Itti, C. Koch, and E. Niebur. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11): 1254--1259, 1998.

Digital Library

[18]

S. Kishk and B. Javidi. Information hiding technique with double phase encoding. Appl. Opt., 41(26): 5462--5470, Sep 2002.

[19]

T. Li, C. An, A. Campbell, and X. Zhou. Hilight: Hiding bits in pixel translucency changes. In Proceedings of the 1st ACM MobiCom Workshop on Visible Light Communication Systems, VLCS '14, pages 45--50. ACM, 2014.

Digital Library

[20]

T. Li, C. An, X. Xiao, A. T. Campbell, and X. Zhou. Real-time screen-camera communication behind any scene. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys '15, pages 197--211, New York, NY, USA, 2015. ACM.

Digital Library

[21]

Q. Liu, A. H. Sung, and M. Qiao. Video steganalysis based on the expanded markov and joint distribution on the transform domains detecting msu stegovideo. In Proceedings of the 2008 Seventh International Conference on Machine Learning and Applications, ICMLA '08, pages 671--674. IEEE Computer Society, 2008.

Digital Library

[22]

O. L. Meur, P. L. Callet, and D. Barba. Predicting visual fixations on video based on low-level visual features. Vision Research, 47(19): 2483--2498, 2007.

[23]

A. Mohan, G. Woo, S. Hiura, Q. Smithwick, and R. Raskar. Bokode: Imperceptible Visual Tags for Camera Based Interaction from a Distance. In ACM SIGGRAPH 2009 Papers, SIGGRAPH '09, pages 98:1--98:8. ACM, 2009.

Digital Library

[24]

S. D. Perli, N. Ahmed, and D. Katabi. PixNet: Interference-free Wireless Links Using LCD-camera Pairs. In Proceedings of the Sixteenth Annual International Conference on Mobile Computing and Networking, MobiCom '10, pages 137--148. ACM, 2010.

Digital Library

[25]

A. Pramila, A. Keskinarkaus, and M. Oulu. Camera based watermark extraction - problems and examples. In In: Proceedings of the Finnish Signal Processing Symposium 2007, 2007.

[26]

A. Pramila, A. Keskinarkaus, and T. Seppänen. Watermark robustness in the print-cam process. In Proceedings of the Fifth IASTED International Conference on Signal Processing, Pattern Recognition and Applications, SPPRA '08, pages 60--65. ACTA Press, 2008.

Digital Library

[27]

A. Pramila, A. Keskinarkaus, and T. Seppänen. Reading watermarks from printed binary images with a camera phone. In Proceedings of the 8th International Workshop on Digital Watermarking, IWDW '09, pages 227--240. Springer-Verlag, 2009.

Digital Library

[28]

M. Rohs. Real-world interaction with camera phones. In Proceedings of the Second International Conference on Ubiquitous Computing Systems, UCS'04, pages 74--89, Berlin, Heidelberg, 2004. Springer-Verlag.

Digital Library

[29]

P. Vartiainen, S. Chande, and K. Rämö. Mobile Visual Interaction: Enhancing Local Communication and Collaboration with Visual Interactions. In Proceedings of the 5th International Conference on Mobile and Ubiquitous Multimedia, MUM '06, New York, NY, USA, 2006. ACM.

Digital Library

[30]

A. Wang, Z. Li, C. Peng, G. Shen, G. Fang, and B. Zeng. Inframe++: Achieve simultaneous screen-human viewing and hidden screen-camera communication. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys '15, pages 181--195, New York, NY, USA, 2015. ACM.

Digital Library

[31]

A. Wang, S. Ma, C. Hu, J. Huai, C. Peng, and G. Shen. Enhancing reliability to boost the throughput over screen-camera links. In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking, MobiCom '14, pages 41--52. ACM, 2014.

Digital Library

[32]

A. Wang, C. Peng, O. Zhang, G. Shen, and B. Zeng. Inframe: Multiflexing full-frame visible communication channel for humans and devices. In Proceedings of the 13th ACM Workshop on Hot Topics in Networks, HotNets-XIII, pages 23:1--23:7. ACM, 2014.

Digital Library

[33]

M. Weiser. Creating the invisible interface: (invited talk). In Proceedings of the 7th Annual ACM Symposium on User Interface Software and Technology, UIST '94, pages 1--, New York, NY, USA, 1994. ACM.

Digital Library

[34]

G. Woo, A. Lippman, and R. Raskar. Vrcodes: Unobtrusive and active visual codes for interaction by exploiting rolling shutter. In Proceedings of the 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), ISMAR '12, pages 59--64. IEEE Computer Society, 2012.

Digital Library

[35]

S. Wu and J. Xiao. Aid: Augmented information display. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp '14, pages 523--527. ACM, 2014.

Digital Library

[36]

X. Wu and G. Zhai. Temporal Psychovisual Modulation: A New Paradigm of Information Display {Exploratory DSP}. Signal Processing Magazine, IEEE, 30(1): 136--141, Jan 2013.

Cited By

Xu JBrauers CKlein JJochims JKays R(2023)Symbol Position Recovery for Optical Camera Communication With High-Density Matrix CodesIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2022.323164833:7(3071-3086)Online publication date: Jul-2023
https://doi.org/10.1109/TCSVT.2022.3231648
Tapia MXu TWu ZZamalloa M(2022)SunBoxProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35346026:2(1-26)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3534602
Klein JXu JBrauers CJochims JKays R(2022)Investigations on Temporal Sampling and Patternless Frame Recovery for Asynchronous Display-Camera CommunicationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2021.310671132:6(4004-4015)Online publication date: Jun-2022
https://doi.org/10.1109/TCSVT.2021.3106711
Show More Cited By

Index Terms

Reading between lines: high-rate, non-intrusive visual codes within regular videos via ImplicitCode
1. Networks
  1. Network types
    1. Mobile networks
    2. Wireless access networks

Recommendations

InFrame++: Achieve Simultaneous Screen-Human Viewing and Hidden Screen-Camera Communication
MobiSys '15: Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services

Recent efforts in visible light communication over screen-camera links have exploited the display for data communication. Such practices, albeit convenient, have led to contention between space allocated for users and content reserved for devices, in ...
ARTcode: preserve art and code in any image
UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

The ubiquitous QR codes and some similar barcodes are becoming a convenient and popular approach to impromptu communication between mobile devices and their surrounding cyber-physical world. However, such codes suffer from two common drawbacks: poor ...
Demo: Achieving Simultaneous Screen-Human Viewing and Hidden Screen-Camera Communication
MobiSys '15: Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services

We present and demonstrate INFRAME++, a novel system that enables concurrent, dual-mode, full-frame communication for both users and devices. It achieves unobtrusive screen-camera data communication without affecting the primary video-viewing experience ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp '15: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 2015

1302 pages

ISBN:9781450335744

DOI:10.1145/2750858

General Chairs:
Kenji Mase
Nagoya University, JP
,
Marc Langheinrich
Università della Svizzera italiana (USI), CH
,
Daniel Gatica-Perez
IDIAP-EPFL, CH
,
Program Chairs:
Hans Gellersen
Lancaster University, UK
,
Tanzeem Choudhury
Cornell University
,
Koji Yatani
Univerisity of Tokyo, JP

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Yahoo! Japan: Yahoo! Japan
SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing
FX Palo Alto Laboratory, Inc.: FX Palo Alto Laboratory, Inc.
Google Inc.
ACM: Association for Computing Machinery
Rakuten Institute of Technology: Rakuten Institute of Technology
Microsoft: Microsoft
Bell Labs: Bell Laboratories
SIGCHI: ACM Special Interest Group on Computer-Human Interaction
KDDI
Panasonic: Panasonic Corporation
NTT DoCoMo
Telefónica: Telefónica
ISTC-PC: Intel Science and Technology Center for Pervasive Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 September 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

US National Science Foundation (NSF)

Conference

UbiComp '15

Sponsor:

Yahoo! Japan
SIGMOBILE
FX Palo Alto Laboratory, Inc.
ACM
Rakuten Institute of Technology
Microsoft
Bell Labs
SIGCHI
Panasonic
Telefónica
ISTC-PC

UbiComp '15: The 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 7 - 11, 2015

Osaka, Japan

Acceptance Rates

UbiComp '15 Paper Acceptance Rate 101 of 394 submissions, 26%;

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
429
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xu JBrauers CKlein JJochims JKays R(2023)Symbol Position Recovery for Optical Camera Communication With High-Density Matrix CodesIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2022.323164833:7(3071-3086)Online publication date: Jul-2023
https://doi.org/10.1109/TCSVT.2022.3231648
Tapia MXu TWu ZZamalloa M(2022)SunBoxProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35346026:2(1-26)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3534602
Klein JXu JBrauers CJochims JKays R(2022)Investigations on Temporal Sampling and Patternless Frame Recovery for Asynchronous Display-Camera CommunicationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2021.310671132:6(4004-4015)Online publication date: Jun-2022
https://doi.org/10.1109/TCSVT.2021.3106711
Xu YDang FXu RChen XLiu Y(2022)LSync: A Universal Event-synchronizing Solution for Live StreamingIEEE INFOCOM 2022 - IEEE Conference on Computer Communications10.1109/INFOCOM48880.2022.9796933(2188-2197)Online publication date: 2-May-2022
https://dl.acm.org/doi/10.1109/INFOCOM48880.2022.9796933
Xu JKlein JJochims JWeissner NKays R(2022)A reliable and unobtrusive approach to display area detection for imperceptible display camera communicationJournal of Visual Communication and Image Representation10.1016/j.jvcir.2022.10351085:COnline publication date: 1-May-2022
https://dl.acm.org/doi/10.1016/j.jvcir.2022.103510
Tran VJayatilaka GAshok AMisra A(2021)DeepLightProceedings of the 20th International Conference on Information Processing in Sensor Networks (co-located with CPS-IoT Week 2021)10.1145/3412382.3458269(238-253)Online publication date: 18-May-2021
https://dl.acm.org/doi/10.1145/3412382.3458269
Zhang KZhao YWu CYang CHuang KPeng CLiu YYang Z(2021)ChromaCode: A Fully Imperceptible Screen-Camera Communication SystemIEEE Transactions on Mobile Computing10.1109/TMC.2019.295649320:3(861-876)Online publication date: 1-Mar-2021
https://doi.org/10.1109/TMC.2019.2956493
Ryu SLee KBae JKim J(2021)Spatial Frequency Modulation for Display-Camera Communication2021 International Conference on Electronics, Information, and Communication (ICEIC)10.1109/ICEIC51217.2021.9369764(1-5)Online publication date: 31-Jan-2021
https://doi.org/10.1109/ICEIC51217.2021.9369764
Liu NGao ZWang JZhai G(2021)Psycho-visual modulation based information display: introduction and surveyFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-019-8265-315:3Online publication date: 1-Jun-2021
https://dl.acm.org/doi/10.1007/s11704-019-8265-3
Abe SHiraki TFukushima SNaemura T(2020)[Paper] Imperceptible Color Vibration for Screen-Camera Communication via 2D Binary PatternITE Transactions on Media Technology and Applications10.3169/mta.8.1708:3(170-185)Online publication date: 2020
https://doi.org/10.3169/mta.8.170
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents