research-article

SENSeTREAM: enhancing online live experience with sensor-federated video stream using animated two-dimensional code

Authors:

Takuro Yonezawa,

Osamu Nakamura,

Hideyuki TokudaAuthors Info & Claims

UbiComp '14: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing

Pages 301 - 305

https://doi.org/10.1145/2632048.2632104

Published: 13 September 2014 Publication History

Abstract

We propose a novel technique that aggregates multiple sensor streams generated by totally different types of sensors into a visually enhanced video stream. This paper shows major features of SENSeTREAM and demonstrates enhancement of user experience in an online live music event. Since SENSeTREAM is a video stream with sensor values encoded in a two-dimensional graphical code, it can transmit multiple sensor data streams while maintaining their synchronization. A SENSeTREAM can be transmitted via existing live streaming services, and can be saved into existing video archive services. We have implemented a prototype SENSeTREAM generator and deployed it to an online live music event. Through the pilot study, we confirmed that SENSeTREAM works with popular streaming services, and provide a new media experience for live performances. We also indicate future direction for establishing visual stream aggregation and its applications.

Supplementary Material

MOV File (p301-yonezawa.mov)

Download
141.82 MB

MP4 File (p301-yonezawa.mp4)

Download
13.67 MB

References

[1]

Adobe systems. media encoder, video capture software, audio capture software adobe flash media live encoder. http://www.adobe.com/products/flashmediaserver/flashmediaencoder/.

[2]

Camtwist. http://camtwiststudio.com.

[3]

Khr 3hv humanoid robot kit. http://robosavvy.com/store/product_info.php/products_id/554.

[4]

Mpeg-7. http://mpeg.chiariglione.org/standards/mpeg-7.

[5]

zxing - multi-format 1d/2d barcode image processing library with clients for android, java. http://code.google.com/p/zxing/.

[6]

Engström, A., Esbjörnsson, M., and Juhlin, O. Mobile collaborative live video mixing. In Proceedings of the 10th international conference on Human computer interaction with mobile devices and services, MobileHCI '08, ACM (New York, NY, USA, 2008), 157--166.

Digital Library

[7]

Inc., U. Ustream.tv: You're on. http://www.ustream.tv/.

[8]

Intille, S., Larson, K., Tapia, E., Beaudin, J., Kaushik, P., Nawyn, J., and Rockinson, R. Using a live-in laboratory for ubiquitous computing research. In Pervasive Computing, K. Fishkin, B. Schiele, P. Nixon, and A. Quigley, Eds., vol. 3968 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2006, 349--365.

Digital Library

[9]

Liu, X., Doermann, D., and Li, H. Vcode - pervasive data transfer using video barcode. Trans. Multi. 10, 3 (Apr. 2008), 361--371.

Digital Library

[10]

LLC, Y. Youtube - live. http://www.youtube.com/live.

[11]

Potdar, V. M., Han, S., and Chang, E. A survey of digital image watermarking techniques. In Industrial Informatics, 2005. INDIN. 2005 3rd IEEE International Conference on, IEEE (Aug. 2005), 709--716.

[12]

Srivastava, M., Muntz, R., and Potkonjak, M. Smart kindergarten: Sensor-based wireless networks for smart developmental problem-solving environments. In Proceedings of the 7th Annual International Conference on Mobile Computing and Networking, MobiCom '01, ACM (New York, NY, USA, 2001), 132--138.

Digital Library

[13]

Su, N. M., Park, H., Bostrom, E., Burke, J., Srivastava, M. B., and Estrin, D. Augmenting film and video footage with sensor data. In In Proceedings of the Second IEEE International Conference on Pervasive Computing and Communications (PerCom 2004 (2004), 3--12.

Digital Library

[14]

Weisz, J. D., Kiesler, S., Zhang, H., Ren, Y., Kraut, R. E., and Konstan, J. A. Watching together: integrating text chat with video. In Proceedings of the SIGCHI conference on Human factors in computing systems, CHI '07, ACM (New York, NY, USA, 2007), 877--886.

Digital Library

[15]

Yonezawa, T., and Tokuda, H. Enhancing communication and dramatic impact of online live performance with cooperative audience control. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, UbiComp '12, ACM (New York, NY, USA, 2012), 103--112.

Digital Library

Cited By

Palomäki RMontoya Freire MDi Francesco M(2020)Distance-Dependent Barcodes for Context-Aware Mobile Applications22nd International Conference on Human-Computer Interaction with Mobile Devices and Services10.1145/3379503.3403534(1-11)Online publication date: 5-Oct-2020
https://dl.acm.org/doi/10.1145/3379503.3403534
Fujisawa KHirabe YSuwa HArakawa YYasumoto K(2018)Automatic Live Sport Video Streams Curation System From User Generated MediaSports Media, Marketing, and Management10.4018/978-1-5225-5475-2.ch022(417-435)Online publication date: 2018
https://doi.org/10.4018/978-1-5225-5475-2.ch022
Kurabayashi SHanaoka H(2017)A Large-Scale Visual Check-In System for TV Content-Aware Web with Client-Side Video Analysis OffloadingWeb Information Systems Engineering – WISE 201710.1007/978-3-319-68786-5_13(159-174)Online publication date: 4-Oct-2017
https://doi.org/10.1007/978-3-319-68786-5_13
Show More Cited By

Index Terms

SENSeTREAM: enhancing online live experience with sensor-federated video stream using animated two-dimensional code
1. Human-centered computing

Recommendations

I'd Hide You: Performing Live Broadcasting in Public
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

We present a study of a mixed reality game called 'I'd Hide You' that involves live video streaming from the city streets. We chart the significant challenges facing performers on the streets who must simultaneously engage in the game, stream compelling ...
A Collaborative Transcoding Strategy for Live Broadcasting Over Peer-to-Peer IPTV Networks

Real-time video transcoding that is often needed for robust video broadcasting over heterogeneous networks is not supported in most existing devices. To address this problem, we propose a collaborative strategy that leverages the peering architecture of ...
Senbay: smartphone-based activity capturing and sharing using sensor-federated video
UbiComp/ISWC'15 Adjunct: Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers

We present Senbay, a novel smartphone-based platform for capturing and sharing synchronously-recorded video/sensor data stream using animated two-dimensional barcode. In the ubicomp environment, mobile devices have enabled us to capture video and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp '14: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 2014

973 pages

ISBN:9781450329682

DOI:10.1145/2632048

General Chairs:
AJ Brush
Microsoft Research
,
Adrian Friday
Lancaster University, UK
,
Program Chairs:
Julie Kientz
University of Washington
,
James Scott
Microsoft Research, UK
,
Junehwa Song
Korea Advanced Institute of Science and Technology (KAIST), KR

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

SIGSPATIAL: ACM Special Interest Group on Spatial Information

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 September 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UbiComp '14

Sponsor:

UbiComp '14: The 2014 ACM Conference on Ubiquitous Computing

September 13 - 17, 2014

Washington, Seattle

Acceptance Rates

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
377
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Palomäki RMontoya Freire MDi Francesco M(2020)Distance-Dependent Barcodes for Context-Aware Mobile Applications22nd International Conference on Human-Computer Interaction with Mobile Devices and Services10.1145/3379503.3403534(1-11)Online publication date: 5-Oct-2020
https://dl.acm.org/doi/10.1145/3379503.3403534
Fujisawa KHirabe YSuwa HArakawa YYasumoto K(2018)Automatic Live Sport Video Streams Curation System From User Generated MediaSports Media, Marketing, and Management10.4018/978-1-5225-5475-2.ch022(417-435)Online publication date: 2018
https://doi.org/10.4018/978-1-5225-5475-2.ch022
Kurabayashi SHanaoka H(2017)A Large-Scale Visual Check-In System for TV Content-Aware Web with Client-Side Video Analysis OffloadingWeb Information Systems Engineering – WISE 201710.1007/978-3-319-68786-5_13(159-174)Online publication date: 4-Oct-2017
https://doi.org/10.1007/978-3-319-68786-5_13
Fujisawa KHirabe YSuwa HArakawa YYasumoto K(2016)Automatic Live Sport Video Streams Curation System from User Generated MediaInternational Journal of Multimedia Data Engineering & Management10.4018/IJMDEM.20160401037:2(36-52)Online publication date: 1-Apr-2016
https://dl.acm.org/doi/10.4018/IJMDEM.2016040103
Freire MDi Francesco M(2016)Reliable and bidirectional camera-display communications with smartphones2016 IEEE 17th International Symposium on A World of Wireless, Mobile and Multimedia Networks (WoWMoM)10.1109/WoWMoM.2016.7523499(1-7)Online publication date: Jun-2016
https://doi.org/10.1109/WoWMoM.2016.7523499
Nishiyama YOkoshi TChen YYonezawa TNakazawa JTokuda HMase KLangheinrich MGatica-Perez DGellersen HChoudhury TYatani K(2015)SenbayAdjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers10.1145/2800835.2800847(21-24)Online publication date: 7-Sep-2015
https://dl.acm.org/doi/10.1145/2800835.2800847
Li TAn CXiao XCampbell AZhou XBorriello GPau GGruteser MHong J(2015)Real-Time Screen-Camera Communication Behind Any SceneProceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services10.1145/2742647.2742667(197-211)Online publication date: 18-May-2015
https://dl.acm.org/doi/10.1145/2742647.2742667
Fujisawa KHirabe YSuwa HArakawa YYasumoto K(2015)Automatic Content Curation System for Multiple Live Sport Video Streams2015 IEEE International Symposium on Multimedia (ISM)10.1109/ISM.2015.17(541-546)Online publication date: Dec-2015
https://doi.org/10.1109/ISM.2015.17
Ogawa MYonezawa TNishiyama YNakazawa JTokuda H(2015)A robot control system for video streaming services by using dynamic encoded QR codes2015 Eighth International Conference on Mobile Computing and Ubiquitous Networking (ICMU)10.1109/ICMU.2015.7061042(86-87)Online publication date: Jan-2015
https://doi.org/10.1109/ICMU.2015.7061042

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten