research-article

TagSense: a smartphone-based approach to automatic image tagging

Authors:

Romit Roy Choudhury,

Srihari NelakuditiAuthors Info & Claims

MobiSys '11: Proceedings of the 9th international conference on Mobile systems, applications, and services

Pages 1 - 14

https://doi.org/10.1145/1999995.1999997

Published: 28 June 2011 Publication History

Abstract

Mobile phones are becoming the convergent platform for personal sensing, computing, and communication. This paper attempts to exploit this convergence towards the problem of automatic image tagging. We envision TagSense, a mobile phone based collaborative system that senses the people, activity, and context in a picture, and merges them carefully to create tags on-the-fly. The main challenge pertains to discriminating phone users that are in the picture from those that are not. We deploy a prototype of TagSense on 8 Android phones, and demonstrate its effectiveness through 200 pictures, taken in various social settings. While research in face recognition continues to improve image tagging, TagSense is an attempt to embrace additional dimensions of sensing towards this end goal. Performance comparison with Apple iPhoto and Google Picasa shows that such an out-of-band approach is valuable, especially with increasing device density and greater sophistication in sensing/learning algorithms.

References

[1]

Tingxin Yan, Deepak Ganesan, and R. Manmatha, 'Distributed image search in camera sensor networks,' ACM SenSys, pp. 155--168, Nov 2008.

Digital Library

[2]

Amazon, 'Amazon Mechanical Turk,' https://www.mturk.com/mturk/welcome.

[3]

Google Image Labeler, 'http://images.google.com/imagelabeler/,'.

[4]

L. Von Ahn and L. Dabbish, 'Labeling images with a computer game,' in ACM SIGCHI, 2004.

Digital Library

[5]

Tingxin Yan, Vikas Kumar, and Deepak Ganesan, 'Crowdsearch: exploiting crowds for accurate real-time image search on mobile phones,' in ACM MobiSys, 2010.

Digital Library

[6]

T. Nakakura, Y. Sumi, and T. Nishida, 'Neary: conversation field detection based on similarity of auditory situation,' ACM HotMobile, 2009.

Digital Library

[7]

H. Lu, W. Pan, N. D. Lane, T. Choudhury, and A. T. Campbell, 'SoundSense: scalable sound sensing for people-centric applications on mobile phones,' in ACM MobiSys, 2009.

Digital Library

[8]

A. Engstrom, M. Esbjornsson, and O. Juhlin, 'Mobile collaborative live video mixing,' Mobile Multimedia Workshop (with MobileHCI), Sep 2008.

Digital Library

[9]

Google Goggles, 'http://www.google.com/mobile/goggles/,'.

[10]

L. Bao and S.S. Intille, 'Activity recognition from user-annotated acceleration data,' Pervasive Computing, 2004.

[11]

D.H. Hu, S.J. Pan, V.W. Zheng, N.N. Liu, and Q. Yang, 'Real world activity recognition with multiple goals,' in ACM UbiComp, 2008.

Digital Library

[12]

M. Azizyan, I. Constandache, and R. Roy Choudhury, 'SurroundSense: mobile phone localization via ambience fingerprinting,' in ACM MobiCom, 2009.

Digital Library

[13]

C. Liu, 'Beyond Pixels: Exploring New Representations and Applications for Motion Analysis,' in Doctoral Thesis MIT, 2009.

Digital Library

[14]

E. Miluzzo, N. D. Lane, K. Fodor, R. Peterson, H. Lu, M. Musolesi, S. B. Eisenman, X. Zheng, and A. T. Campbell, 'Sensing Meets Mobile Social Networks: The Design, Implementation and Evaluation of CenceMe Application,' in ACM Sensys, 2008.

Digital Library

[15]

M. Braun and R. Spring, 'Enkin,' http://enkinblog.blogspot.com/.

[16]

E. Aronson, N. Blaney, C. Stephan, J. Sikes, and M. Snapp, 'The jigsaw classroom,' Improving Academic Achievement: Impact of Psychological Factors on Education, 2002.

[17]

A.A. Sani, L. Zhong, and A. Sabharwal, 'Directional Antenna Diversity for Mobile Devices: Characterizations and Solutions,' in ACM MobiCom, 2010.

Digital Library

[18]

K. Chintalapudi, A. Padmanabha Iyer, and V.N. Padmanabhan, 'Indoor localization without the pain,' in ACM Mobicom, 2010.

Digital Library

[19]

C. Peng, G. Shen, Z. Han, Y. Zhang, Y. Li, and K. Tan, 'A beepbeep ranging system on mobile phones,' in ACM SenSys, 2007.

Digital Library

[20]

Nokia Siemens Networks, 'Unite: Trends and insights 2009,' 2009.

[21]

Sam Grobart, 'In Smartphone Era, Point-and-Shoots Stay Home,' New York Times, Dec 2010.

[22]

R. Datta, D. Joshi, J. Li, and J.Z. Wang, 'Image retrieval: Ideas, influences, and trends of the new age,' ACM CSUR, 2008.

Digital Library

[23]

Gustavo Carneiro, Antoni B. Chan, Pedro J. Moreno, and Nuno Vasconcelos, 'Supervised learning of semantic classes for image annotation and retrieval,' IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, pp. 2007, 2007.

Digital Library

[24]

Alipr, 'Automatic Photo Tagging and Visual Image Search,' http://alipr.com/.

[25]

Mor Naaman, Ron B. Yeh, Hector Garcia-Molina, and Andreas Paepcke, 'Leveraging context to resolve identity in photo albums,' in Proc. of the 5th ACM/IEEE-CS joint conference on Digital libraries, 2005, JCDL '05.

Digital Library

[26]

Risto Sarvas, Erick Herrarte, Anita Wilhelm, and Marc Davis, 'Metadata creation system for mobile images,' in ACM MobiSys, 2004.

Digital Library

[27]

Shwetak N. Patel and Gregory D. Abowd, 'The contextcam: Automated point of capture video annotation,' in Proc. of the 6th International Conference on Ubiquitous Computing, 2004.

[28]

R. Want, 'When cell phones become computers,' IEEE Pervasive Computing, IEEE, 2009.

Digital Library

[29]

R.K. Balan, D. Gergle, M. Satyanarayanan, and J. Herbsleb, 'Simplifying cyber foraging for mobile devices,' in ACM MobiSys, 2007.

Digital Library

[30]

D.H. Nguyen, G. Marcu, G.R. Hayes, K.N. Truong, J. Scott, M. Langheinrich, and C. Roduner, 'Encountering SenseCam: personal recording technologies in everyday life,' in ACM Ubiquitous computing, 2009.

Digital Library

[31]

P. Mohan, V. N. Padmanabhan, and R. Ramjee, 'Nericell: Rich monitoring of road and traffic conditions using mobile smartphones,' in ACM SenSys, 2008.

Digital Library

[32]

J. Lester, B. Hannaford, and G. Borriello, 'ÒAre You with Me?Ó-Using Accelerometers to Determine If Two Devices Are Carried by the Same Person,' Pervasive Computing, 2004.

[33]

T. van Kasteren, A. Noulas, G. Englebienne, and B. Krose, 'Accurate activity recognition in a home setting,' in ACM Ubicomp, 2008.

Digital Library

[34]

M. Leo, T. D'Orazio, I. Gnoni, P. Spagnolo, and A. Distante, 'Complex human activity recognition for monitoring wide outdoor environments,' in IEEE ICPR, 2004.

Digital Library

[35]

B. Logan, 'Mel frequency cepstral coefficients for music modeling,' in ISMIR, 2000.

[36]

S. Baker, D. Scharstein, JP Lewis, S. Roth, M.J. Black, and R. Szeliski, 'A database and evaluation methodology for optical flow,' in IEEE ICCV, 2007.

[37]

Joshua J. Romero, 'Smartphones: The Pocketable PC,' IEEE Spectrum, Jan 2011.

Cited By

Mehmeti FFelemban NLu ZWheatman KCirincione GLa Porta T(2021)Quality of Information in Gathering Information via Video Analytics for Military NetworksIEEE Communications Magazine10.1109/MCOM.001.200085259:2(50-55)Online publication date: Feb-2021
https://doi.org/10.1109/MCOM.001.2000852
Lu ZChan KUrgaonkar RPu SLa Porta T(2020)NetVision: On-Demand Video Processing in Wireless NetworksIEEE/ACM Transactions on Networking10.1109/TNET.2019.295490928:1(196-209)Online publication date: Feb-2020
https://doi.org/10.1109/TNET.2019.2954909
Kiptoo C(2020)Ontology enhancement using crowdsourcing: a conceptual architectureInternational Journal of Crowd Science10.1108/IJCS-10-2019-00284:3(231-243)Online publication date: 27-Apr-2020
https://doi.org/10.1108/IJCS-10-2019-0028
Show More Cited By

Index Terms

TagSense: a smartphone-based approach to automatic image tagging

Recommendations

TagSense: Leveraging Smartphones for Automatic Image Tagging

Mobile phones are becoming the convergent platform for personal sensing, computing, and communication. This paper attempts to exploit this convergence toward the problem of automatic image tagging. We envision TagSense, a mobile phone-based collaborative ...
Demo: an out-of-band alternative to face recognition
MobiSys '11: Proceedings of the 9th international conference on Mobile systems, applications, and services

Smartphones are becoming the convergent platform for personal sensing, computing, and communication. Our work attempts to exploit this convergence towards the problem of automatic image tagging. We envision TagSense, a smartphone based collaborative ...
Design and implementation of algorithm for estimation of elevator travel distance using smartphone accelerometer
UbiComp/ISWC'15 Adjunct: Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers

Since the advent of smartphones equipped with sophisticated sensing hardware, human activity recognition research has moved from utilizing dedicated sensing devices to using commercial smartphones. This paper presents the design of an algorithm to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MobiSys '11: Proceedings of the 9th international conference on Mobile systems, applications, and services

June 2011

430 pages

ISBN:9781450306430

DOI:10.1145/1999995

General Chair:
Ashok Agrawala
University of Maryland, USA
,
Program Chairs:
Mark D. Corner
UMass Amherst, USA
,
David Wetherall
University of Wash. & Intel Labs, USA

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

In-Cooperation

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 June 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MobiSys'11

Sponsor:

SIGMOBILE

MobiSys'11: The 9th International Conference on Mobile Systems, Applications, and Services

June 28 - July 1, 2011

Maryland, Bethesda, USA

Acceptance Rates

Overall Acceptance Rate 274 of 1,679 submissions, 16%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

71
Total Citations
View Citations
1,072
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)2

Reflects downloads up to 14 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mehmeti FFelemban NLu ZWheatman KCirincione GLa Porta T(2021)Quality of Information in Gathering Information via Video Analytics for Military NetworksIEEE Communications Magazine10.1109/MCOM.001.200085259:2(50-55)Online publication date: Feb-2021
https://doi.org/10.1109/MCOM.001.2000852
Lu ZChan KUrgaonkar RPu SLa Porta T(2020)NetVision: On-Demand Video Processing in Wireless NetworksIEEE/ACM Transactions on Networking10.1109/TNET.2019.295490928:1(196-209)Online publication date: Feb-2020
https://doi.org/10.1109/TNET.2019.2954909
Kiptoo C(2020)Ontology enhancement using crowdsourcing: a conceptual architectureInternational Journal of Crowd Science10.1108/IJCS-10-2019-00284:3(231-243)Online publication date: 27-Apr-2020
https://doi.org/10.1108/IJCS-10-2019-0028
Ustunkok TAcar OKarakaya M(2019)Image Tag Refinement with Self Organizing Maps2019 1st International Informatics and Software Engineering Conference (UBMYK)10.1109/UBMYK48245.2019.8965477(1-6)Online publication date: Nov-2019
https://doi.org/10.1109/UBMYK48245.2019.8965477
Debnath HGehani NDing XCurtmola RBorcea C(2018)Sentio: Distributed Sensor Virtualization for Mobile Apps2018 IEEE International Conference on Pervasive Computing and Communications (PerCom)10.1109/PERCOM.2018.8444605(1-9)Online publication date: Mar-2018
https://doi.org/10.1109/PERCOM.2018.8444605
Alahmadi MYang J(2018)Towards Efficient Mobile Augmented Reality in Indoor EnvironmentsArtificial Intelligence and Mobile Services – AIMS 201810.1007/978-3-319-94361-9_9(106-122)Online publication date: 21-Jun-2018
https://doi.org/10.1007/978-3-319-94361-9_9
Wang LYang JLiu W(2017)Leveraging Participatory Extraction to Mobility Sensing for Individual Discovery in Crowded EnvironmentsInternational Journal of Distributed Sensor Networks10.1155/2013/2469169:10(246916)Online publication date: 27-Mar-2017
https://doi.org/10.1155/2013/246916
Kim SPatra KKim ALee KSegev ALee UMark GFussell SLampe Cschraefel mHourcade JAppert CWigdor D(2017)Sensors Know Which Photos Are MemorableProceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems10.1145/3027063.3053198(2706-2713)Online publication date: 6-May-2017
https://dl.acm.org/doi/10.1145/3027063.3053198
Ren SGuo BWu JWang QYu Z(2017)The emergence of visual-based localization and navigation using smartphone sensing2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI)10.1109/UIC-ATC.2017.8397508(1-8)Online publication date: Aug-2017
https://doi.org/10.1109/UIC-ATC.2017.8397508
Zheng YShen GLi LZhao CLi MZhao FYuanqing Zheng Guobin Shen Liqun Li Chunshui Zhao Mo Li Feng Zhao (2017)Travi-NaviIEEE/ACM Transactions on Networking10.1109/TNET.2017.270710125:5(2655-2669)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1109/TNET.2017.2707101
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents