Article

PhoneGuide: museum guidance supported by on-device object recognition on mobile phones

Authors:

Thomas Zeidler,

Benjamin Brombach,

Oliver BimberAuthors Info & Claims

MUM '05: Proceedings of the 4th international conference on Mobile and ubiquitous multimedia

Pages 3 - 10

https://doi.org/10.1145/1149488.1149490

Published: 08 December 2005 Publication History

Abstract

We present PhoneGuide -- an enhanced museum guidance system that uses camera-equipped mobile phones and on-device object recognition.Our main technical achievement is a simple and light-weight object recognition approach that is realized with single-layer perceptron neuronal networks. In contrast to related systems which perform computationally intensive image processing tasks on remote servers, our intention is to carry out all computations directly on the phone. This ensures little or even no network traffic and consequently decreases cost for online times. Our laboratory experiments and field surveys have shown that photographed museum exhibits can be recognized with a probability of over 90%.We have evaluated different feature sets to optimize the recognition rate and performance. Our experiments revealed that normalized color features are most effective for our method. Choosing such a feature set allows recognizing an object below one second on up-to-date phones. The amount of data that is required for differentiating 50 objects from multiple perspectives is less than 6KBytes.

References

[1]

{And03} Andreasson, H. and Duckett, T., "Object Recognition by a Mobile Robot using Omni-directional Vision", Proceedings of the Eighth Scandinavian Conference on Artificial Intelligence, Norway, 2003.]]

[2]

{Aok00} Aoki, P. M. and Woodruff A., "Improving Electronic Guidebook Interfaces Using a Task-Oriented Design Approach", Proc. 3rd ACM Conf. on Designing Interactive Systems, New York, NY, pp. 319--325, 2000.]]

Digital Library

[3]

{Ass03} Assad, M., Carmichael, D. J., Cutting, D., and Hudson, A., "AR phone: Accessible Augmented Reality in the Intelligent Environment", In OZCHI'03, pp. 232--235, 2003.]]

[4]

{Bom03} Bombara, M, Cali, D., and Santoro, C, "KORE: A Multi-Agent System to Assist Museum Visitors", Joint Workshop "From Objects to Agents": Intelligent Systems and Pervasive Computing, pp. 175--178, 2003.]]

[5]

{Bon04} University of Bonn, "Das Fotohandy als Fremdenführer", retrieved from WWW, http://www.ipb.uni-bonn.de/FotoNav/andhttp://www.geosciencenline.de/index.php?cmd=wissen_details&id=1713&datum=2004-10-12, 2004.]]

[6]

{Cla05} C-Lab, "Kickreal", retrieved from WWW, www.c-lab.de or www.kickreal.de, 2005.]]

[7]

{Cor00} Coors, V., Huch, T., and Kretschmar, U., "Matching Buildings: Pose Estimation in an Urban Environment", Proc. ISMAR'00, Munich, Germany, pp. 89--92, 2000.]]

[8]

{Fei97} Feiner, S., Maclntyre, B., Höllerer, T., and Webster, A., "A Touring Machine: Prototyping 3D Mobile Augmented Reality Systems for Exploring the Urban Environment", Proceedings 1st International Symposium on Wearable Computing, Cambridge, MA, pp. 74--81, 1997.]]

Digital Library

[9]

{Fri04a} Fritz, G., Seifert, C., Paletta, L., and Bischof H., "Rapid Object Recognition from Discriminative Regions of Interest", Proc. National Conference on Artificial Intelligence, AAAI, San Rose, CA, 2004.]]

[10]

{Fri04b} Fritz, G., Seifert, C, Luley, P., Paletta, L., and Almer, A., "Mobile Vision for Ambient Learning in Urban Enviroments", In MLEARN 2004, Lake Bracciano, Rome, July 2004.]]

[11]

{Har88} Harris, C. and Stephens, M., "A combined corner and edge detector", In Alvey Vision Conference, pages 147--151, 1988.]]

[12]

{Hel04} Helmer, S. and Lowe, D. G., "Object Class Recognition with Many Local Features", In GMBV 2004, Washington, D.C., July 2004.]]

[13]

{Iqb02} Iqbal, Q. and Aggarwal, J. K., "CIRES: A System for content-based retrieval in digital image libraries", In ICARCV'02, 2002.]]

[14]

{Leh00} Lehmann, T. M., Wein, B. B., Dahmen, J., Bredno, J., Vogelsang, F., and Kohnen, M., "Content-Based Image Retrieval in Medical Applications": A Novel Multi-Step Approach, Proceedings SPIE'00, vol. 3972, pp. 312--320, 2002.]]

[15]

{Low99} Lowe, D. G., "Object Recognition from Local Scale-Invariant Features", International Conference on Computer Vision, Greece, pp. 1150--1157, 1999.]]

Digital Library

[16]

{Low04} Lowe, D. G., "Distinctive image features from scale-invariant keypoints", International Journal on Computer Vision, vol. 60, pp. 91--110, 2004.]]

Digital Library

[17]

{Mac04} Macedonia, M., "Small is Beautiful", IEEE Computer, vol. 37, no. 12, pp. 122--123, 2004.]]

Digital Library

[18]

{Mik02} Mikolajczyk, K and Schmid, C, "An affine invariant interest point detector", European Conference on Computer Vision, Copenhagen, pp. 128--142, 2002.]]

Digital Library

[19]

{MIT04} MIT's Technology Review, "Markets and Trends", p. 16, February 2004.]]

[20]

{Mnh04} Museum of Natural History, "Museum Puts Tags on Stuffed Birds", retrieved from WWW, http://rfidjournal.com/article/articleview/1110/1/1/, 2004.]]

[21]

{Moe04} Moehring, M., Lessig, C, and Bimber, O., "Optical Tracking and Video See-Through AR on Consumer Cell Phones", In proceedings of Workshop on Virtual and Augmented Reality of the GI-Fachgruppe AR/VR, pp. 193--204, 2004.]]

Digital Library

[22]

{Pol03} Porikli, F. M., "Inter-Camera Color Calibration by Cross-Correlation Model Function", IEEE International Conference on Image Processing (ICIP), Vol. 2, pp. 133--136, September 2003.]]

[23]

{Sch96} Schiele, B. and Crowley, J. L., "Object recognition using multidimensional receptive field histograms", Fourth European Conference on Computer Vision, Cambridge, UK, pp. 610--619, 1996.]]

Digital Library

[24]

{Sch97} Schmid C. and Mohr, R., "Local grayvalue invariants for image retrieval", IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(5):530--534, 1997.]]

Digital Library

[25]

{Sei04} Seifert, C, Paletta, L., Jeitler, A., Hoedl, E., Andreu, J. P., Luley, P., and Almer A., "Visual Object Detection for Mobile Road Sign Inventory", In: Brewster S. and Dunlop M., (Eds.): Mobile HCI, LNCS 3160, pp. 491--495, Springer Verlag Berlin, 2004.]]

[26]

{Sem04} Semacode Cooperation, "Semacode", retrieved from WWW, http://www.semacode.org, 2004.]]

[27]

{Swa91} Swain, M. and Ballard, D., "Color indexing", International Journal of Computer Vision, vol. 7, no. 1, pp. 11--32, 1991.]]

Digital Library

[28]

{Wag03} Wagner, D. and Schmalstieg, D., "First steps towards handheld augmented reality", In proceedings of International Conference on Wearable Computers, pp. 127--136, 2003.]]

Digital Library

Cited By

Vetrivel SSabareeshwari VArun VSowmiya K(2024)Managing Smart Technologies in the Digital AgeSmart Technologies and Innovations in E-Business10.4018/978-1-6684-7840-0.ch001(1-24)Online publication date: 19-Jul-2024
https://doi.org/10.4018/978-1-6684-7840-0.ch001
Adel A(2022)Future of industry 5.0 in society: human-centric solutions, challenges and prospective research areasJournal of Cloud Computing10.1186/s13677-022-00314-511:1Online publication date: 8-Sep-2022
https://doi.org/10.1186/s13677-022-00314-5
Shang JChen SWu JYin S(2022)ARSpy: Breaking Location-Based Multi-Player Augmented Reality Application for User Location TrackingIEEE Transactions on Mobile Computing10.1109/TMC.2020.300774021:2(433-447)Online publication date: 1-Feb-2022
https://doi.org/10.1109/TMC.2020.3007740
Show More Cited By

Index Terms

PhoneGuide: museum guidance supported by on-device object recognition on mobile phones
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Scene understanding
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens

Recommendations

Enabling Mobile Phones To Support Large-Scale Museum Guidance

The authors present a museum guidance system called PhoneGuide that uses widespread camera-equipped mobile phones for on-deviceobject recognition in combination with pervasive tracking. It provides additional location- and object-aware multimedia ...
Localization and Classification through Adaptive Pathway Analysis

Evaluating user-generated spatiotemporal pathway data helps determine both the present and future location of museum visitors. The PhoneGuide adaptive mobile museum guidance system shows how this approach improves classification performance and achieves ...
PhoneGuide: Adaptive Image Classification for Mobile Museum Guidance
ISUVR '11: Proceedings of the 2011 International Symposium on Ubiquitous Virtual Reality

This paper summarizes the various components of our mobile museum guidance system PhoneGuide. It explains how practically viable object recognition rates can be achieved under realistic conditions using adaptive image classification.

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

MUM '05: Proceedings of the 4th international conference on Mobile and ubiquitous multimedia

December 2005

148 pages

ISBN:0473106582

DOI:10.1145/1149488

General Chair:
Mark Billinghurst
University of Canterbury, NZ

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 December 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

MUM05

MUM05: The Fourth International Conference on Mobile and Ubiquitous Multimedia

December 8 - 10, 2005

Christchurch, New Zealand

Acceptance Rates

Overall Acceptance Rate 190 of 465 submissions, 41%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

57
Total Citations
View Citations
1,348
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)2

Reflects downloads up to 02 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Vetrivel SSabareeshwari VArun VSowmiya K(2024)Managing Smart Technologies in the Digital AgeSmart Technologies and Innovations in E-Business10.4018/978-1-6684-7840-0.ch001(1-24)Online publication date: 19-Jul-2024
https://doi.org/10.4018/978-1-6684-7840-0.ch001
Adel A(2022)Future of industry 5.0 in society: human-centric solutions, challenges and prospective research areasJournal of Cloud Computing10.1186/s13677-022-00314-511:1Online publication date: 8-Sep-2022
https://doi.org/10.1186/s13677-022-00314-5
Shang JChen SWu JYin S(2022)ARSpy: Breaking Location-Based Multi-Player Augmented Reality Application for User Location TrackingIEEE Transactions on Mobile Computing10.1109/TMC.2020.300774021:2(433-447)Online publication date: 1-Feb-2022
https://doi.org/10.1109/TMC.2020.3007740
Peng ZHou SYuan Y(2022)EPAR: An Efficient and Privacy-Aware Augmented Reality Framework for Indoor Location-Based Services2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS47612.2022.9981149(8948-8955)Online publication date: 23-Oct-2022
https://doi.org/10.1109/IROS47612.2022.9981149
Alwis CPham QLiyanage M(2022)Bibliography6G Frontiers10.1002/9781119862321.ref(241-291)Online publication date: 25-Nov-2022
https://doi.org/10.1002/9781119862321.ref
Ahmetovic DBernareggi CKeller KMascetti SVazquez SDrake TAhmetovic DYaneva V(2021)MusAProceedings of the 18th International Web for All Conference10.1145/3430263.3452441(1-9)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3430263.3452441
Siriwardhana YPorambage PLiyanage MYlianttila M(2021)A Survey on Mobile Augmented Reality With 5G Mobile Edge Computing: Architectures, Applications, and Technical AspectsIEEE Communications Surveys & Tutorials10.1109/COMST.2021.306198123:2(1160-1192)Online publication date: Oct-2022
https://doi.org/10.1109/COMST.2021.3061981
Kotut LHoang APanda HShenk JNewbill PKellogg DScott McCrickard D(2021)Supporting child–group interactions with hands-off museum exhibitInternational Journal of Child-Computer Interaction10.1016/j.ijcci.2020.10024027:COnline publication date: 1-Mar-2021
https://dl.acm.org/doi/10.1016/j.ijcci.2020.100240
Mairittha NMairittha TInoue S(2019)On-Device Deep Learning Inference for Efficient Activity Data CollectionSensors10.3390/s1915343419:15(3434)Online publication date: 5-Aug-2019
https://doi.org/10.3390/s19153434
Pierdicca RMarques-Pita MPaolanti MMalinverni E(2019)IoT and Engagement in the Ubiquitous MuseumSensors10.3390/s1906138719:6(1387)Online publication date: 21-Mar-2019
https://doi.org/10.3390/s19061387
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents