research-article

Tell me what i see: recognize RFID tagged objects in augmented reality systems

Authors:

Sanglu LuAuthors Info & Claims

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

Pages 916 - 927

https://doi.org/10.1145/2971648.2971661

Published: 12 September 2016 Publication History

Abstract

Nowadays, people usually depend on augmented reality (AR) systems to obtain an augmented view in a real-world environment. With the help of advanced AR technology (e.g. object recognition), users can effectively distinguish multiple objects of different types. However, these techniques can only offer limited degrees of distinctions among different objects and cannot provide more inherent information about these objects. In this paper, we leverage RFID technology to further label different objects with RFID tags. We deploy additional RFID antennas to the COTS depth camera and propose a continuous scanning-based scheme to scan the objects, i.e., the system continuously rotates and samples the depth of field and RF-signals from these tagged objects. In this way, by pairing the tags with the objects according to the correlations between the depth of field and RF-signals, we can accurately identify and distinguish multiple tagged objects to realize the vision of "tell me what I see" from the augmented reality system. For example, in front of multiple unknown people wearing RFID tagged badges in public events, our system can identify these people and further show their inherent information from the RFID tags, such as their names, jobs, titles, etc. We have implemented a prototype system to evaluate the actual performance. The experiment results show that our solution achieves an average match ratio of 91% in distinguishing up to dozens of tagged objects with a high deployment density.

References

[1]

S. Azzouzi, M. Cremer, U. Dettmar, R. Kronberger, and T. Knie. 2011. New measurement results for the localization of UHF RFID transponders using an Angle of Arrival (AoA) approach. In Proc. of IEEE RFID.

[2]

M. Buettner, R. Prasad, M. Philipose, and D. Wetherall. 2009. Recognizing Daily Activities with RFID-based Sensors. In Proc. of UbiComp. 51--60.

Digital Library

[3]

C. Cerrada, S. Salamanca, E. Perez, J. A. Cerrada, and I. Abad. 2007. Fusion of 3D vision techniques and RFID technology for object recognition in complex scenes. In IEEE International Symposium on Intelligent Signal Processing.

[4]

H. Ding, L. Shangguan, Z. Yang, J. Han, Z. Zhou, P. Yang, W. Xi, and J. Zhao. 2015. FEMO: A Platform for Free-weight Exercise Monitoring with RFIDs. In Proc. of ACM SenSys.

Digital Library

[5]

B. Drost, M. Ulrich, N. Navab, and S. Ilic. 2010. Model globally, match locally: Efficient and robust 3D object recognition. In Proc. of IEEE CVPR.

[6]

G. Galatas and F. Makedon. 2013. A system for multimodal context-awareness. International Journal of Advanced Computer Science and Applications (2013).

[7]

C. Godfrey and A. W. Siddons. 1908. Apollonius' Theorem. Modern Geometry (1908), 20.

[8]

J. Han, L. Shao, D. Xu, and J. Shotton. 2013. Enhanced computer vision with microsoft kinect sensor: A review. IEEE Transactions on Cybernetics 43, 5 (2013), 1318--1334.

[9]

J.D. Hincapié-Ramos, A.n Tabard, and J. E. Bardram. 2011. Mediated Tabletop Interaction in the Biology Lab: Exploring the Design Space of the Rabbit. In Proc. of UbiComp. 301--310.

Digital Library

[10]

ImpinJ 2016. ImpinJ. (2016). http://www.impinj.com.

[11]

S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, and A. Davison. 2011. KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In Proc. of ACM UIST.

Digital Library

[12]

K. Kendig. 2000. Is a 2000-Year-Old Formula Still Keeping Some Secrets? Amer. Math. Monthly 107 (2000), 402--415.

[13]

Kinect 2016. Kinect. (2016). http://www.microsoft.com/en-us/kinectforwindows.

[14]

F. Klompmaker, H. Fischer, and H. Jung. 2012. Authenticated Tangible Interaction using RFID and Depth-Sensing Cameras. In Proc. of ACHI.

[15]

J. Liu, B. Xiao, K. Bu, and L. Chen. 2014. Efficient Distributed Query Processing in Large RFID-enabled Supply Chains. In Proc. of IEEE INFOCOM. 163--171.

[16]

J. Liu, B. Xiao, S. Chen, F. Zhu, and L. Chen. 2015. Fast RFID grouping protocols. In Proc. of IEEE INFOCOM. 1948--1956.

[17]

T. Liu, L. Yang, Q. Lin, Y. Guo, and Y. Liu. 2014. Anchor-free backscatter positioning for RFID tags with high accuracy. In Proc. of IEEE INFOCOM.

[18]

A. Mian, M. Bennamoun, and R. Owens. 2010. On the repeatability and quality of keypoints for local feature-based 3d object retrieval from cluttered scenes. International Journal of Computer Vision (2010).

Digital Library

[19]

J. Nickels, P. Knierim, B. Könings, F. Schaub, B. Wiedersheim, S. Musiol, and M. Weber. 2013. Find My Stuff: Supporting Physical Objects Search with Relative Positioning. In Proc. of UbiComp.

Digital Library

[20]

S. Nirjon and J. Stankovic. 2012. Kinsight: Localizing and tracking household objects using depth-camera sensors. In Proc. of IEEE DCOSS.

Digital Library

[21]

S. Obdrzalek, G. Kurillo, F. Ofli, R. Bajcsy, E. Seto, H. Jimison, and M. Pavel. 2012. Accuracy and robustness of Kinect pose estimation in the context of coaching of elderly population. In Proc. of IEEE EMBC.

[22]

I. Oikonomidis, N. Kyriazis, and A. A Argyros. 2011. Efficient model-based 3D tracking of hand articulations using Kinect. In British Machine Vision Conference, Vol. 1. 3.

[23]

Z. Ren, J. Yuan, and Z. Zhang. 2011. Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera. In Proc. of ACM Multimedia.

Digital Library

[24]

L. Shangguan, Z. Li, Z. Yang, M. Li, and Y. Liu. 2013. Otrack: Order tracking for luggage in mobile RFID systems. In Proc. of IEEE INFOCOM.

[25]

L. Shangguan, Z. Yang, A. X. Liu, Z. Zhou, and Y. Liu. 2015. Relative Localization of RFID Tags using Spatial-Temporal Phase Profiling. In Proc. of NSDI.

Digital Library

[26]

C. S. Wang, C. L. Chen, and Y. M. Guo. 2013. Real-Time Indoor Positioning System Based on RFID and Kinect. In Proc. of Information Technology Convergence.

[27]

J. Wang, F. Adib, R. Knepper, D. Katabi, and D. Rus. 2013. RF-compass: Robot Object Manipulation Using RFIDs. In Proc. of ACM MobiCom.

Digital Library

[28]

J. Wang and D. Katabi. 2013. Dude, where's my card?: RFID positioning that works with multipath and non-line of sight. In Proc. of the ACM SIGCOMM.

Digital Library

[29]

J. Wang, D. Vasisht, and D. Katabi. 2014. RF-IDraw: Virtual Touch Screen in the Air Using RF Signals. In Proc. of ACM SIGCOMM.

Digital Library

[30]

L. Xia, C. C. Chen, and J. K. Aggarwal. 2011. Human detection using depth information by Kinect. In Proc. of IEEE CVPRW. 15--22.

[31]

L. Xie, Q. Li, X. Chen, S. Lu, and D. Chen. 2013. Continuous scanning with mobile reader in RFID systems: an experimental study. In Proc. of ACM MobiHoc.

Digital Library

[32]

L. Yang, Y. Chen, X. Li, C. Xiao, M. Li, and Y. Liu. 2014. Tagoram: Real-time Tracking of Mobile RFID Tags to High Precision Using COTS Devices. In Proc. of ACM MobiCom.

Digital Library

[33]

L. Yang, Q. Lin, X. Y. Li, T. Liu, and Y. Liu. 2015. See Through Walls with COTS RFID System. In Proc. of MobiCom.

Digital Library

[34]

L. Yang, Y. Qi, J. Fang, X. Ding, T. Liu, and M. Li. 2014. Frogeye: Perception of the slightest tag motion. In Proc. of IEEE INFOCOM.

Cited By

Tavanti ENepa PGabbrielli RPirozzi M(2025)Review on Systems Combining Computer Vision and Radio Frequency IdentificationIEEE Internet of Things Journal10.1109/JIOT.2024.348475512:2(1291-1319)Online publication date: 15-Jan-2025
https://doi.org/10.1109/JIOT.2024.3484755
Shang FYang PXiong JFeng YLi X(2023)Tamera: Contactless Commodity Tracking, Material and Shopping Behavior Recognition Using COTS RFIDsACM Transactions on Sensor Networks10.1145/356377719:2(1-24)Online publication date: 3-Feb-2023
https://dl.acm.org/doi/10.1145/3563777
Boroushaki TLam MChen WDodds LEid AAdib F(2023)Exploiting Synergies between Augmented Reality and RFIDs for Item Localization and Retrieval2023 IEEE International Conference on RFID (RFID)10.1109/RFID58307.2023.10178452(30-35)Online publication date: 13-Jun-2023
https://doi.org/10.1109/RFID58307.2023.10178452
Show More Cited By

Index Terms

Tell me what i see: recognize RFID tagged objects in augmented reality systems
1. Human-centered computing

Recommendations

Interactive representation of virtual object in hand-held box by finger-worn haptic display
HAPTICS '08: Proceedings of the 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems

To deliver a realistic presence of virtual objects with a simple haptic display in an augmented reality system, we have developed a wearable haptic display to present the sensation of weight and inertial force of the virtual objects. In this study, we ...
A new unpredictability-based radio frequency identification forward privacy model and a provably secure construction

The privacy model of radio frequency identification RFID systems is for formalizing the adversarial capabilities and the security requirements of RFID anonymity and untraceability. Existing unpredictability-based privacy models such as unp-privacy, eunp-...
On two RFID privacy notions and their relations

Privacy of RFID systems is receiving increasing attention in the RFID community. Basically, there are two kinds of RFID privacy notions in the literature: one based on the indistinguishability of two tags, denoted as ind-privacy, and the other based on ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 2016

1288 pages

ISBN:9781450344616

DOI:10.1145/2971648

General Chairs:
Paul Lukowicz
DFKI and University of Kaiserslautern
,
Antonio Krüger
DFKI and Saarland University
,
Program Chairs:
Andreas Bulling
Max Planck Institute for Informatics
,
Youn-Kyung Lim
KAIST
,
Shwetak N. Patel
University of Washington

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 September 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
EU FP7 IRSES MobileCloud Project
Natural Science Foundation of Jiangsu Province
NSF grants

Conference

UbiComp '16

Sponsor:

UbiComp '16: The 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 12 - 16, 2016

Germany, Heidelberg

Acceptance Rates

UbiComp '16 Paper Acceptance Rate 101 of 389 submissions, 26%;

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

42
Total Citations
View Citations
719
Total Downloads

Downloads (Last 12 months)36
Downloads (Last 6 weeks)5

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tavanti ENepa PGabbrielli RPirozzi M(2025)Review on Systems Combining Computer Vision and Radio Frequency IdentificationIEEE Internet of Things Journal10.1109/JIOT.2024.348475512:2(1291-1319)Online publication date: 15-Jan-2025
https://doi.org/10.1109/JIOT.2024.3484755
Shang FYang PXiong JFeng YLi X(2023)Tamera: Contactless Commodity Tracking, Material and Shopping Behavior Recognition Using COTS RFIDsACM Transactions on Sensor Networks10.1145/356377719:2(1-24)Online publication date: 3-Feb-2023
https://dl.acm.org/doi/10.1145/3563777
Boroushaki TLam MChen WDodds LEid AAdib F(2023)Exploiting Synergies between Augmented Reality and RFIDs for Item Localization and Retrieval2023 IEEE International Conference on RFID (RFID)10.1109/RFID58307.2023.10178452(30-35)Online publication date: 13-Jun-2023
https://doi.org/10.1109/RFID58307.2023.10178452
Dai FLi FZhao LMa F(2023)Intelligent Warehouse Material Detection Based on RFID and Deep Learning2023 International Conference on Networks, Communications and Intelligent Computing (NCIC)10.1109/NCIC61838.2023.00032(157-161)Online publication date: 17-Nov-2023
https://doi.org/10.1109/NCIC61838.2023.00032
Lopez-De-Teruel PRuiz AGarcia-Clemente F(2023)Computer Vision Based Auto-ID for Optimizing Logistics Operations2023 IEEE International Conference on Omni-layer Intelligent Systems (COINS)10.1109/COINS57856.2023.10189304(1-6)Online publication date: 23-Jul-2023
https://doi.org/10.1109/COINS57856.2023.10189304
Feng YHuang WWang SZhang YJiang SCao Z(2023)Anti-Clone: A Lightweight Approach for RFID Cloning Attacks DetectionCollaborative Computing: Networking, Applications and Worksharing10.1007/978-3-031-24386-8_5(75-90)Online publication date: 25-Jan-2023
https://doi.org/10.1007/978-3-031-24386-8_5
Ning JXie LWang CBu YXiao FYe BLu S(2022)Revolving Scanning on Tagged Objects: 3D Structure Detection of Logistics Packages via RFID SystemsACM Transactions on Sensor Networks10.1145/349017118:2(1-29)Online publication date: 16-Mar-2022
https://dl.acm.org/doi/10.1145/3490171
Zhang SMa ZYang CKui XLiu XWang WWang JGuo S(2022)Real-time and Accurate Gesture Recognition with Commercial RFID DevicesIEEE Transactions on Mobile Computing10.1109/TMC.2022.3211324(1-16)Online publication date: 2022
https://doi.org/10.1109/TMC.2022.3211324
Zhang SMa ZLu KLiu XLiu JGuo SZomaya AZhang JWang J(2022)HearMe: Accurate and Real-time Lip Reading based on Commercial RFID DevicesIEEE Transactions on Mobile Computing10.1109/TMC.2022.3208019(1-14)Online publication date: 2022
https://doi.org/10.1109/TMC.2022.3208019
Ding HZhao CWang GZhao KXi WZhao J(2022)Utilizing Tag Interference for Refined Localization of Passive RFIDIEEE Internet of Things Journal10.1109/JIOT.2021.31379689:14(12656-12672)Online publication date: 15-Jul-2022
https://doi.org/10.1109/JIOT.2021.3137968
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten