research-article

Tasking networked CCTV cameras and mobile phones to identify and localize multiple people

Authors:

Thiago Teixeira,

Andreas SavvidesAuthors Info & Claims

UbiComp '10: Proceedings of the 12th ACM international conference on Ubiquitous computing

Pages 213 - 222

https://doi.org/10.1145/1864349.1864367

Published: 26 September 2010 Publication History

Abstract

We present a method to identify and localize people by leveraging existing CCTV camera infrastructure along with inertial sensors (accelerometer and magnetometer) within each person's mobile phones. Since a person's motion path, as observed by the camera, must match the local motion measurements from their phone, we are able to uniquely identify people with the phones' IDs by detecting the statistical dependence between the phone and camera measurements. For this, we express the problem as consisting of a two-measurement HMM for each person, with one camera measurement and one phone measurement. Then we use a maximum a posteriori formulation to find the most likely ID assignments. Through sensor fusion, our method largely bypasses the motion correspondence problem from computer vision and is able to track people across large spatial or temporal gaps in sensing. We evaluate the system through simulations and experiments in a real camera network testbed.

References

[1]

}}N. Anjum and A. Cavallaro. Multi-camera calibration and global trajectory fusion. In Y. Ma and G. Qian, editors, Intelligent Video Surveillance: Systems and Technologies. CRC Press, 2009.

[2]

}}Y. Bar-Shalom and W. D. Blair, editors. Multitarget-Multisensor Tracking. Artech House Publishers, 1990.

[3]

}}K. Bernardin, A. Elbs, and R. Stiefelhagen. Multiple object tracking performance metrics and evaluation in a smart room environment. In IEEE VS, 2006.

[4]

}}J. E. A. Bertram and A. Ruina. Multiple walking speed-frequency relations are predicted by constrained optimization. Journal of Theoretical Biology, 209(4):445 -- 453, 2001.

[5]

}}D. C. Brogan and N. L. Johnson. Realistic human walking paths. In IEEE CASA, 200.

Digital Library

[6]

}}O. Cappe, E. Moulines, and T. Ryden. Inference in hidden Markov models. Springer Verlag, 2005.

Digital Library

[7]

}}J. Connell, A. Senior, A. Hampapur, Y. Tian, L. Brown, and S. Pankanti. Detection and tracking in the IBM PeopleVision system. In IEEE ICME, 2004.

[8]

}}N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In IEEE CVPR, 2005.

Digital Library

[9]

}}D. Devarajan and R. Radke. Distributed metric calibration for large-scale camera networks. In BASENETS, 2004.

[10]

}}N. Fisher. Statistical analysis of circular data. Cambridge Univ Pr, 1996.

[11]

}}E. Foxlin. Pedestrian tracking with shoe-mounted inertial sensors. IEEE Computer Graphics and Applications, 25(6):38--46, 2005.

Digital Library

[12]

}}M. Kourogi and T. Kurata. Personal positioning based on walking locomotion analysis with self-contained sensors and a wearable camera. IEEE/ACM ISMAR, 2003.

Digital Library

[13]

}}H. W. Kuhn. The hungarian method for the assignment problem. In Naval Research Logistic Quarterly, volume 52, 1955.

[14]

}}B. Kusy, A. Ledeczi, and X. Koutsoukos. Tracking mobile nodes using rf doppler shifts. In ACM SenSys, 2007. ACM.

Digital Library

[15]

}}P. Lichtsteiner, J. Kramer, and T. Delbruck. Improved on/off temporally differentiating address-event imager. In IEEE ICECS, 2004.

[16]

}}P. Lichtsteiner, C. Posch, and T. Delbruck. A 128x128 120dB 15us latency asynchronous temporal contrast vision sensor. IEEE Journal of Solid State Circuits, 43(2):566--576, 2008.

[17]

}}D. G. Lowe. Distinctive image features from scale-invariant keypoints. volume 60, pages 91--110, 2004.

Digital Library

[18]

}}D. Lymberopoulos, Q. Lindsey, and A. Savvides. An Empirical Analysis of Radio Signal Strength Variability in IEEE 802.15.4 Networks using Monopole Antennas. In Lecture Notes in Computer Science, volume 3868, page 326, 2006.

Digital Library

[19]

}}D. Lymberopoulos, T. Teixeira, and A. Savvides. Macroscopic human behavior interpretation using distributed sensor networks. Proceedings of IEEE, October 2008.

[20]

}}W. Mayol, A. Davison, B. Tordoff, and D. Murray. Applying active vision and slam to wearables. In IEEE ISRR, 2003.

[21]

}}T. Murakita, T. Ikeda, and H. Ishiguro. Human tracking using floor sensors based on the Markov chain Monte Carlo method. In IEEE ICPR, 2004.

Digital Library

[22]

}}A. Savvides, C. Han, and M. B. Srivastava. Dynamic fine grained localization in ad-hoc sensor networks. In ACM Mobicom 2001, 2001.

Digital Library

[23]

}}E. Schoonderwaldt, N. Rasamimanana, and F. Bevilacqua. Combining accelerometer and video camera: Reconstruction of bow velocity profiles. In NIME, 2006.

Digital Library

[24]

}}D. Schulz, D. Fox, and J. Hightower. People tracking with anonymous and id-sensors using rao-blackwellised particle filters. In IJCAI, 2003.

Digital Library

[25]

}}M. Shankar, J. Burchett, Q. Hao, B. Guenther, D. Brady, et al. Human-tracking systems using pyroelectric infrared detectors. Optical Engineering, 45:106401, 2006.

[26]

}}N. Shrivastava, R. Madhow, and S. Suri. Target tracking with binary proximity sensors: fundamental limits, minimal descriptions, and algorithms. In ACM SenSys, 2006.

Digital Library

[27]

}}L. D. Stone, C. A. Barlow, and T. L. Corwin. Bayesian Multiple Target Tracking. Artech House Publishers, 1999.

Digital Library

[28]

}}M. Taj, E. Maggio, and A. Cavallaro. Multi-feature graph-based object tracking. In CLEAR, pages 190--199, 2006.

Digital Library

[29]

}}Y. Tao, H. Hu, and H. Zhou. Integration of vision and inertial sensors for 3D arm motion tracking in home-based rehabilitation. The International Journal of Robotics Research, 26(6):607, 2007.

Digital Library

[30]

}}T. Teixeira, D. Jung, G. Dublon, and A. Savvides. Identifying people by gait-matching using cameras and wearable accelerometers. In ACM/IEEE ICDSC, 2009.

[31]

}}T. Teixeira, D. Jung, G. Dublon, and A. Savvides. Identifying people in camera networks using wearable accelerometers. In ACM PETRA, 2009.

Digital Library

[32]

}}T. Teixeira and A. Savvides. Lightweight people counting and localizing in indoor spaces using camera sensor nodes. In ACM/IEEE (ICDSC), 2007.

[33]

}}C. J. Veenman, M. Reinders, and E. Backer. Resolving motion correspondence for densely moving points. In IEEE Transactions on Pattern Analysis and Machine Intelligence, March.

Digital Library

[34]

}}J. Vermaak, S. Godsill, and P. Perez. Monte Carlo filtering for multi-target tracking and data association. IEEE Transactions on Aerospace and Electronic systems, 41(1):309--332, 2005.

[35]

}}W. T. Willis, K. J. Ganley, and R. M. Herman. Fuel oxidation during human walking. Metabolism - Clinical and Experimental, June 2005.

[36]

}}J. Wilson and N. Patwari. Radio tomographic imaging with wireless networks. IEEE Trans. Mobile Computing, 2009.

Digital Library

[37]

}}Z. Xiang, S. Song, J. Chen, H. Wang, J. Huang, and X. Gao. A wireless lan-based indoor positioning technology. IBM J. Res. Dev., 48(5/6):617--626, 2004.

Digital Library

Cited By

Sharma RKowsigan MKalra EGupta A(2024)Real time people detection on entrance gate using signal processing2024 5th International Conference on Recent Trends in Computer Science and Technology (ICRTCST)10.1109/ICRTCST61793.2024.10578470(453-458)Online publication date: 9-Apr-2024
https://doi.org/10.1109/ICRTCST61793.2024.10578470
Truong HJaisinghani DJain SSinha AKo JBalan R(2024)Tracking people across ultra populated indoor spaces by matching unreliable Wi-Fi signals with disconnected video feedsPervasive and Mobile Computing10.1016/j.pmcj.2023.10186097(101860)Online publication date: Jan-2024
https://doi.org/10.1016/j.pmcj.2023.101860
Chen HMunir SLin S(2022)RFCamProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35345886:2(1-29)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3534588
Show More Cited By

Index Terms

Tasking networked CCTV cameras and mobile phones to identify and localize multiple people
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Tracking
      2. Computer vision tasks
        Scene understanding
        Vision for robotics

Recommendations

A General Framework for Tracking Multiple People from a Moving Camera

In this paper, we present a general framework for tracking multiple, possibly interacting, people from a mobile vision platform. To determine all of the trajectories robustly and in a 3D coordinate system, we estimate both the camera's ego-motion and ...
Calibration of multiple fish‐eye cameras using a wand

Fish‐eye cameras are becoming increasingly popular in computer vision, but their use for three‐dimensional measurement is limited partly because of the lack of an accurate, efficient and user‐friendly calibration procedure. For such a purpose, the authors ...
Principal Axis-Based Correspondence between Multiple Cameras for People Tracking

Visual surveillance using multiple cameras has attracted increasing interest in recent years. Correspondence between multiple cameras is one of the most important and basic problems which visual surveillance using multiple cameras brings. In this paper, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp '10: Proceedings of the 12th ACM international conference on Ubiquitous computing

September 2010

366 pages

ISBN:9781605588438

DOI:10.1145/1864349

General Chairs:
Jakob E. Bardram
IT University of Copenhagen, Denmark
,
Marc Langheinrich
University of Lugano, Switzerland
,
Program Chairs:
Khai N. Truong
University of Toronto, Canada
,
Paddy Nixon
University College Dublin, Ireland

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

University of Florida: University of Florida

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 September 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

Ubicomp '10

Sponsor:

Ubicomp '10: The 2010 ACM Conference on Ubiquitous Computing

September 26 - 29, 2010

Copenhagen, Denmark

Acceptance Rates

UbiComp '10 Paper Acceptance Rate 39 of 202 submissions, 19%;

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

53
Total Citations
View Citations
843
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sharma RKowsigan MKalra EGupta A(2024)Real time people detection on entrance gate using signal processing2024 5th International Conference on Recent Trends in Computer Science and Technology (ICRTCST)10.1109/ICRTCST61793.2024.10578470(453-458)Online publication date: 9-Apr-2024
https://doi.org/10.1109/ICRTCST61793.2024.10578470
Truong HJaisinghani DJain SSinha AKo JBalan R(2024)Tracking people across ultra populated indoor spaces by matching unreliable Wi-Fi signals with disconnected video feedsPervasive and Mobile Computing10.1016/j.pmcj.2023.10186097(101860)Online publication date: Jan-2024
https://doi.org/10.1016/j.pmcj.2023.101860
Chen HMunir SLin S(2022)RFCamProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35345886:2(1-29)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3534588
Fagert JMirshekari MZhang PNoh H(2022)Recursive Sparse Representation for Identifying Multiple Concurrent Occupants Using Floor Vibration SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35172296:1(1-33)Online publication date: 29-Mar-2022
https://dl.acm.org/doi/10.1145/3517229
Liu HAlali AIbrahim MCao BMeegan NLi HGruteser MJain SDana KAshok ACheng BLu H(2022)Vi-Fi: Associating Moving Subjects across Vision and Wireless Sensors2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)10.1109/IPSN54338.2022.00024(208-219)Online publication date: May-2022
https://doi.org/10.1109/IPSN54338.2022.00024
Li XLi YLi NGirard AKolmanovsky I(2022)Set‐theoretic localization for mobile robots with infrastructure‐based sensingAdvanced Control for Applications10.1002/adc2.1175:1Online publication date: 21-Nov-2022
https://doi.org/10.1002/adc2.117
Zhou YHua YLiu J(2021)Study workplace space occupancy: a review of measures and technologiesJournal of Facilities Management10.1108/JFM-01-2021-001320:3(350-368)Online publication date: 29-Jul-2021
https://doi.org/10.1108/JFM-01-2021-0013
Van LZhang LChang CTong KWu KTseng Y(2021)Things in the air: tagging wearable IoT information on drone videosDiscover Internet of Things10.1007/s43926-021-00005-81:1Online publication date: 24-Feb-2021
https://doi.org/10.1007/s43926-021-00005-8
Minoli DDressendofer J(2021)De‐densification of Spaces and Work EnvironmentsHigh‐Density and De‐Densified Smart Campus Communications10.1002/9781119716075.ch6(184-221)Online publication date: 3-Dec-2021
https://doi.org/10.1002/9781119716075.ch6
Masullo ABurghardt TDamen DPerrett TMirmehdi M(2020)Person Re-ID by Fusion of Video Silhouettes and Wearable Signals for Home Monitoring ApplicationsSensors10.3390/s2009257620:9(2576)Online publication date: 1-May-2020
https://doi.org/10.3390/s20092576
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents