research-article

Occupancy Sensing and Activity Recognition with Cameras and Wireless Sensors

Authors:

Ming-Ching ChangAuthors Info & Claims

DATA'19: Proceedings of the 2nd Workshop on Data Acquisition To Analysis

Pages 1 - 6

https://doi.org/10.1145/3359427.3361911

Published: 10 November 2019 Publication History

Abstract

We present a system work combining visual cameras and wireless sensors for human occupancy detection and activity recognition. We describe our testbed system, data collected from a human subject study, observations from long-term occupancy experiments, and preliminary analytical results. We apply machine learning algorithms to the human activity recognition data, and identify challenges in applying the state-of-the-art deep learning techniques to wireless sensing of human activity. We find that packet loss due to wireless interference has a significant effect on time series classification. We also find that the convolutional neural networks significantly outperforms the conventional support vector machine method, but further experiments need to be performed to investigate environment-independent classification and the overfitting issue. Finally, we discuss future research topics that can use our testbed of wireless sensors and visual cameras to automate data labeling in deep learning model training.

References

[1]

Ming-Ching Chang, Ting Yu, Jiajia Luo, Kun Duan, Peter Tu, Yang Zhao, Nandini Nagraj, Vrinda Rajiv, Michael Priebe, Elena A Wood, et al. 2017. Multimodal sensor system for pressure ulcer wound assessment and care. IEEE Transactions on Industrial Informatics 14, 3 (2017), 1186--1196.

[2]

Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. 2018. Recurrent Neural Networks for Multivariate Time Series with Missing Values. Scientific Reports 8, 1 (2018), 6085. https://doi.org/10.1038/s41598-018-24271-9

[3]

Jixu Chen, Ming-Ching Chang, Tai-Peng Tian, Ting Yu, and Peter Tu. 2015. Bridging computer vision and social science: A multi-camera vision system for social interaction training analysis. In 2015 IEEE International Conference on Image Processing (ICIP). IEEE, 823--826.

[4]

Wenjun Jiang, Chenglin Miao, Fenglong Ma, Shuochao Yao, Yaqing Wang, Ye Yuan, Hongfei Xue, Chen Song, Xin Ma, Dimitrios Koutsonikolas, Wenyao Xu, and Lu Su. 2018. Towards Environment Independent Device Free Human Activity Recognition. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking (MobiCom '18). ACM, New York, NY, USA, 289--304. https://doi.org/10.1145/3241539.3241548

Digital Library

[5]

Christopher Donald Johnson, Brandon Stephen Good, Andrew Phelps Day, David S Toledano, Yang Zhao, and Jeffrey Richardson Terry. 2017. Methods and systems to sense situational awareness with a dual doppler and control for optimized operations. Patent Application No. 14885169.

[6]

Oscar D Lara and Miguel A Labrador. 2012. A survey on human activity recognition using wearable sensors. IEEE communications surveys & tutorials 15, 3 (2012), 1192--1209.

[7]

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521, 7553 (27 5 2015), 436--444. https://doi.org/10.1038/nature14539

[8]

Yongsen Ma, Gang Zhou, and Shuangquan Wang. 2019. WiFi Sensing with Channel State Information: A Survey. ACM Comput. Surv. 52, 3, Article 46 (June 2019), 36 pages. https://doi.org/10.1145/3310194

[9]

Neal Patwari and Joey Wilson. 2010. RF sensor networks for device-free localization: Measurements, models, and algorithms. Proc. IEEE 98, 11 (2010), 1961--1973.

[10]

Neal Patwari, Joey Wilson, Sai Ananthanarayanan, Sneha K Kasera, and Dwayne R Westenskow. 2013. Monitoring breathing via signal strength in wireless networks. IEEE Transactions on Mobile Computing 13, 8 (2013), 1774--1786.

[11]

Dairazalia Sánchez, Monica Tentori, and Jesús Favela. 2008. Activity recognition for the smart hospital. IEEE intelligent systems 23, 2 (2008), 50--57.

[12]

Bex George Thomas, Srinivas Bollapragada, Kunter Akbay, David Toledano, Peter Katlic, Onur Dulgeroglu, and Dan Yang. 2013. Automated Bed Assignments in a Complex and Dynamic Hospital Environment. Interfaces 43, 5 (2013), 435--448. http://www.jstor.org/stable/43699330

Digital Library

[13]

Bo Wei, Wen Hu, Mingrui Yang, and Chun Tung Chou. 2015. Radio-based Device-free Activity Recognition with Radio Frequency Interference. In Proceedings of the 14th International Conference on Information Processing in Sensor Networks (IPSN '15). ACM, New York, NY, USA, 154--165. https://doi.org/10.1145/2737095.2737117

Digital Library

[14]

Mingmin Zhao, Tianhong Li, Mohammad Abu Alsheikh, Yonglong Tian, Hang Zhao, Antonio Torralba, and Dina Katabi. 2018. Through-Wall Human Pose Estimation Using Radio Signals. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]

Yang Zhao, Zhaoyuan Yang, and Weizhong Yan. 2019. Adversarial Attack on Time Series Classification in Human Activity Sensing. Technical Report 2019GRC0211. GE Research, General Electric, Niskayuna, NY.

[16]

Yang Zhao, Ting Yu, and Jeff Ashe. 2015. Poster: Non-invasive Human Activity Monitoring Using a Low-cost Doppler Sensor and an RF Link. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems (SenSys '15). ACM, New York, NY, USA, 397--398. https://doi.org/10.1145/2809695.2817882

Digital Library

Cited By

Song YPitafi ZDou FSun JZhang XPhillips BSong W(2024)Self-Supervised Representation Learning and Temporal-Spectral Feature Fusion for Bed Occupancy DetectionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785148:3(1-25)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678514
Das HLin YAgwan USpangher LDevonport AYang YDrgoňa JChong ASchiavon SSpanos C(2024)Machine Learning for Smart and Energy-Efficient BuildingsEnvironmental Data Science10.1017/eds.2023.433Online publication date: 4-Jan-2024
https://doi.org/10.1017/eds.2023.43
Ho Kim SJun Moon H(2024)A detailed occupant activity classification model in a residential environment using building monitoring data: Considering occupant characteristicsEnergy and Buildings10.1016/j.enbuild.2023.113867305(113867)Online publication date: Feb-2024
https://doi.org/10.1016/j.enbuild.2023.113867
Show More Cited By

Index Terms

Occupancy Sensing and Activity Recognition with Cameras and Wireless Sensors

Recommendations

Activity recognition with hand-worn magnetic sensors

Activity recognition is a key technology for realizing ambient assisted living applications such as care of the elderly and home automation. This paper proposes a new activity recognition method that employs hand-worn magnetic sensors to recognize a ...
Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks
MM '15: Proceedings of the 23rd ACM international conference on Multimedia

Human physical activity recognition based on wearable sensors has applications relevant to our daily life such as healthcare. How to achieve high recognition accuracy with low computational cost is an important issue in the ubiquitous computing. Rather ...
Transferring activity recognition models for new wearable sensors with deep generative domain adaptation
IPSN '19: Proceedings of the 18th International Conference on Information Processing in Sensor Networks

Wearable sensors provide enormous opportunities to identify activities and events of interest for various applications. However, a major limitation of the current systems is the fact that machine learning algorithms trained on particular sensors need to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

DATA'19: Proceedings of the 2nd Workshop on Data Acquisition To Analysis

November 2019

71 pages

ISBN:9781450369930

DOI:10.1145/3359427

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Department of Veterans Affairs

Conference

SenSys '19

Sponsor:

SenSys '19: The 17th ACM Conference on Embedded Networked Sensor Systems

November 10, 2019

NY, New York, USA

Acceptance Rates

DATA'19 Paper Acceptance Rate 16 of 21 submissions, 76%;

Overall Acceptance Rate 74 of 167 submissions, 44%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
362
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)1

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Song YPitafi ZDou FSun JZhang XPhillips BSong W(2024)Self-Supervised Representation Learning and Temporal-Spectral Feature Fusion for Bed Occupancy DetectionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785148:3(1-25)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678514
Das HLin YAgwan USpangher LDevonport AYang YDrgoňa JChong ASchiavon SSpanos C(2024)Machine Learning for Smart and Energy-Efficient BuildingsEnvironmental Data Science10.1017/eds.2023.433Online publication date: 4-Jan-2024
https://doi.org/10.1017/eds.2023.43
Ho Kim SJun Moon H(2024)A detailed occupant activity classification model in a residential environment using building monitoring data: Considering occupant characteristicsEnergy and Buildings10.1016/j.enbuild.2023.113867305(113867)Online publication date: Feb-2024
https://doi.org/10.1016/j.enbuild.2023.113867
Alsafery WRana OPerera C(2023)Sensing within Smart Buildings: A SurveyACM Computing Surveys10.1145/359660055:13s(1-35)Online publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1145/3596600
Suresha PHegde CJiang ZClifford G(2022)An Edge Computing and Ambient Data Capture System for Clinical and Home EnvironmentsSensors10.3390/s2207251122:7(2511)Online publication date: 25-Mar-2022
https://doi.org/10.3390/s22072511
Fang SMunir SNirjon S(2020)Person tracking and identification using cameras and wi-fi channel state information (CSI) from smartphonesProceedings of the Third Workshop on Data: Acquisition To Analysis10.1145/3419016.3431488(26-30)Online publication date: 16-Nov-2020
https://dl.acm.org/doi/10.1145/3419016.3431488

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents