research-article

Spatial-Temporal Masked Autoencoder for Multi-Device Wearable Human Activity Recognition

Authors:

Shenghuan Miao,

Rong HuAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 7, Issue 4

Article No.: 172, Pages 1 - 25

https://doi.org/10.1145/3631415

Published: 12 January 2024 Publication History

Abstract

The widespread adoption of wearable devices has led to a surge in the development of multi-device wearable human activity recognition (WHAR) systems. Nevertheless, the performance of traditional supervised learning-based methods to WHAR is limited by the challenge of collecting ample annotated wearable data. To overcome this limitation, self-supervised learning (SSL) has emerged as a promising solution by first training a competent feature extractor on a substantial quantity of unlabeled data, followed by refining a minimal classifier with a small amount of labeled data. Despite the promise of SSL in WHAR, the majority of studies have not considered missing device scenarios in multi-device WHAR. To bridge this gap, we propose a multi-device SSL WHAR method termed Spatial-Temporal Masked Autoencoder (STMAE). STMAE captures discriminative activity representations by utilizing the asymmetrical encoder-decoder structure and two-stage spatial-temporal masking strategy, which can exploit the spatial-temporal correlations in multi-device data to improve the performance of SSL WHAR, especially on missing device scenarios. Experiments on four real-world datasets demonstrate the efficacy of STMAE in various practical scenarios.

References

[1]

Alireza Abedin, Mahsa Ehsanpour, Qinfeng Shi, Hamid Rezatofighi, and Damith C Ranasinghe. 2021. Attend and discriminate: Beyond the state-of-the-art for human activity recognition using wearable sensors. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 1 (2021), 1--22.

Digital Library

[2]

Karan Ahuja, Eyal Ofek, Mar Gonzalez-Franco, Christian Holz, and Andrew D Wilson. 2021. Coolmoves: User motion accentuation in virtual reality. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1--23.

Digital Library

[3]

Roman Bachmann, David Mizrahi, Andrei Atanov, and Amir Zamir. 2022. MultiMAE: Multi-modal multi-task masked autoencoders. In Computer Vision--ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part XXXVII. Springer, 348--367.

Digital Library

[4]

Shaojie Bai, J Zico Kolter, and Vladlen Koltun. 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018).

[5]

Pierre Baldi. 2012. Autoencoders, unsupervised learning, and deep architectures. In Proceedings of ICML Workshop on Unsupervised and Transfer Learning. 37--49.

Digital Library

[6]

Oresti Baños, Miguel Damas, Héctor Pomares, Ignacio Rojas, Máté Attila Tóth, and Oliver Amft. 2012. A benchmark dataset to evaluate sensor displacement in activity recognition. In Proceedings of the ACM Conference on Ubiquitous Computing. 1026--1035.

[7]

Abigail Bartolome, Sahaj Shah, and Temiloluwa Prioleau. 2021. GlucoMine: A case for improving the use of wearable device data in diabetes management. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 3 (2021), 1--24.

Digital Library

[8]

Kaixuan Chen, Dalin Zhang, Lina Yao, Bin Guo, Zhiwen Yu, and Yunhao Liu. 2021. Deep learning for sensor-based human activity recognition: Overview, challenges, and opportunities. Computing Surveys 54, 4 (2021), 1--40.

Digital Library

[9]

Ling Chen, Yi Zhang, Shenghuan Miao, Sirou Zhu, Rong Hu, Liangying Peng, and Mingqi Lv. 2022. SALIENCE: An unsupervised user adaptation model for multiple wearable sensors based human activity recognition. IEEE Transactions on Mobile Computing (2022), 1--12.

[10]

Ling Chen, Yi Zhang, and Liangying Peng. 2020. METIER: A deep multi-task learning based activity and user recognition model using wearable sensors. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (2020), 1--18.

Digital Library

[11]

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Proceedings of the International Conference on Machine Learning. 1597--1607.

[12]

Xianda Chen, Yifei Xiao, Yeming Tang, Julio Fernandez-Mendoza, and Guohong Cao. 2021. ApneaDetector: Detecting sleep apnea with smartwatches. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1--22.

Digital Library

[13]

Shohreh Deldari, Hao Xue, Aaqib Saeed, Jiayuan He, Daniel V Smith, and Flora D Salim. 2022. Beyond just vision: A review on self-supervised representation learning on multimodal and temporal data. arXiv preprint arXiv:2206.02353 (2022).

[14]

Shohreh Deldari, Hao Xue, Aaqib Saeed, Daniel V Smith, and Flora D Salim. 2022. COCOA: Cross modality contrastive learning for sensor data. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (2022), 1--28.

Digital Library

[15]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).

[16]

Elena Di Lascio, Shkurta Gashi, Juan Sebastian Hidalgo, Beatrice Nale, Maike E Debus, and Silvia Santini. 2020. A multi-sensor approach to automatically recognize breaks and work activities of knowledge workers in academia. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 3 (2020), 1--20.

Digital Library

[17]

Aiden Doherty, Dan Jackson, Nils Hammerla, Thomas Plötz, Patrick Olivier, Malcolm H Granat, Tom White, Vincent T Van Hees, Michael I Trenell, Christoper G Owen, et al. 2017. Large scale population assessment of physical activity using wrist worn accelerometers: the UK biobank study. PloS one 12, 2 (2017), e0169649.

[18]

Christoph Feichtenhofer, Yanghao Li, Kaiming He, et al. 2022. Masked autoencoders as spatiotemporal learners. Advances in neural information processing systems 35 (2022), 35946--35958.

[19]

Berihun Fekade, Taras Maksymyuk, Maryan Kyryk, and Minho Jo. 2017. Probabilistic recovery of incomplete sensed data in IoT. IEEE Internet of Things Journal 5, 4 (2017), 2282--2292.

[20]

Nan Gao, Mohammad Saiedur Rahaman, Wei Shao, Kaixin Ji, and Flora D Salim. 2022. Individual and group-wise classroom seating experience: Effects on student engagement in different courses. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (2022), 1--23.

Digital Library

[21]

Nan Gao, Wei Shao, Mohammad Saiedur Rahaman, and Flora D Salim. 2020. N-gage: Predicting in-class emotional, behavioural and cognitive engagement in the wild. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 3 (2020), 1--26.

Digital Library

[22]

Jian Gong, Xinyu Zhang, Yuanjun Huang, Ju Ren, and Yaoxue Zhang. 2021. Robust inertial motion tracking through deep sensor fusion across smart earbuds and smartphone. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1--26.

Digital Library

[23]

Fuqiang Gu, Mu-Huan Chung, Mark Chignell, Shahrokh Valaee, Baoding Zhou, and Xue Liu. 2021. A survey on deep learning for human activity recognition. Computing Surveys 54, 8 (2021), 1--34.

[24]

Harish Haresamudram, David V Anderson, and Thomas Plötz. 2019. On the role of features in human activity recognition. In Proceedings of the International Symposium on Wearable Computers. 78--88.

Digital Library

[25]

Harish Haresamudram, Apoorva Beedu, Varun Agrawal, Patrick L Grady, Irfan Essa, Judy Hoffman, and Thomas Plötz. 2020. Masked reconstruction based self-supervision for human activity recognition. In Proceedings of the International Symposium on Wearable Computers. 45--49.

Digital Library

[26]

Harish Haresamudram, Irfan Essa, and Thomas Plötz. 2021. Contrastive predictive coding for human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1--26.

Digital Library

[27]

Harish Haresamudram, Irfan Essa, and Thomas Plötz. 2022. Assessing the state of self-supervised human activity recognition using wearables. arXiv preprint arXiv:2202.12938 (2022).

[28]

Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. 2022. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16000--16009.

[29]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735--1780.

Digital Library

[30]

Rong Hu, Ling Chen, Shenghuan Miao, and Xing Tang. 2023. SWL-Adapt: An unsupervised domain adaptation model with sample weight learning for cross-user wearable human activity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 6012--6020.

Digital Library

[31]

Po-Yao Huang, Hu Xu, Juncheng Li, Alexei Baevski, Michael Auli, Wojciech Galuba, Florian Metze, and Christoph Feichtenhofer. 2022. Masked autoencoders that listen. Advances in Neural Information Processing Systems 35 (2022), 28708--28720.

[32]

Xiaohu Huang, Hao Zhou, Bin Feng, Xinggang Wang, Wenyu Liu, Jian Wang, Haocheng Feng, Junyu Han, Errui Ding, and Jingdong Wang. 2023. Graph contrastive learning for skeleton-based action recognition. arXiv preprint arXiv:2301.10900 (2023).

[33]

Farhad Imani, Changqing Cheng, Ruimin Chen, and Hui Yang. 2018. Nested gaussian process modeling for high-dimensional data imputation in healthcare systems. In Proceedings of the IIE Annual Conference. 1312--1317.

[34]

Yash Jain, Chi Ian Tang, Chulhong Min, Fahim Kawsar, and Akhil Mathur. 2022. ColloSSL: Collaborative self-supervised learning for human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 1 (2022), 1--28.

Digital Library

[35]

Natasha Jaques, Sara Taylor, Akane Sano, and Rosalind Picard. 2017. Multimodal autoencoder: A deep learning approach to filling in missing sensor data and enabling better mood prediction. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction. 202--208.

[36]

Hong Jia, Jiawei Hu, and Wen Hu. 2021. SwingNet: Ubiquitous fine-grained swing tracking framework via stochastic neural architecture search and adversarial learning. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 3 (2021), 1--21.

Digital Library

[37]

Xiao Jiang, Zean Tian, and Kenli Li. 2021. A graph-based approach for missing sensor data imputation. IEEE Sensors Journal 21, 20 (2021), 23133--23144.

[38]

Woosub Jung, Amanda Watson, Scott Kuehn, Erik Korem, Ken Koltermann, Minglong Sun, Shuangquan Wang, Zhenming Liu, and Gang Zhou. 2021. LAX-Score: Quantifying team performance in lacrosse and exploring IMU features towards performance enhancement. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 3 (2021), 1--28.

Digital Library

[39]

Hua Kang, Qianyi Huang, and Qian Zhang. 2022. Augmented adversarial learning for human activity recognition with partial sensor sets. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (2022), 1--30.

Digital Library

[40]

Bulat Khaertdinov, Esam Ghaleb, and Stylianos Asteriadis. 2021. Contrastive self-supervised learning for sensor-based human activity recognition. In Proceedings of the International Joint Conference on Biometrics. 1--8.

Digital Library

[41]

Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations. 1--15.

[42]

Shengzhong Liu, Shuochao Yao, Yifei Huang, Dongxin Liu, Huajie Shao, Yiran Zhao, Jinyang Li, Tianshi Wang, Ruijie Wang, Chaoqi Yang, et al. 2020. Handling missing sensors in topology-aware iot applications with gated graph neural network. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 3 (2020), 1--31.

Digital Library

[43]

Shengzhong Liu, Shuochao Yao, Jinyang Li, Dongxin Liu, Tianshi Wang, Huajie Shao, and Tarek Abdelzaher. 2020. GlobalFusion: A global attentional deep learning framework for multisensor information fusion. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (2020), 1--27.

Digital Library

[44]

Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. 2021. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In Proceedings of the International Conference on Learning Representations.

[45]

Yang Liu, Zhenjiang Li, Zhidan Liu, and Kaishun Wu. 2019. Real-time arm skeleton tracking and gesture inference tolerant to missing wearable sensors. In Proceedings of the Annual International Conference on Mobile Systems, Applications, and Services. 287--299.

Digital Library

[46]

Haojie Ma, Wenzhong Li, Xiao Zhang, Songcheng Gao, and Sanglu Lu. 2019. AttnSense: Multi-level attention mechanism for multimodal human activity recognition. In Proceedings of the International Joint Conference on Artificial Intelligence. 3109--3115.

[47]

Qianwen Meng, Hangwei Qian, Yong Liu, Yonghui Xu, Zhiqi Shen, and Lizhen Cui. 2022. MHCCL: Masked Hierarchical Cluster-wise Contrastive Learning for Multivariate Time Series. arXiv preprint arXiv:2212.01141 (2022).

[48]

Shenghuan Miao, Ling Chen, Rong Hu, and Yingsong Luo. 2022. Towards a dynamic inter-sensor correlations learning framework for multi-sensor-based wearable human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (2022), 1--25.

Digital Library

[49]

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018).

[50]

Francisco Javier Ordóñez and Daniel Roggen. 2016. Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 1 (2016), 115.

[51]

Xiaomin Ouyang, Xian Shuai, Jiayu Zhou, Ivy Wang Shi, Zhiyuan Xie, Guoliang Xing, and Jianwei Huang. 2022. Cosmo: Contrastive fusion learning with small data for multimodal human activity recognition. In Proceedings of the Annual International Conference on Mobile Computing And Networking. 324--337.

Digital Library

[52]

Liangying Peng, Ling Chen, Zhenan Ye, and Yi Zhang. 2018. AROMA: A deep multi-task learning based simple and complex human activity recognition method using wearable sensors. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 2 (2018), 1--16.

Digital Library

[53]

Thomas PlÖtz. 2021. Applying machine learning for sensor data analysis in interactive systems: Common pitfalls of pragmatic use and ways to avoid them. Computing Surveys 54, 6 (2021), 1--25.

Digital Library

[54]

Hangwei Qian, Tian Tian, and Chunyan Miao. 2022. What makes good contrastive learning on small-scale wearable-based tasks? arXiv preprint arXiv:2202.05998 (2022).

[55]

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.

[56]

Setareh Rahimi Taghanaki, Michael J Rainbow, and Ali Etemad. 2021. Self-supervised human activity recognition by learning to predict cross-dimensional motion. In Proceedings of the International Symposium on Wearable Computers. 23--27.

Digital Library

[57]

Attila Reiss and Didier Stricker. 2012. Introducing a new benchmarked dataset for activity monitoring. In Proceedings of the International Symposium on Wearable Computers. 108--109.

Digital Library

[58]

Vitor Fortes Rey, Sungho Suh, and Paul Lukowicz. 2022. Learning from the best: Contrastive representations learning across sensor locations for wearable activity recognition. arXiv preprint arXiv:2210.01459 (2022).

[59]

Daniel Roggen, Alberto Calatroni, Mirco Rossi, Thomas Holleczek, Kilian Förster, Gerhard Tröster, Paul Lukowicz, David Bannach, Gerald Pirkl, Alois Ferscha, et al. 2010. Collecting complex activity datasets in highly rich networked sensor environments. In Proceedings of the International Conference on Networked Sensing Systems. 233--240.

[60]

Aaqib Saeed, Tanir Ozcelebi, and Johan Lukkien. 2018. Synthesizing and reconstructing missing sensory modalities in behavioral context recognition. Sensors 18, 9 (2018), 2967.

[61]

Aaqib Saeed, Tanir Ozcelebi, and Johan Lukkien. 2019. Multi-task self-supervised learning for human activity detection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 2 (2019), 1--30.

Digital Library

[62]

Timo Sztyler and Heiner Stuckenschmidt. 2016. On-body localization of wearable devices: An investigation of position-aware activity recognition. In Proceedings of the International Conference on Pervasive Computing and Communications. 1--9.

[63]

Chi Ian Tang, Ignacio Perez-Pozuelo, Dimitris Spathis, Soren Brage, Nick Wareham, and Cecilia Mascolo. 2021. Selfhar: Improving human activity recognition through self-training with unlabeled data. arXiv preprint arXiv:2102.06073 (2021).

[64]

Chi Ian Tang, Ignacio Perez-Pozuelo, Dimitris Spathis, and Cecilia Mascolo. 2020. Exploring contrastive learning in human activity recognition for healthcare. arXiv preprint arXiv:2011.11542 (2020).

[65]

Yonatan Vaizman, Nadir Weibel, and Gert Lanckriet. 2018. Context recognition in-the-wild: Unified model for multi-modal sensors and multi-label classification. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 4 (2018), 1--22.

Digital Library

[66]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems. 5998--6008.

[67]

Chongyang Wang, Yuan Gao, Akhil Mathur, Amanda C De C. Williams, Nicholas D Lane, and Nadia Bianchi-Berthouze. 2021. Leveraging activity recognition to enable protective behavior detection in continuous data. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1--27.

Digital Library

[68]

Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. 2022. Transformers in time series: A survey. arXiv preprint arXiv:2202.07125 (2022).

[69]

Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. 2022. CoST: Contrastive learning of disentangled seasonal-trend representations for time series forecasting. arXiv preprint arXiv:2202.01575 (2022).

[70]

Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. 2021. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Proceedings of the Advances in Neural Information Processing Systems, 22419--22430.

[71]

Huatao Xu, Pengfei Zhou, Rui Tan, Mo Li, and Guobin Shen. 2021. LIMU-BERT: Unleashing the potential of unlabeled data for IMU sensing applications. In Proceedings of the ACM Conference on Embedded Networked Sensor Systems. 220--233.

Digital Library

[72]

Ling Yang and Shenda Hong. 2022. Unsupervised time-series representation learning with iterative bilinear temporal-spectral fusion. In International Conference on Machine Learning. PMLR, 25038--25054.

[73]

Shuochao Yao, Shaohan Hu, Yiran Zhao, Aston Zhang, and Tarek Abdelzaher. 2017. DeepSense: A unified deep learning framework for time-series mobile sensing data processing. In Proceedings of the International Conference on World Wide Web. 351--360.

Digital Library

[74]

Shuochao Yao, Yiran Zhao, Huajie Shao, Dongxin Liu, Shengzhong Liu, Yifan Hao, Ailing Piao, Shaohan Hu, Su Lu, and Tarek F Abdelzaher. 2019. SADeepSense: Self-attention deep learning framework for heterogeneous on-device sensors in internet of things applications. In Proceedings of the INFOCOM Conference on Computer Communications. 1243--1251.

Digital Library

[75]

Jinsung Yoon, James Jordon, and Mihaela Schaar. 2018. Gain: Missing data imputation using generative adversarial nets. In Proceedings of the International Conference on Machine Learning. 5689--5698.

[76]

Hang Yuan, Shing Chan, Andrew P Creagh, Catherine Tong, David A Clifton, and Aiden Doherty. 2022. Self-supervised learning for human activity recognition using 700,000 person-days of wearable data. arXiv preprint arXiv:2206.02909 (2022).

[77]

Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, and Bixiong Xu. 2022. TS2Vec: Towards universal representation of time series. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 8980--8987.

[78]

Piero Zappi, Thomas Stiefmeier, Elisabetta Farella, Daniel Roggen, Luca Benini, and Gerhard Troster. 2007. Activity recognition from on-body sensors by classifier fusion: sensor scalability and robustness. In Proceedings of the International Conference on Intelligent Sensors, Sensor Networks and Information. 281--286.

[79]

George Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff. 2021. A transformer-based framework for multivariate time series representation learning. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2114--2124.

Digital Library

[80]

Bing Zhai, Yu Guan, Michael Catt, and Thomas Plötz. 2021. Ubi-SleepNet: Advanced multimodal fusion techniques for three-stage sleep classification using ubiquitous sensing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 4 (2021), 1--33.

Digital Library

[81]

Hanbin Zhang, Li Zhu, Viswam Nathan, Jilong Kuang, Jacob Kim, Jun Alex Gao, and Jeffrey Olgin. 2021. Towards early detection and burden estimation of atrial fibrillation in an ambulatory free-living environment. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1--19.

Digital Library

[82]

Shibo Zhang, Yuqi Zhao, Dzung Tri Nguyen, Runsheng Xu, Sougata Sen, Josiah Hester, and Nabil Alshurafa. 2020. Necksense: A multi-sensor necklace for detecting eating activities in free-living conditions. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 2 (2020), 1--26.

Digital Library

[83]

Xiang Zhang, Ziyuan Zhao, Theodoros Tsiligkaridis, and Marinka Zitnik. 2022. Self-supervised contrastive pre-training for time series via time-frequency consistency. arXiv preprint arXiv:2206.08496 (2022).

[84]

Ye Zhang, Longguang Wang, Huiling Chen, Aosheng Tian, Shilin Zhou, and Yulan Guo. 2022. IF-ConvTransformer: A framework for human activity recognition using IMU fusion and ConvTransformer. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 2 (2022), 1--26.

Digital Library

[85]

Jingguang Zhou and Zili Huang. 2018. Recover missing sensor data with iterative imputing network. In Proceedings of the Workshops at the AAAI Conference on Artificial Intelligence.

[86]

Zhongyi Zhou, Anran Xu, and Koji Yatani. 2021. SyncUp: Vision-based practice support for synchronized dancing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 3 (2021), 1--25.

Digital Library

[87]

Wentao Zhu, Xiaoxuan Ma, Zhaoyang Liu, Libin Liu, Wayne Wu, and Yizhou Wang. 2022. MotionBERT: Unified pretraining for human motion analysis. arXiv preprint arXiv:2210.06551 (2022).

Cited By

Huang JGuo XGu CMiao YHe SShu YChen J(2025)MagKey: Empowering Wearables with Ballistocardiography-based Key Generation through Magnetic Field Vibration SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/37122729:1(1-28)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3712272
Logacjov ABach KMork P(2025)Long-term self-supervised learning for accelerometer-based sleep–wake recognitionEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109758141:COnline publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1016/j.engappai.2024.109758
M JA ND SR SN BN SSingasani TVatambeti R(2024)Segment-Based Unsupervised Deep Learning for Human Activity Recognition using Accelerometer Data and SBOA based Channel Attention NetworksInternational Research Journal of Multidisciplinary Technovation10.54392/irjmt2461(1-16)Online publication date: 29-Oct-2024
https://doi.org/10.54392/irjmt2461
Show More Cited By

Index Terms

Spatial-Temporal Masked Autoencoder for Multi-Device Wearable Human Activity Recognition
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing theory, concepts and paradigms
      1. Mobile computing

Recommendations

Assessing the State of Self-Supervised Human Activity Recognition Using Wearables

The emergence of self-supervised learning in the field of wearables-based human activity recognition (HAR) has opened up opportunities to tackle the most pressing challenges in the field, namely to exploit unlabeled data to derive reliable recognition ...
Few-Shot Human Activity Recognition on Noisy Wearable Sensor Data
Database Systems for Advanced Applications
Abstract
Most existing wearable sensor-based human activity recognition (HAR) models are trained on substantial labeled data. It is difficult for HAR to learn new-class activities unseen during training from a few samples. Very few researches of few-shot ...
Wearable-based behaviour interpolation for semi-supervised human activity recognition
Abstract
While traditional feature engineering for Human Activity Recognition (HAR) involves a trial-and-error process, deep learning has emerged as a preferred method for high-level representations of sensor-based human activities. However, most deep ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 7, Issue 4

December 2023

1613 pages

EISSN:2474-9567

DOI:10.1145/3640795

Issue’s Table of Contents

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 January 2024

Published in IMWUT Volume 7, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
858
Total Downloads

Downloads (Last 12 months)609
Downloads (Last 6 weeks)45

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Huang JGuo XGu CMiao YHe SShu YChen J(2025)MagKey: Empowering Wearables with Ballistocardiography-based Key Generation through Magnetic Field Vibration SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/37122729:1(1-28)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3712272
Logacjov ABach KMork P(2025)Long-term self-supervised learning for accelerometer-based sleep–wake recognitionEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109758141:COnline publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1016/j.engappai.2024.109758
M JA ND SR SN BN SSingasani TVatambeti R(2024)Segment-Based Unsupervised Deep Learning for Human Activity Recognition using Accelerometer Data and SBOA based Channel Attention NetworksInternational Research Journal of Multidisciplinary Technovation10.54392/irjmt2461(1-16)Online publication date: 29-Oct-2024
https://doi.org/10.54392/irjmt2461
Logacjov A(2024)Self-supervised Learning for Accelerometer-based Human Activity Recognition: A SurveyProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997678:4(1-42)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699767
Miao SChen L(2024)GOAT: A Generalized Cross-Dataset Activity Recognition Framework with Natural Language SupervisionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997368:4(1-28)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699736
Haresamudram HEssa IPlötz T(2024)A Washing Machine is All You Need? On the Feasibility of Machine Data for Self-Supervised Human Activity Recognition2024 International Conference on Activity and Behavior Computing (ABC)10.1109/ABC61795.2024.10651688(1-10)Online publication date: 29-May-2024
https://doi.org/10.1109/ABC61795.2024.10651688
Contoli CFreschi VLattanzi E(2024)Energy-aware human activity recognition for wearable devices: A comprehensive reviewPervasive and Mobile Computing10.1016/j.pmcj.2024.101976104(101976)Online publication date: Nov-2024
https://doi.org/10.1016/j.pmcj.2024.101976

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents