Generating Multivariate Synthetic Time Series Data for Absent Sensors from Correlated Sources
Authors: 
Julián Jerónimo Bañuelos, 
Stephan Sigg, Jiayuan He, Flora Salim, Jose Costa-RequenaAuthors Info & Claims
Pages 19 - 24
Abstract
Missing sensor data in human activity recognition is an active field of research that is being targeted with generative models for synthetic data generation. In contrast to most previous approaches, we aim to generate data of a sensor exclusively from data available at sensors in different body locations. Particularly, we evaluate existing approaches proposed in the literature for their suitability in this scenario. In this paper, we focus on the prediction of acceleration data and generate machine learning models based on generative adversarial networks and trained using correlated data from sensors in different body positions to generate synthetic sensor data that can replace the missing data from a sensor in a specific body position. The accuracy of the generated synthetic data is evaluated using a classification model based on a convolutional neural network for human activity recognition.
References
[1]
Fayez Alharbi, Lahcen Ouarbya, and Jamie A Ward. 2020. Synthetic sensor data for human activity recognition. In 2020 International Joint Conference on Neural Networks (IJCNN). IEEE, 1--9.
[2]
M. Alzantot, S. Chakraborty, and M. Srivastava. 2017. SenseGen: A Deep Learning Architecture for Synthetic Sensor Data Generation. 2017 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops) (2017).
[3]
M. Alzantot, L. Garcia, and M. Srivastava. 2022. PhysioGAN: Training High Fidelity Generative Model for Physiological Sensor Readings. arXiv preprint arXiv:2204.13597 (2022).
[4]
Daniele Antonucci, Francesca Conselvan, Philipp Mascherbauer, Daniel Harringer, and Cristian Pozza. 2024. Synthetic data on buildings. In Machine Learning Applications for Intelligent Energy Management: Invited Chapters from Experts on the Energy Field. Springer, 203--226.
[5]
V. Baljak, K. Tei, and S. Honiden. 2012. Classification of Faults in Sensor Readings with Statistical Pattern Recognition. SENSORCOMM 2012: The Sixth International Conference on Sensor Technologies and Applications (2012).
[6]
Binod Bhattarai, Seungryul Baek, Rumeysa Bodur, and Tae-Kyun Kim. 2020. Sampling strategies for gan synthetic data. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2303--2307.
[7]
Ling Chen, Rong Hu, Menghan Wu, and Xin Zhou. 2023. HMGAN: A Hierarchical Multi-Modal Generative Adversarial Network Model for Wearable Human Activity Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 3 (2023), 1--27.
[8]
Robbert Claeys, Rémy Cleenwerck, Jos Knockaert, and Jan Desmet. 2024. Capturing multiscale temporal dynamics in synthetic residential load profiles through Generative Adversarial Networks (GANs). Applied Energy 360 (2024), 122831.
[9]
S. Deldari, D. Spathis, M. Malekzadeh, F. Kawsar, F. D. Salim, and A. Mathur. 2024. CroSSL: Cross-modal Self-Supervised Learning for Time-series through Latent Masking. WSDM'24 (2024).
[10]
Chance DeSmet and Diane J Cook. 2024. HydraGAN: A Cooperative Agent Model for Multi-Objective Data Generation. ACM Transactions on Intelligent Systems and Technology (2024).
[11]
Google Developers. 2024. Handling Different Sensor Configurations. https://developer.android.com/develop/sensors-and-location/sensors/sensors_overview#sensors-configs
[12]
T. Domínguez-Bolaño, V. Barral, C. J. Escudero, and J. A. García-Naya. 2024. An IoT system for a smart campus: Challenges and solutions illustrated over several real-world use cases. Internet of Things 25, 101099 (2024).
[13]
Thomas Erlebach, Michael Hoffmann, and Frank Kammer. 2021. On temporal graph exploration. J. Comput. System Sci. 119 (2021), 1--18.
[14]
André Ferreira, Jianning Li, Kelsey L Pomykala, Jens Kleesiek, Victor Alves, and Jan Egger. 2024. GAN-based generation of realistic 3D volumetric data: A systematic review and taxonomy. Medical Image Analysis (2024), 103100.
[15]
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. 2014. Generative Adversarial Nets. Advances in Neural Information Processing Systems 27 (NIPS 2014) (2014).
[16]
G. Grouios, E. Ziagkas, A. Loukovitis, K. Chatzinikolaou, and E. Koidou. 2023. Accelerometers in Our Pocket: Does Smartphone Accelerometer Technology Provide Accurate Data? Sensors 23, 192 (2023).
[17]
J. Han, Y. He, J. Liu, Q. Zhang, and X. Jing. 2019. GraphConvLSTM: Spatiotemporal Learning for Activity Recognition with Wearable Sensors. 2019 IEEE Global Communications Conference (GLOBECOM) (2019).
[18]
Mikel Hernandez, Gorka Epelde, Ane Alberdi, Rodrigo Cilla, and Debbie Rankin. 2022. Synthetic data generation for tabular health records: A systematic review. Neurocomputing 493 (2022), 28--45.
[19]
Kexin Huang, Ying Jin, Emmanuel Candes, and Jure Leskovec. 2024. Uncertainty quantification over graph with conformalized graph neural networks. Advances in Neural Information Processing Systems 36 (2024).
[20]
Md Milon Islam, Sheikh Nooruddin, Fakhri Karray, and Ghulam Muhammad. 2022. Human activity recognition using tools of convolutional neural networks: A state of the art review, data sets, challenges, and future prospects. Computers in Biology and Medicine 149 (2022), 106060.
[21]
Rashi Jaiswal. 2024. A Prediction Model for Synthetic Time Series Meta Data Fusion. In 2024 14th International Conference on Cloud Computing, Data Science & Engineering (Confluence). IEEE, 232--238.
[22]
S. Jansen. 2020. Machine Learning for Algorithmic Trading: Predictive models to extract signals from market and alternative data for systematic trading strategies with Python. Packt Publishing.
[23]
Qi Jiang, Guichuan Zhao, Xindi Ma, Meng Li, Youliang Tian, and Xinghua Li. 2024. Cross-modal Learning based Flexible Bimodal Biometric Authentication with Template Protection. IEEE Transactions on Information Forensics and Security (2024).
[24]
A Kiran and S Saravana Kumar. 2024. A methodology and an empirical analysis to determine the most suitable synthetic data generator. IEEE Access (2024).
[25]
Hendrik Klopries and Andreas Schwung. 2024. ITF-GAN: Synthetic time series dataset generation and manipulation by interpretable features. Knowledge-Based Systems 283 (2024), 111131.
[26]
Vassilis Kostakos. 2009. Temporal graphs. Physica A: Statistical Mechanics and its Applications 388, 6 (2009), 1007--1023.
[27]
X. Li, J. Luo, and R. Younes. 2020. ActivityGAN: Generative Adversarial Networks for Data Augmentation in Sensor-Based Human Activity Recognition. UbiComp/ISWC '20 Adjunct: Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers (2020).
[28]
Z. Lin, A. Jain, C. Wang, G. Fanti, and V. Sekar. 2021. Using GANs for Sharing Networked Time Series Data: Challenges, Initial Promise, and Open Questions. IMC '20: Proceedings of the ACM Internet Measurement Conference (2021).
[29]
ZG Liu, TY Ji, JW Chen, LJ Zhang, LL Zhang, and QH Wu. 2024. Conditional-TimeGAN for Realistic and High-quality Appliance Trajectories Generation and Data Augmentation in Non-intrusive Load Monitoring. IEEE Transactions on Instrumentation and Measurement (2024).
[30]
Antonio Longa, Veronica Lachi, Gabriele Santin, Monica Bianchini, Bruno Lepri, Pietro Lio, Franco Scarselli, and Andrea Passerini. 2023. Graph neural networks for temporal graphs: State of the art, open challenges, and opportunities. arXiv preprint arXiv:2302.01018 (2023).
[31]
Ciyuan Peng, Jiayuan He, and Feng Xia. 2024. Learning on Multimodal Graphs: A Survey. arXiv preprint arXiv:2402.05322 (2024).
[32]
Yang Qin, Yuan Sun, Dezhong Peng, Joey Tianyi Zhou, Xi Peng, and Peng Hu. 2024. Cross-modal Active Complementary Learning with Self-refining Correspondence. Advances in Neural Information Processing Systems 36 (2024).
[33]
T. Sztyler and H. Stuckenschmidt. 2016. On-body localization of wearable devices: an investigation of position-aware activity recognition. IEEE International Conference on Pervasive Computing and Communications, PerCom (2016).
[34]
M. Tang, G. Dong, J. Zoellner, B. Bowman, E. Abel-Rahman, and M. Boukhechba. 2022. Using Ubiquitous Mobile Sensing and Temporal Sensor-Relation Graph Neural Network to Predict Fluid Intake of End Stage Kidney Patients. 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) (2022).
[35]
Belén Vega-Márquez, Cristina Rubio-Escudero, José C Riquelme, and Isabel Nepomuceno-Chamorro. 2020. Creation of synthetic data with conditional generative adversarial networks. In 14th International Conference on Soft Computing Models in Industrial and Environmental Applications (SOCO 2019) Seville, Spain, May 13-15, 2019, Proceedings 14. Springer, 231--240.
[36]
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems 32, 1 (2020), 4--24.
[37]
Shuochao Yao, Yiran Zhao, Huajie Shao, ShengZhong Liu, Dongxin Liu, Lu Su, and Tarek Abdelzaher. 2018. Fastdeepiot: Towards understanding and optimizing neural network execution time on mobile and embedded devices. In Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems. 278--291.
[38]
S. Yao, Y. Zhao, H. Shao, C. Zhang, A. Zhang, S. Hu, D. Liu, S. Liu, L. Su, and T. Abdelzaher. 2018. SenseGAN: Enabling Deep Learning for Internet of Things with a Semi-Supervised Framework. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2 (2018).
[39]
Shuochao Yao, Yiran Zhao, Aston Zhang, Shaohan Hu, Huajie Shao, Chao Zhang, Lu Su, and Tarek Abdelzaher. 2018. Deep learning for the internet of things. Computer 51, 5 (2018), 32--41.
[40]
Jinsung Yoon, Lydia N Drumright, and Mihaela Van Der Schaar. 2020. Anonymization through data synthesis using generative adversarial networks (ads-gan). IEEE journal of biomedical and health informatics 24, 8 (2020), 2378--2388.
[41]
J. Yoon, D. Jarret, and M. van der Schaar. 2019. Time-series Generative Adversarial Networks. 33rd Conference on Neural Information Processing Systems (NeurIPS 2019) (2019).
[42]
Lu Zhang, Jingliang Peng, and Na Lv. 2024. MoCap-Video Data Retrieval with Deep Cross-Modal Learning. In International Conference on Multimedia Modeling. Springer, 494--506.
[43]
Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2020. Graph neural networks: A review of methods and applications. AI open 1 (2020), 57--81.
Index Terms
- Generating Multivariate Synthetic Time Series Data for Absent Sensors from Correlated Sources
Recommendations
Generating Synthetic Data to Match Data Mining Patterns
Synthetic data sets can be useful for repeatable regression testing and for providing realistic — but not real — data to third parties for testing new software. In some cases, it is desirable that the synthetic data set be realistic, preserving various ...
Human Activity Recognition by Using Different Deep Learning Approaches for Wearable Sensors
AbstractWith the spread of wearable sensors, the solutions to the task of activity recognition by using the data obtained from the sensors have become widespread. Recognition of activities owing to wearable sensors such as accelerometers, gyroscopes, and ...
Generating Longitudinal Synthetic EHR Data with Recurrent Autoencoders and Generative Adversarial Networks
Heterogeneous Data Management, Polystores, and Analytics for HealthcareAbstractSynthetic electronic health records (EHR) can facilitate effective use of clinical data in software development, medical education, and medical research without the concerns of data privacy. We propose a novel Generative Adversarial Network (GAN) ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
June 2024
39 pages
ISBN:9798400706615
DOI:10.1145/3662004
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- European Union?s H2020 research and innovation programme under the Marie Sk?odowska-Curie grant agreement
Conference
MOBISYS '24
Sponsor:
MOBISYS '24: The 22nd Annual International Conference on Mobile Systems, Applications and Services
June 3 - 7, 2024
Tokyo, Minato-ku, Japan
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 175Total Downloads
- Downloads (Last 12 months)175
- Downloads (Last 6 weeks)49
Reflects downloads up to 20 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in