Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Data Generation Process Modeling for Activity Recognition

  • Conference paper
  • First Online:
Machine Learning and Knowledge Discovery in Databases: Applied Data Science Track (ECML PKDD 2020)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12460))

Abstract

The dynamics of body movements are often driven by large and intricate low-level interactions involving various body parts. These dynamics are part of an underlying data generation process. Incorporating the data generation process into data-driven activity recognition systems has the potential to enhance their robustness and data-efficiency. In this paper, we propose to model the underlying data generation process and use it to constrain training of simpler learning models via sample selection. As deriving such models using human expertise is hard, we propose to frame this task as a large-scale exploration of architectures in charge of relating sensory information coming from the data sources. We report on experiments conducted on the Sussex-Huawei locomotion dataset featuring a sensor-rich environment in real-life settings. The derived model is found to be consistent with existing domain knowledge. Compared to the basic setting, our approach achieves up to 17.84% improvement, by simultaneously reducing the number of required data sources by one-half. Promising results open perspectives for deploying more robust and data-efficient learning models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Software package and code to reproduce empirical results are publicly available at: https://github.com/sensor-rich/shl-nas.

  2. 2.

    https://github.com/microsoft/nni.

References

  1. Abadi, M., et al.: Tensorflow: a system for large-scale machine learning. In: OSDI, vol. 16, pp. 265–283 (2016)

    Google Scholar 

  2. Atrey, P.K., Hossain, M.A., El Saddik, A., Kankanhalli, M.S.: Multimodal fusion for multimedia analysis: a survey. Multimed. Syst. 16(6), 345–379 (2010). https://doi.org/10.1007/s00530-010-0182-0

    Article  Google Scholar 

  3. Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. JMLR 13, 281–305 (2012)

    MathSciNet  MATH  Google Scholar 

  4. Bergstra, J.S., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyper-parameter optimization. In: NIPS, pp. 2546–2554 (2011)

    Google Scholar 

  5. Bevilacqua, A., MacDonald, K., Rangarej, A., Widjaya, V., Caulfield, B., Kechadi, T.: Human activity recognition with convolutional neural networks. In: Brefeld, U., et al. (eds.) ECML PKDD 2018. LNCS (LNAI), vol. 11053, pp. 541–552. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-10997-4_33

    Chapter  Google Scholar 

  6. Carpineti, C., Lomonaco, V., Bedogni, L., Di Felice, M., Bononi, L.: Custom dual transportation mode detection by smartphone devices exploiting sensor diversity. In: International Conference on Pervasive Computing and Communications Workshops, pp. 367–372. IEEE (2018)

    Google Scholar 

  7. Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Measur. 20(1), 37–46 (1960)

    Article  Google Scholar 

  8. Dhiman, C., Vishwakarma, D.K., Aggarwal, P.: Skeleton based activity recognition by fusing part-wise spatio-temporal and attention driven residues. arXiv preprint arXiv:1912.00576 (2019)

  9. Elsken, T., Metzen, J.H., Hutter, F.: Neural architecture search: a survey. JMLR 20(55), 1–21 (2019)

    MathSciNet  MATH  Google Scholar 

  10. Falkner, S., Klein, A., Hutter, F.: Bohb: robust and efficient hyperparameter optimization at scale. In: ICML, pp. 1437–1446 (2018)

    Google Scholar 

  11. Foerster, F., Smeja, M., Fahrenberg, J.: Detection of posture and motion by accelerometry: a validation study in ambulatory monitoring. Comput. Hum. Behav. 15(5), 571–583 (1999)

    Article  Google Scholar 

  12. Forman, G., Scholz, M.: Apples-to-apples in cross-validation studies: pitfalls in classifier performance measurement. SIGKDD Explor. Newslet. 12(1), 49–57 (2010)

    Article  Google Scholar 

  13. Gaier, A., Ha, D.: Weight agnostic neural networks. In: NeurIPS (2019)

    Google Scholar 

  14. Gjoreski, H., Ciliberto, M., Morales, F.J.O., Roggen, D., Mekki, S., Valentin, S.: A versatile annotated dataset for multimodal locomotion analytics with mobile devices. In: Conference on Embedded Network Sensor Systems, p. 61. ACM (2017)

    Google Scholar 

  15. Gjoreski, H., et al.: The university of Sussex-Huawei locomotion and transportation dataset for multimodal analytics with mobile devices. IEEE Access, 6, 42592–42604 (2018)

    Google Scholar 

  16. Hammerla, N.Y., Plötz, T.: Let’s (not) stick together: pairwise similarity biases cross-validation in activity recognition. In: UbiComp, pp. 1041–1051. ACM (2015)

    Google Scholar 

  17. Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)

  18. Hoeffding, W.: A non-parametric test of independence. Ann. Math. Stat. 546–557 (1948)

    Google Scholar 

  19. Hoos, H., Leyton-Brown, K.: An efficient approach for assessing hyperparameter importance. In: International Conference on Machine Learning, pp. 754–762 (2014)

    Google Scholar 

  20. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: ICML, vol. 37, pp. 448–456. PMLR (2015)

    Google Scholar 

  21. Kovalenko, M., Antoshchuk, S., Sieck, J.: Real-time hand tracking and gesture recognition using semantic-probabilistic network. In: International Conference on Computer Modelling and Simulation, pp. 269–274. IEEE (2014)

    Google Scholar 

  22. Kurle, R., Günnemann, S., van der Smagt, P.: Multi-source neural variational inference. In: AAAI, vol. 33, pp. 4114–4121 (2019)

    Google Scholar 

  23. Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., Talwalkar, A.: Hyperband: a novel bandit-based approach to hyperparameter optimization. JMLR 18(1), 6765–6816 (2017)

    MathSciNet  MATH  Google Scholar 

  24. Liu, H., Simonyan, K., Yang, Y.: Darts: differentiable architecture search. In: ICLR (2019)

    Google Scholar 

  25. Mantyjarvi, J., Himberg, J., Seppanen, T.: Recognizing human motion with multiple acceleration sensors. In: SMC, vol. 2, pp. 747–752. IEEE (2001)

    Google Scholar 

  26. Muehlenstaedt, T., Roustant, O., Carraro, L., Kuhnt, S.: Data-driven Kriging models based on FANOVA-decomposition. Stat. Comput. 22(3), 723–738 (2012). https://doi.org/10.1007/s11222-011-9259-7

    Article  MathSciNet  MATH  Google Scholar 

  27. Nabian, M.A., Meidani, H.: Physics-driven regularization of deep neural networks for enhanced engineering design and analysis. J. Comput. Inf. Sci. in Eng. 20(1) (2020)

    Google Scholar 

  28. Osmani, A., Hamidi, M., Bouhouche, S.: Monitoring of a dynamical system based on autoencoders. In: IJCAI (2019)

    Google Scholar 

  29. Ousmer, M., Vanderdonckt, J., Buraga, S.: An ontology for reasoning on body-based gestures. In: SIGCHI EICS, pp. 1–6. ACM (2019)

    Google Scholar 

  30. Papadopoulos, G.T., Axenopoulos, A., Daras, P.: Real-time skeleton-tracking-based human action recognition using kinect data. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds.) MMM 2014. LNCS, vol. 8325, pp. 473–483. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-04114-8_40

    Chapter  Google Scholar 

  31. Parisi, G.I., Tani, J., Weber, C., Wermter, S.: Emergence of multimodal action representations from neural network self-organization. Cogn. Syst. Res. 43, 208–221 (2017)

    Article  Google Scholar 

  32. Pham, H., Guan, M., Zoph, B., Le, Q., Dean, J.: Efficient neural architecture search via parameters sharing. In: ICML, vol. 80, pp. 4095–4104. PMLR (2018)

    Google Scholar 

  33. Radu, V., et al.: Multimodal deep learning for activity and context recognition. IMWUT 1(4), 157 (2018)

    Google Scholar 

  34. Real, E., et al.: Large-scale evolution of image classifiers. In: ICML, pp. 2902–2911 (2017)

    Google Scholar 

  35. Reddy, S., Mun, M., Burke, J., Estrin, D., Hansen, M., Srivastava, M.: Using mobile phones to determine transportation modes. TOSN 6(2), 13 (2010)

    Article  Google Scholar 

  36. Díaz Rodríguez, N., Wikström, R., Lilius, J., Cuéllar, M.P., Delgado Calvo Flores, M.: Understanding movement and interaction: an ontology for kinect-based 3D depth sensors. In: Urzaiz, G., Ochoa, S.F., Bravo, J., Chen, L.L., Oliveira, J. (eds.) UCAmI 2013. LNCS, vol. 8276, pp. 254–261. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-03176-7_33

    Chapter  Google Scholar 

  37. Stewart, R., Ermon, S.: Label-free supervision of neural networks with physics and domain knowledge. In: AAAI, vol. 1, pp. 1–7 (2017)

    Google Scholar 

  38. Vapnik, V., Izmailov, R.: Learning using privileged information: similarity control and knowledge transfer. JMLR 16(2023–2049), 2 (2015)

    MathSciNet  MATH  Google Scholar 

  39. Vatavu, R.D., Pentiuc, S.G.: Multi-level representation of gesture as command for human computer interaction. Comput. Inform. 27(6), 837–851 (2012)

    Google Scholar 

  40. Wang, S., Chen, C., Ma, J.: Accelerometer based transportation mode recognition on mobile phones. In: APWCS, pp. 44–46. IEEE (2010)

    Google Scholar 

  41. Yu, M.C., Yu, T., Wang, S.C., Lin, C.J., Chang, E.Y.: Big data small footprint: the design of a low-power classifier for detecting transportation modes. Proc. VLDB Endow. 7(13), 1429–1440 (2014)

    Article  Google Scholar 

  42. Zhang, M., Sawchuk, A.A.: USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors. In: UbiComp, pp. 1036–1043 (2012)

    Google Scholar 

  43. Zheng, Y., Xie, X., Ma, W.Y.: GeoLife: a collaborative social networking service among user, location and trajectory. IEEE Data Eng. Bull. 33(2), 32–39 (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratoire LIPN-UMR CNRS 7030, Univ. Sorbonne Paris Nord, Villetaneuse, France

    Massinissa Hamidi & Aomar Osmani

Authors
  1. Massinissa Hamidi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aomar Osmani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Massinissa Hamidi .

Editor information

Editors and Affiliations

  1. Microsoft Research, Redmond, WA, USA

    Yuxiao Dong

  2. Jožef Stefan Institute, Ljubljana, Slovenia

    Dunja Mladenić

  3. Amazon Alexa Knowledge, Cambridge, UK

    Craig Saunders

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hamidi, M., Osmani, A. (2021). Data Generation Process Modeling for Activity Recognition. In: Dong, Y., Mladenić, D., Saunders, C. (eds) Machine Learning and Knowledge Discovery in Databases: Applied Data Science Track. ECML PKDD 2020. Lecture Notes in Computer Science(), vol 12460. Springer, Cham. https://doi.org/10.1007/978-3-030-67667-4_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-67667-4_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-67666-7

  • Online ISBN: 978-3-030-67667-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation