Abstract
In this chapter, we describe a system that provides continuous localization and behavioral analysis of a person’s motion pattern over an indoor living space using multiple Kinect sensors. The skeleton data from all sensors is transferred to the host computer via TCP sockets into a program where the data is integrated into a single world coordinate system using a calibration technique. Multiple cameras are placed with some overlap in the field of view for the successful calibration of the cameras and continuous tracking of the patients. Localization and behavioral data is stored in a CSV file for further analysis. The experiments show that the system can reliably detect sitting and standing poses, as well as basic gait parameters of a user who is walking within the field of view. This system may be used in an assistive living environment to track the activities of daily living of seniors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Atrsaei A, Salarieh H, Alasty A, Abediny M (2018) Human arm motion tracking by inertial/magnetic sensors using unscented Kalman filter and relative motion constraint. J Intell Rob Syst 90(1–2):161–170
Azis NA, Choi HJ, Iraqi Y (2015) Substitutive skeleton fusion for human action recognition. In: 2015 International conference on big data and smart computing (BigComp). IEEE, pp 170–177
Beymer D, Konolige K (1999) Real-time tracking of multiple people using continuous detection. In: IEEE Frame rate workshop, pp 1–8
Breitenstein MD, Reichlin F, Leibe B, Koller-Meier E, Van Gool L (2009) Robust tracking-by-detection using a detector confidence particle filter. In: 2009 IEEE 12th international conference on computer vision. IEEE, pp 1515–1522
Caon M, Yue Y, Tscherrig J, Mugellini E, Khaled OA (2011) Context-aware 3d gesture interaction based on multiple kinects. In: Proceedings of the first international conference on ambient computing, applications, services and technologies, AMBIENT. Citeseer, pp 7–12
Chen D, Bharucha AJ, Wactlar HD (2007) Intelligent video monitoring to improve safety of older persons. In: 2007 29th Annual international conference of the IEEE engineering in medicine and biology society. IEEE, pp 3814–3817
Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE Trans Pattern Anal Mach Intell 5:564–575
Ercan AO, Gamal AE, Guibas LJ (2013) Object tracking in the presence of occlusions using multiple cameras: a sensor network approach. ACM Trans Sensor Netw (TOSN) 9(2):16
Fuentes LM, Velastin SA (2001) People tracking in surveillance applications. In: Proceedings of 2nd IEEE international workshop on PETS, Kauai, Hawaii, USA
Ikemura S, Fujiyoshi H (2010) Real-time human detection using relational depth similarity features. In: Asian conference on computer vision. Springer, Berlin, pp 25–38
Jones B, Sodhi R, Murdock M, Mehra R, Benko H, Wilson A, Ofek E, MacIntyre B, Raghuvanshi N, Shapira L (2014) Roomalive: magical experiences enabled by scalable, adaptive projector-camera units. In: Proceedings of the 27th annual ACM symposium on user interface software and technology. ACM, pp 637–644
Kalman RE (1960) A new approach to linear filtering and prediction problems. J Basic Eng 82(1):35–45
Klingbeil L, Wark T (2008) A wireless sensor network for real-time indoor localisation and motion monitoring. In: 2008 International conference on information processing in sensor networks (IPSN 2008). IEEE, pp 39–50
Lin Q, Zhang D, Chen L, Ni HB, Zhou S (2014) Managing elders’ wandering behavior using sensors-based solutions: a survey. Int J Gerontol 8(2):49–56
Lun R, Zhao W (2015) A survey of applications and human motion recognition with microsoft kinect. Int J Pattern Recognit Artif Intell 29(05):1555008
Lun R, Gordon C, Zhao W (2016) The design and implementation of a kinect-based framework for selective human activity tracking. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, pp 002890–002895
Lun R, Gordon C, Zhao W (2016) Tracking the activities of daily lives: an integrated approach. In: 2016 Future technologies conference (FTC). IEEE, pp 466–475
Masuyama G, Kawashita T, Umeda K (2017) Complementary human detection and multiple feature based tracking using a stereo camera. ROBOMECH J 4(1):24
Munaro M, Basso F, Menegatti E (2016) Openptrack: open source multi-camera calibration and people tracking for RGB-D camera networks. Robot Auton Syst 75:525–538
Papoulis A, Pillai SU (2002) Probability, random variables, and stochastic processes. Tata McGraw-Hill Education
Poland MP, Nugent CD, Wang H, Chen L (2012) Genetic algorithm and pure random search for exosensor distribution optimisation. Int J Bio-Inspired Comput 4(6):359–372
Ponraj G, Ren H (2018) Sensor fusion of leap motion controller and flex sensors using Kalman filter for human finger tracking. IEEE Sens J 18(5):2042–2049
Spinello L, Arras KO (2011) People detection in RGB-D data. In: 2011 IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 3838–3843
Torres-Solis J, Chau T (2010) Wearable indoor pedestrian dead reckoning system. Pervasive Mob Comput 6(3):351–361
Williamson B, LaViola J, Roberts T, Garrity P (2012) Multi-kinect tracking for dismounted soldier training. In: Proceedings of the interservice/industry training, simulation, and education conference (I/ITSEC), pp 1727–1735
Wren CR, Pentland AP (1998) Dynamic models of human motion. In: Proceedings third IEEE international conference on automatic face and gesture recognition. IEEE, pp 22–27
Zhao W, Lun R, Espy DD, Reinthal MA (2014) Realtime motion assessment for rehabilitation exercises: integration of kinematic modeling with fuzzy inference. J Artif Intell Soft Comput Res 4(4):267–285
Zhao W, Lun R, Gordon C, Fofana ABM, Espy DD, Reinthal A, Ekelman B, Goodman GD, Niederriter JE, Luo C et al (2016) Liftingdoneright: a privacy-aware human motion tracking system for healthcare professionals. Int J Handheld Comput Res (IJHCR) 7(3):1–15
Zhao W, Lun R, Gordon C, Fofana ABM, Espy DD, Reinthal MA, Ekelman B, Goodman GD, Niederriter JE, Luo X (2017) A human-centered activity tracking system: toward a healthier workplace. IEEE Trans Human-Mach Syst 47(3):343–355
Zhao W, Reinthal MA, Espy DD, Luo X (2017) Rule-based human motion tracking for rehabilitation exercises: realtime assessment, feedback, and guidance. IEEE Access 5:21382–21394
Zhao W, Wu Q, Reinthal A, Zhang N (2018) Design, implementation, and field testing of a privacy-aware compliance tracking system for bedside care in nursing homes. Appl Syst Innov 1(1):3
Zhao W (2016) A concise tutorial on human motion tracking and recognition with Microsoft kinect. Sci China Inf Sci 59(9):93101
Zhao W (2016) On automatic assessment of rehabilitation exercises with realtime feedback. In: 2016 IEEE international conference on electro information technology (EIT). IEEE, pp 0376–0381
Zhao W, Lun R (2016) A kinect-based system for promoting healthier living at home. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, pp 000258–000263
Zhao W, Espy DD, Reinthal MA, Feng H (2014) A feasibility study of using a single kinect sensor for rehabilitation exercises monitoring: a rule based approach. In: 2014 IEEE symposium on computational intelligence in healthcare and e-health (CICARE). IEEE, pp 1–8
Zhao W, Feng H, Lun R, Espy DD, Reinthal MA (2014) A kinect-based rehabilitation exercise monitoring and guidance system. In: 2014 IEEE 5th international conference on software engineering and service science. IEEE, pp 762–765
Zhao W, Lun R, Espy DD, Reinthal MA (2014) Rule based realtime motion assessment for rehabilitation exercises. In: 2014 IEEE symposium on computational intelligence in healthcare and e-health (CICARE). IEEE, pp 133–140
Zhao W, Espy DD, Reinthal MA, Ekelman B, Goodman G, Niederriter J (2015) Privacy-aware human motion tracking with realtime haptic feedback. In: 2015 IEEE international conference on mobile services. IEEE, pp 446–453
Zhao W, Lun R, Gordon C, Fofana AB, Espy DD, Reinthal MA, Ekelman B, Goodman G, Niederriter J, Luo C et al (2016) A privacy-aware kinect-based system for healthcare professionals. In: 2016 IEEE international conference on electro information technology (EIT). IEEE, pp 0205–0210
Zhao W, Wu Q, Espy DD, Reinthal MA, Luo X, Peng Y (2017) A feasibility study on using a kinect-based human motion tracking system to promote safe patient handling. In: 2017 IEEE international conference on electro information technology (EIT). IEEE, pp 462–466
Zhao W, Wu Q, Padaraju V, Bbela M, Reinthal A, Espy D, Luo X, Qiu T (2017) A privacy-aware compliance tracking system for skilled nursing facilities. In: 2017 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, pp 3568–3573
Zhao W, Pillai JA, Leverenz JB, Luo X (2018) Technology-facilitated detection of mild cognitive impairment: a review. In: 2018 IEEE international conference on electro/information technology (EIT). IEEE, pp 0284–0289
Zhu L, Wong KH (2013) Human tracking and counting using the kinect range sensor based on Adaboost and Kalman filter. In: International symposium on visual computing. Springer, Berlin, pp 582–591
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Yang, S., Hans, A., Zhao, W., Luo, X. (2020). Indoor Localization and Human Activity Tracking with Multiple Kinect Sensors. In: Chen, F., García-Betances, R., Chen, L., Cabrera-Umpiérrez, M., Nugent, C. (eds) Smart Assisted Living. Computer Communications and Networks. Springer, Cham. https://doi.org/10.1007/978-3-030-25590-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-25590-9_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-25589-3
Online ISBN: 978-3-030-25590-9
eBook Packages: Computer ScienceComputer Science (R0)