Abstract
Understanding human-object interaction is important for recognizing the activity and the sequence of actions performed. Egocentric tracking of people’s actions and interactions has long been a research topic in many fields. Humans use their hands to manipulate objects in their daily lives to perform various activities. We contend that it is possible to determine human activity by watching how the wrist, palm, and fingers move and how they affect objects in the immediate area. There is a need to recognize the sequence of human actions. This is the key to understanding the activities and inferring the success or failure of the activity when manipulating objects. In this paper, we present a new perspective view, the wrist-centric view, a view from the wrist of the person while performing activities of daily living (ADLs). We explored activities of daily living (ADLs) through the wrist-centric view to identify activities where this novel view is advantageous over other egocentric views. This paper explores the importance of understanding human-object interaction in identifying activities and recognizing ADLs in finer detail. ADLs such as cooking, laundry, eating, drinking, doing dishes, interacting with people, gesturing, shopping, reading, walking, and interacting with everyday objects such as keys, glasses, and medication were selected to depict the representational motions a person needs to perform to carry out daily tasks. We provide different perspectives on these activities, including chest-centric and wrist-centric views, and demonstrate which scenarios the wrist-centric view is most advantageous.
Supported by National Science Foundation under Grant No. 1828010 and 2142774 and Arizona State University.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Jones, L.A., Lederman, S.J.: Human Hand Function. Oxford Academic, New York (2006). https://doi.org/10.1093/acprof:oso/9780195173154.001.0001. Accessed 12 Aug 2022
Gjoreski, M., Gjoreski, H., Luštrek, M., Gams, M.: How accurately can your wrist device recognize daily activities and detect falls? Sensors 16, 800 (2016). https://doi.org/10.3390/s16060800
Bucks, R.S., Ashworth, D.L., Wilcock, G.K., Siegfried, K.: Assessment of activities of daily living in dementia: development of the Bristol activities of daily living scale. Age Ageing. 25(2), 113–120 (1996). PMID: 8670538. https://doi.org/10.1093/ageing/25.2.113
Katz, S.: Assessing self-maintenance: activities of daily living, mobility, and instrumental activities of daily living. J. Am. Geriatr. Soc. 31(12), 721–7 (1983)
Bieńkiewicz, M.M., Brandi, M.L., Goldenberg, G., Hughes, C.M., Hermsdörfer, J.: The tool in the brain: apraxia in ADL: behavioral and neurological correlates of apraxia in daily living. Front. Psychol. 5, 353 (2014)
Geriatric Medicine Research Collaborative. Delirium is prevalent in older hospital inpatients and associated with adverse outcomes: results of a prospective multi-centre study on World Delirium Awareness Day. BMC Med. 17(1), 229 (2019). PMID: 31837711; PMCID: PMC6911703. https://doi.org/10.1186/s12916-019-1458-7
Edemekong, P.F., Bomgaars, D.L., Sukumaran, S., et al.: Activities of Daily Living. In: StatPearls [Internet]. Treasure Island (FL). StatPearls Publishing (2022). https://www.ncbi.nlm.nih.gov/books/NBK470404/
Farias, S.T., et al.: Everyday cognition in older adults: associations with neuropsychological performance and structural brain imaging. J. Int. Neuropsychol. Soc. 19(4), 430–441 (2013 ). PMID: 23369894; PMCID: PMC3818105. https://doi.org/10.1017/S1355617712001609
Farias, S.T., Harrell, E., Neumann, C., Houtz, A.: The relationship between neuropsychological performance and daily functioning in individuals with Alzheimer’s disease: ecological validity of neuropsychological tests. Arch. Clin. Neuropsychol. 18(6), 655–72 (2003). PMID: 14591439
Chu, N.M., et al.: Functional independence, access to kidney transplantation and waitlist mortality. Nephrol. Dial. Transplant. 35(5), 870–877 (2020). PMID: 31860087; PMCID: PMC7849992. https://doi.org/10.1093/ndt/gfz265
Li, Y., Ye, Z., Rehg, J.M.: Delving into egocentric actions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)
Soomro, K., Amir, Z., Mubarak, S.: UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild. CoRR (2012)
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., Serre,T.: HMDB: a large video database for human motion recognition. In: 2011 International Conference on Computer Vision, Barcelona, Spain, pp. 2556–2563 (2011). https://doi.org/10.1109/ICCV.2011.6126543
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: Large-scale video classification with convolutional neural networks. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, pp. 1725–1732 (2014). https://doi.org/10.1109/CVPR.2014.223
Marszalek, M., Ivan, L., Cordelia, S.: Actions in context. In: Proceedings CVPR, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2929–2936 (2009). https://doi.org/10.1109/CVPR.2009.5206557
Heilbron, F.C., Escorcia, V., Ghanem, B., Niebles, J.C.: ActivityNet: a large-scale video benchmark for human activity understanding. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015, pp. 961–970 (2015). https://doi.org/10.1109/CVPR.2015.7298698
Smaira, L., Carreira, J., Noland, E., Clancy, E., Wu, A., Zisserman, A.: A Short Note on the Kinetics-700-2020 Human Action Dataset (2020)
Rohrbach, M., et al.: Recognizing fine-grained and composite activities using hand-centric features and script data. Int. J. Comput. Vision 119(3), 346–373 (2015). https://doi.org/10.1007/s11263-015-0851-8
Alahari, K.: Actor and observer: joint modeling of first and third-person videos. In: Proceedings of the 1st Workshop and Challenge on Comprehensive Video Understanding in the Wild (CoVieW 2018), vol. 3. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3265987.3265995
Damen, D., et al.: Rescaling egocentric vision: collection, pipeline and challenges for EPIC-KITCHENS-100. Int. J. Comput. Vision 130(1), 33–55 (2021). https://doi.org/10.1007/s11263-021-01531-2
Tavakolizadeh, F., Gu, J., Saket, B.: Traceband: locating missing items by visual remembrance. In Proceedings of the Adjunct Publication of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST 2014 Adjunct), pp. 109–110. Association for Computing Machinery, New York (2014)
Maekawa, T., Kishino, Y., Yanagisawa, Y., Sakurai, Y.: WristSense: wrist-worn sensor device with camera for daily activity recognition. In: 2012 IEEE International Conference on Pervasive Computing and Communications Workshops (2012)
. Vardy, A., Robinson, J., Cheng, L.T.: The WristCam as input device. In: Digest of Papers: Third International Symposium on Wearable Computers, pp. 199–202 (1999). https://doi.org/10.1109/ISWC.1999.806928
Kakaraparthi, V., McDaniel, T., Venkateswara, H., Goldberg, M.: PERACTIV: personalized activity monitoring - ask my hands. In: Distributed, Ambient and Pervasive Interactions. Smart Living, Learning, Well-being and Health, Art and Creativity: 10th International Conference, DAPI 2022, Held as Part of the 24th HCI International Conference, HCII 2022, Virtual Event, Proceedings, Part II, 26 June–1 July 2022, pp. 255–272. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-031-05431-0_18
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kakaraparthi, V., Goldberg, M., McDaniel, T. (2023). Wrist View: Understanding Human Activity Through the Hand. In: Antona, M., Stephanidis, C. (eds) Universal Access in Human-Computer Interaction. HCII 2023. Lecture Notes in Computer Science, vol 14021. Springer, Cham. https://doi.org/10.1007/978-3-031-35897-5_41
Download citation
DOI: https://doi.org/10.1007/978-3-031-35897-5_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35896-8
Online ISBN: 978-3-031-35897-5
eBook Packages: Computer ScienceComputer Science (R0)