Article

Multimodal detection of human interaction events in a nursing home environment

Authors:

Jie YangAuthors Info & Claims

ICMI '04: Proceedings of the 6th international conference on Multimodal interfaces

Pages 82 - 89

https://doi.org/10.1145/1027933.1027949

Published: 13 October 2004 Publication History

Abstract

In this paper, we propose a multimodal system for detecting human activity and interaction patterns in a nursing home. Activities of groups of people are firstly treated as interaction patterns between any pair of partners and are then further broken into individual activities and behavior events using a multi-level context hierarchy graph. The graph is implemented using a dynamic Bayesian network to statistically model the multi-level concepts. We have developed a coarse-to-fine prototype system to illustrate the proposed concept. Experimental results have demonstrated the feasibility of the proposed approaches. The objective of this research is to automatically create concise and comprehensive reports of activities and behaviors of patients to support physicians and caregivers in a nursing facility.

References

[1]

J. Nelson, "The influence of environmental factors in incidents of disruptive behavior", Journal of Gerontological Nursing 21(5):19--24, 1995.

[2]

P. D. Sloane, C. M. Mitchell, K. Long and M. Lynn, "TESS 2+ Instrument B: Unit observation checklist - physical environment: A report on the psychometric properties of individual items, and initial recommendations on scaling", University of North Carolina 1995.

[3]

F. J. Eppig and J. A. Poisal, "Mental health of medicare beneficiaries: 1995", Health Care Financing Review, 15, pages: 207--210, 1995.

[4]

F. Carp, "Assessing the environment", Annul review of gerhierarchy and geriatrics, 14, pages: 302--314, 1994.

[5]

Kimberle Koile, Konrad Tollmar, David Demirdjian, Howard E. Shrobe, Trevor Darrell, "Activity Zones for Context-Aware Computing", Ubicomp 2003, pp. 90--106, 2003.

[6]

Cory D. Kidd, Robert Orr, Gregory D. Abowd, Christopher G. Atkeson, Irfan A. Essa, Blair MacIntyre, Elizabeth Mynatt, and Thad E. Starner and Wendy Newstetter, "The Aware Home: A Living Laboratory for Ubiquitous Computing Research". Proc. of CoBuild '99, pp.191--198, 1999.

Digital Library

[7]

J. K. Aggarwal, Q. Cai, "Human Motion Analysis: A Review," Computer Vision and Image Understanding, Vol. 73, pp. 428--440, 1999.

Digital Library

[8]

D. Ayers, M. Shah, "Monitoring Human Behavior from Video Taken in an Office Environment," Image and Vision Computing, Vol. 19, pp. 833--846, 2001.

[9]

T. Jebara, A. Pentland, "Action Reaction Learning: Analysis and Synthesis of Human Behavior," IEEE Workshop on the Interpretation of Visual Motion, 1998.

[10]

Neal Checka, Kevin Wilson, Michael Siracusa, Trevor Darrell, "Multiple Person and Speaker Activity Tracking with a Particle Filter," ICASSP, 2004

[11]

S. Intille and A. Bobick, "Recognizing planned, multi-person action", Computer Vision and Image Understanding, vol. 81, no. 3, pp. 414--445, March 2001

Digital Library

[12]

L. Wang; H. Ning; T. Tan and W. Hu, "Fusion of static and dynamic body biometrics for gait recognition," IEEE Trans. Circuits and Systems for Video Technology, 14 (2), pp. 149 -- 158, 2004.

Digital Library

[13]

Y. A. Ivanov, A. F. Bobick, "Recognition of Visual Activities and Interactions by Stochastic Parsing," IEEE Trans. PAMI, Vol. 22, pp. 852--872, 2000.

Digital Library

[14]

Y. Yacoob, M. J. Black, "Parameterized Modeling and Recognition of Activities," ICCV, pp. 232--247, 1998.

Digital Library

[15]

O. Chomat and J.L. Crowley, "Probabilistic recognition of activity using local appearance", in ICVPR, pp. 104-109 June 1999.

[16]

A. D. Wilson, A. F. Bobick, "Realtime Online Adaptive Gesture Recognition," International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, pp. 111--117, 1999.

Digital Library

[17]

D. J. Moore, "I. A. Essa, M. H. Hayes, "Exploiting Human Actions and Object Context for Recognition Tasks," Proc. of ICCV, Vol. 1, pp. 80--86, 1999.

[18]

Milind Napahade, Ashutosh Garg and T. S. Huang, "Duration Dependent Input Output Markov Models for Audio-Visual Event Detection," ICME, 2001.

[19]

N. Oliver, A. Garg, E. Horvitz, "Layered Representation for Learning and Inferring Office Activity from Multiple Sensory Channels," Fourth IEEE Conference on Multimodal Interfaces, pp. 3-8, 2002.

Digital Library

[20]

N. Badler, "Temporal Scene Analysis: Conceptual Description of Object Movements," University of Toronto TR No. 80, 1975.

[21]

C. Stauffer and W.E.L Grimson. "Adaptive background mixture models for real-time tracking", Proc. of CVPR, 1999.

[22]

I. McCowan, D. Gatica-Perez, S. Bengio, G. Lathoud, M. Barnard and Dong Zhang, "Automatic Analysis of Multi-modal Group Actions in Meetings," IEEE Trans. on PAMI, 2004.

Digital Library

[23]

J. Yang, W. Lu, and A. Waibel, "Skin-color modeling and adaptation". In Proc. of ACCV, vol. II, pp. 687--694, 1998.

Digital Library

[24]

H. Hartley, "Maximum likelihood estimation from incomplete data". Bio-metrics, 14:174--194, 1958.

[25]

A. Doucet, N. de Freitas, and N. Gordon. Sequential Monte Carlo Methods in Practice. Springer-Verlag, 2001.

[26]

K. Nummiaro, E. Koller-Meier, and L. Van Gool. Object tracking with an adaptive color-based particle filter. In Proc. Symposium for Pattern Recognition of the DAGM, Sep. 2000.

Digital Library

[27]

P. Perez, A. Blake, and M. Gangnet. Jetstream: Probabilistic contour extraction with particles. Proc. of ICCV, pages 424--531, Vancouver, July 2001.

[28]

B. Clarkson and A. Pentland. Framing Through Peripheral Perception. Proc. of ICIP, Vancouver, September 2000.

[29]

B. Clarkson and A. Pentland. Unsupervised Clustering of Ambulatory Audio and Video. Proc. of the ICASSP, Phoenix, 1998.

Digital Library

[30]

J. Foote. Visualizing Music and Audio using Self-Similarity. Proc. of ACM Multimedia, Orlando, October 1999.

Digital Library

[31]

J. Foote. Automatic Audio Segmentation using a Measure of Audio Novelty, Proc. ICME, July 2000.

[32]

M. Siegler, U. Jain, B. Raj, R. Stern. Automatic Segmentation, Classification, and Clustering of Broadcast News Audio, Proc. of the 9th DARPA Spoken Language Systems Technology Workshop, New York, 1997.

[33]

S. Chen and P.S. Gopalakrishnan. Speaker, Environment, and Channel Change Detection and Clustering via the Bayesian Information Criterion, Proc. of the DARPA Broadcast News Transcription and Understanding Workshop, Virginia, 1998.

Cited By

Tocchetti ABrusadelli SFugini M(2024)Detecting Social Interaction2024 32nd International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE)10.1109/WETICE64632.2024.00018(50-55)Online publication date: 26-Jun-2024
https://doi.org/10.1109/WETICE64632.2024.00018
Curran KDougan T(2018)Detection of Social Interaction Using Mobile Phones via Device Free Passive LocalisationInternational Journal of Handheld Computing Research10.4018/IJHCR.20141001025:4(20-35)Online publication date: 16-Dec-2018
https://dl.acm.org/doi/10.4018/IJHCR.2014100102
Liu CChung YChung P(2018)An interaction-embedded HMM framework for human behavior understandingIEEE Transactions on Information Technology in Biomedicine10.1109/TITB.2010.205206114:5(1236-1246)Online publication date: 25-Dec-2018
https://dl.acm.org/doi/10.1109/TITB.2010.2052061
Show More Cited By

Index Terms

Multimodal detection of human interaction events in a nursing home environment
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Animation
      1. Motion capture
      2. Motion processing
    2. Shape modeling

Recommendations

Detecting social interactions of the elderly in a nursing home environment

Social interaction plays an important role in our daily lives. It is one of the most important indicators of physical or mental changes in aging patients. In this article, we investigate the problem of detecting social interaction patterns of patients ...
Towards automatic analysis of social interaction patterns in a nursing home environment from video
MIR '04: Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval

In this paper, we propose an ontology-based approach for analyzing social interaction patterns in a nursing home from video. Social interaction patterns are broken into individual activities and behavior events using a multi-level context hierarchy ...
Duration and Determinants of First Unplanned Hospitalization among Nursing Home Residents
ICMHI '18: Proceedings of the 2nd International Conference on Medical and Health Informatics

Background: Unplanned hospitalization is an important indicator of changes in nursing home care quality. Institutional residents have higher healthcare usage and hospitalization rates than do the elderly in the general population, leading to negative ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '04: Proceedings of the 6th international conference on Multimodal interfaces

October 2004

368 pages

ISBN:1581139950

DOI:10.1145/1027933

General Chairs:
Rajeev Sharma
Advanced Interfaces
,
Trevor Darrell
Massachusetts Institute of Technology
,
Program Chairs:
Mary Harper
Purdue University, West Lafayette, IN
,
Gianni Lazzari
ITC-IRST
,
Matthew Turk
University of California, Santa Barbara, CA

Copyright © 2004 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 October 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICMI04

Sponsor:

ICMI04: Sixth International Conference on Multimodal Interfaces 2004

October 13 - 15, 2004

PA, State College, USA

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
730
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tocchetti ABrusadelli SFugini M(2024)Detecting Social Interaction2024 32nd International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE)10.1109/WETICE64632.2024.00018(50-55)Online publication date: 26-Jun-2024
https://doi.org/10.1109/WETICE64632.2024.00018
Curran KDougan T(2018)Detection of Social Interaction Using Mobile Phones via Device Free Passive LocalisationInternational Journal of Handheld Computing Research10.4018/IJHCR.20141001025:4(20-35)Online publication date: 16-Dec-2018
https://dl.acm.org/doi/10.4018/IJHCR.2014100102
Liu CChung YChung P(2018)An interaction-embedded HMM framework for human behavior understandingIEEE Transactions on Information Technology in Biomedicine10.1109/TITB.2010.205206114:5(1236-1246)Online publication date: 25-Dec-2018
https://dl.acm.org/doi/10.1109/TITB.2010.2052061
Jaimes ASebe N(2018)Multimodal human-computer interactionComputer Vision and Image Understanding10.1016/j.cviu.2006.10.019108:1-2(116-134)Online publication date: 31-Dec-2018
https://dl.acm.org/doi/10.1016/j.cviu.2006.10.019
Grünerbl ABahle GHanser FLukowicz P(2013)UWB Indoor Location for Monitoring Dementia PatientsInternational Journal of Ambient Computing and Intelligence10.4018/ijaci.20131001045:4(45-59)Online publication date: Oct-2013
https://doi.org/10.4018/ijaci.2013100104
Zhou FJiao JChen SZhang D(2011)A Case-Driven Ambient Intelligence System for Elderly in-Home Assistance ApplicationsIEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews10.1109/TSMCC.2010.205245641:2(179-189)Online publication date: 1-Mar-2011
https://dl.acm.org/doi/10.1109/TSMCC.2010.2052456
Grunerbl ABahle GLukowicz PHanser F(2011)Using Indoor Location to Assess the State of Dementia PatientsProceedings of the 2011 Seventh International Conference on Intelligent Environments10.1109/IE.2011.22(32-39)Online publication date: 25-Jul-2011
https://dl.acm.org/doi/10.1109/IE.2011.22
Wai AHuang WFook VBiswas JChi-Chun HKoujuch L(2010)Situation-Aware Patient Monitoring in and around the Bed Using Multimodal Sensing IntelligenceProceedings of the 2010 Sixth International Conference on Intelligent Environments10.1109/IE.2010.31(128-133)Online publication date: 18-Jul-2010
https://dl.acm.org/doi/10.1109/IE.2010.31
ElSayed MAlsebai ASalaheldin AEl Gayar NElHelw M(2010)Ambient and wearable sensing for gait classification in pervasive healthcare environmentsThe 12th IEEE International Conference on e-Health Networking, Applications and Services10.1109/HEALTH.2010.5556563(240-245)Online publication date: Jul-2010
https://doi.org/10.1109/HEALTH.2010.5556563
Wu CLiu CChung P(2010)Assistance Instruments Detection Using Geometry Constrained Knowledge for Health Care Centers2010 5th International Conference on Future Information Technology10.1109/FUTURETECH.2010.5482662(1-5)Online publication date: May-2010
https://doi.org/10.1109/FUTURETECH.2010.5482662
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten