research-article

mConverse: inferring conversation episodes from respiratory measurements collected in the field

Authors:

Md. Mahbubur Rahman,

Amin Ahsan Ali,

Mustafa al'Absi,

Santosh KumarAuthors Info & Claims

WH '11: Proceedings of the 2nd Conference on Wireless Health

Article No.: 10, Pages 1 - 10

https://doi.org/10.1145/2077546.2077557

Published: 10 October 2011 Publication History

Abstract

Automated detection of social interactions in the natural environment has resulted in promising advances in organizational behavior, consumer behavior, and behavioral health. Progress, however, has been limited since the primary means of assessing social interactions today (i.e., audio recording) has several issues in field usage such as microphone occlusion, lack of speaker specificity, and high energy drain, in addition to significant privacy concerns.

In this paper, we present mConverse, a new mobile-based system to infer conversation episodes from respiration measurements collected in the field from an unobtrusively wearable respiratory inductive plethysmograph (RIP) band worn around the user's chest. The measurements are wire-lessly transmitted to a mobile phone, where they are used in a novel machine learning model to determine whether the wearer is speaking, listening, or quiet. Our model incorporates several innovations to address issues that naturally arise in the noisy field environment such as confounding events, poor data quality due to sensor loosening and detachment, losses in the wireless channel, etc. Our basic model obtains 83% accuracy for the three class classification. We formulate a Hidden Markov Model to further improve the accuracy to 87%. Finally, we apply our model to data collected from 22 subjects who wore the sensor for 2 full days in the field to observe conversation behavior in daily life and find that people spend 25% of their day in conversations.

References

[1]

http://mathforum.org/library/drmath/view/52720.html.

[2]

T. Choudhury. Sensing and modeling human networks. PhD thesis, Massachusetts Institute of Technology, February 2004.

Digital Library

[3]

E. Ertin, N. Stohs, S. Kumar, A. Raij, M. al'Absi, T. Kwon, S. Mitra, and S. Shah. AutoSense: Unobtrusively Wearable Sensor Suite for Inferencing of Onset, Causality, and Consequences of Stress in the Field. In ACM SenSys, 2011.

Digital Library

[4]

G. Forney Jr. The viterbi algorithm. Proceedings of the IEEE, 61(3):268--278, 1973.

[5]

M. Hall. Correlation-based Feature Selection for Machine Learning. PhD thesis, The University of Waikato, 1999.

[6]

J. Han, H. Shin, D. Jeong, and K. Park. Detection of apneic events from single channel nasal airflow using 2nd derivative method. Computer methods and programs in biomedicine, 91(3):199--207, 2008.

Digital Library

[7]

G. Iachello and K. Truong. Prototyping and sampling experience to evaluate ubiquitous computing privacy in the real world. In ACM CHI, pages 1009--1018, 2006.

Digital Library

[8]

T. Kim, M. Chu, O. Brdiczka, and J. Begole. Predicting shoppers' interest from social interactions using sociometric sensors. In ACM CHI Extended Abstracts, pages 4513--4518, 2009.

Digital Library

[9]

P. Klasnja, S. Consolvo, T. Choudhury, R. Beckwith, and J. Hightower. Exploring privacy concerns about personal sensing. Pervasive Computing, pages 176--183, 2009.

Digital Library

[10]

H. Lu, B. Brush, B. Priyantha, A. Karlson, and J. Liu. Speakersense: Energy efficient unobtrusive speaker identification on mobile phones. In Pervasive Computing, pages 188--205. Springer, 2011.

Digital Library

[11]

H. Lu, W. Pan, N. D. Lane, T. Choudhury, and A. T. Campbell. Soundsense: scalable sound sensing for people-centric applications on mobile phones. In ACM MobiSys, pages 165--178, 2009.

Digital Library

[12]

H. Lu, J. Yang, Z. Liu, N. Lane, T. Choudhury, and A. Campbell. The Jigsaw continuous sensing engine for mobile phone applications. In ACM SenSys, pages 71--84, 2010.

Digital Library

[13]

A. Madan, M. Cebrian, D. Lazer, and A. Pentland. Social sensing for epidemiological behavior change. In ACM UbiComp, pages 291--300, 2010.

Digital Library

[14]

A. Madan, S. Moturu, D. Lazer, and A. Pentland. Social sensing: obesity, unhealthy eating and exercise in face-to-face networks. In ACM Wireless Health, pages 104--110, 2010.

Digital Library

[15]

D. McFarland. Respiratory markers of conversational interaction. Journal of Speech, Language, and Hearing Research, 44(1):128--143, 2001.

[16]

M. Mun, S. Reddy, K. Shilton, N. Yau, J. Burke, D. Estrin, M. Hansen, E. Howard, R. West, and P. Boda. Peir, the personal environmental impact report, as a platform for participatory sensing systems research. In ACM MobiSys, pages 55--68, 2009.

Digital Library

[17]

D. Olguín and A. Pentland. Social sensors for automatic data collection. In Americas Conference on Information Systems, 2008.

[18]

A. Pentland. How Social Networks Network Best. Harvard Business Review, February, 2009.

[19]

K. Plarre, A. Raij, S. Guha, and S. Kumar. Automated detection of sensor detachments for physiological sensors in the wild. In ACM Wireless Health, 2010.

Digital Library

[20]

K. Plarre, A. Raij, S. Hossain, A. Ali, M. Nakajima, M. al'Absi, E. Ertin, T. Kamarck, S. Kumar, M. Scott, D. Siewiorek, A. Smailagic, and L. E. Wittmers. Continuous Inference of Psychological Stress From Sensory Measurements Collected in the Natural Environment. In ACM IPSN, 2011.

[21]

M. Rahman, A. A. Ali, A. Raij, E. Ertin, M. al'Absi, and S. Kumar. Online Detection of Speaking from Respiratory Measurements Collected in the Natural Environment. In ACM IPSN Demo Abstract, 2011.

[22]

P. Rainville, A. Bechara, N. Naqvi, and A. Damasio. Basic emotions are associated with distinct patterns of cardiorespiratory activity. International journal of psychophysiology, 61(1):5--18, 2006.

[23]

B. Todd and D. Andrews. The Identification of Peaks in Physiological Signals. Computers and biomedical research, 32(4):322--335, 1999.

Digital Library

[24]

P. Várady, T. Micsik, S. Benedek, and Z. Benyó. A novel method for the detection of apnea and hypopnea events in respiration signals. Biomedical Engineering, IEEE Transactions on, 49(9):936--942, 2002.

[25]

A. Vinciarelli, M. Pantic, H. Bourlard, and A. Pentland. Social signal processing: state-of-the-art and future perspectives of an emerging domain. In ACM international conference on Multimedia, pages 1061--1070, 2008.

Digital Library

[26]

B. N. Waber, D. O. Olguin, T. Kim, and A. Pentland. Productivity Through Coffee Breaks: Changing Social Networks by Changing Break Structure. In 30th International Sunbelt Social Network Conference, 2010.

[27]

Y. Wang, J. Lin, M. Annavaram, Q. Jacobson, J. Hong, B. Krishnamachari, and N. Sadeh. A framework of energy efficient mobile sensing for automatic user state recognition.

[28]

F. Wilhelm, E. Handke, and W. Roth. Detection of speaking with a new respiratory inductive plethysmography system. Biomedical Sciences Instrumentation, 39:136, 2003.

[29]

A. J. Wilson, C. I. Franks, and I. L. Freeston. Algorithms for the detection of breaths from respiratory waveform recordings of infants. Medical and Biological Engineering and Computing, 20(3):286--292, 1982.

[30]

I. Witten. Weka: Practical machine learning tools and techniques with Java implementations. Dept. of Computer Science, University of Waikato, 1999.

Digital Library

[31]

I. Witten and E. Frank. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann Pub, 2005.

Digital Library

[32]

D. Wyatt, T. Choudhury, J. Bilmes, and J. Kitts. Inferring colocation and conversation networks from privacy-sensitive audio with implications for computational social science. Transactions on Intelligent Systems and Technology (TIST), 2(1):7, 2011.

Digital Library

Cited By

Liang DZhang AThomaz E(2023)Automated Face-To-Face Conversation Detection on a Commodity Smartwatch with Acoustic SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108827:3(1-29)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610882
Rashid NRahman MAhmed TKuang JGao J(2023)BreathIE: Estimating Breathing Inhale Exhale Ratio Using Motion Sensor Data from Consumer EarbudsICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP49357.2023.10096084(1-5)Online publication date: 4-Jun-2023
https://doi.org/10.1109/ICASSP49357.2023.10096084
Coppers SVanacken DLuyten K(2020)FORTNIoTProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34322254:4(1-24)Online publication date: 18-Dec-2020
https://dl.acm.org/doi/10.1145/3432225
Show More Cited By

Index Terms

mConverse: inferring conversation episodes from respiratory measurements collected in the field
1. Applied computing
  1. Law, social and behavioral sciences

Recommendations

Elderly Assistance Using Wearable Sensors by Detecting Fall and Recognizing Fall Patterns
UbiComp '18: Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers

Falling is a serious threat to the elderly people. One severe fall can cause hazardous problems like bone fracture or may lead to some permanent disability or even death. Thus, it has become the need of the hour to continuously monitor the activities of ...
Activity recognition with hand-worn magnetic sensors

Activity recognition is a key technology for realizing ambient assisted living applications such as care of the elderly and home automation. This paper proposes a new activity recognition method that employs hand-worn magnetic sensors to recognize a ...
Simultaneous Activity and Respiratory Monitoring Using an Accelerometer
BSN '11: Proceedings of the 2011 International Conference on Body Sensor Networks

Simultaneous monitoring of respiratory function and activity level would be of benefit in the monitoring of chronic conditions such as chronic obstructive pulmonary disorder (COPD), but is ill-addressed by existing methods. Current solutions for ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

WH '11: Proceedings of the 2nd Conference on Wireless Health

October 2011

170 pages

ISBN:9781450309820

DOI:10.1145/2077546

General Chairs:
Irwin Mark Jacobs
Qualcomm Incorporated
,
Patrick Soon-Shiong
UCLA Wireless Health Institute
,
Eric Topol
West Wireless Institute
,
Chris Toumazou
Institute of Biomedical Engineering, Imperial College London

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

University of California: University of California

In-Cooperation

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

WH '11

Sponsor:

University of California

WH '11: Wireless Health 2011

October 10 - 13, 2011

California, San Diego

Acceptance Rates

Overall Acceptance Rate 35 of 139 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

57
Total Citations
View Citations
355
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)2

Reflects downloads up to 28 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liang DZhang AThomaz E(2023)Automated Face-To-Face Conversation Detection on a Commodity Smartwatch with Acoustic SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108827:3(1-29)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610882
Rashid NRahman MAhmed TKuang JGao J(2023)BreathIE: Estimating Breathing Inhale Exhale Ratio Using Motion Sensor Data from Consumer EarbudsICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP49357.2023.10096084(1-5)Online publication date: 4-Jun-2023
https://doi.org/10.1109/ICASSP49357.2023.10096084
Coppers SVanacken DLuyten K(2020)FORTNIoTProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34322254:4(1-24)Online publication date: 18-Dec-2020
https://dl.acm.org/doi/10.1145/3432225
Cai HKorany BKaranam CMostofi Y(2020)Teaching RF to Sense without RF Training MeasurementsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34322244:4(1-22)Online publication date: 18-Dec-2020
https://dl.acm.org/doi/10.1145/3432224
Ruensuk MCheon EHong HOakley I(2020)How Do You Feel OnlineProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34322234:4(1-32)Online publication date: 18-Dec-2020
https://dl.acm.org/doi/10.1145/3432223
Hafiz PMiskowiak KMaxhuni AKessing LBardram J(2020)Wearable Computing Technology for Assessment of Cognitive Functioning of Bipolar Patients and Healthy ControlsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34322194:4(1-22)Online publication date: 18-Dec-2020
https://dl.acm.org/doi/10.1145/3432219
Zeng SWu RTruong KChevalier F(2020)Evaluating the Gradual Delivery of Knowledge-focused and Mindset-focused Messages for Facilitating the Acceptance of COPDProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34322184:4(1-23)Online publication date: 18-Dec-2020
https://dl.acm.org/doi/10.1145/3432218
Zhu HJin WXiao MMurali SLi M(2020)BlinKeyProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34322174:4(1-29)Online publication date: 18-Dec-2020
https://dl.acm.org/doi/10.1145/3432217
Jin WXiao MZhu HDeb SKan CLi M(2020)AcoussistProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34322164:4(1-30)Online publication date: 18-Dec-2020
https://dl.acm.org/doi/10.1145/3432216
Shao QSniffen ABlanchet JHillis MShi XHaris TLiu JLamberton JMalzkuhn MQuandt LMahoney JKraemer DZhou XBalkcom D(2020)Teaching American Sign Language in Mixed RealityProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34322114:4(1-27)Online publication date: 18-Dec-2020
https://dl.acm.org/doi/10.1145/3432211
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents