research-article

Open access

Exploring Opportunities for Multimodality and Multiple Devices in Food Journaling

Authors:

Lucas M. Silva,

Elizabeth A. Ankrah,

Daniel A. EpsteinAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 7, Issue MHCI

Article No.: 209, Pages 1 - 27

https://doi.org/10.1145/3604256

Published: 13 September 2023 Publication History

Abstract

Digital food journaling can support personal goals, such as weight loss and developing healthy eating behaviors. However, traditional manual tracking demands great effort, often leading to lapses or abandonment. We explore opportunities for journaling with multiple input modalities and devices, leveraging people's daily interactions with a range of technologies. We report on an extended analysis of 15 participants' experiences with ModEat, a prototype supporting journaling with several input modalities on phone, computer, and voice assistants. Participants' modality and device preferences were largely influenced by their goals, but they frequently deviated from those preferences depending on device availability, perceived affordances, and characteristics of foods eaten. Participants rarely combined input modalities in entries, but some described that doing so allowed for more detailed journaling or serve as a placeholder for later. We discuss advantages and drawbacks of multimodal tracking and potential strategies for improving interactions.

References

[1]

Tawfiq Ammari, Jofish Kaye, Janice Y. Tsai, and Frank Bentley. 2019. Music, Search, and IoT: How People (Really) Use Voice Assistants. ACM Trans. Comput.-Hum. Interact., Vol. 26, 3, Article 17 (apr 2019), 28 pages. https://doi.org/10.1145/3311956

Digital Library

[2]

Adrienne H. Andrew, Gaetano Borriello, and James Fogarty. 2013. Simplifying Mobile Phone Diaries: Design and Evaluation of a Food Index-Based Nutrition Diary. In Proceedings of the International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth 2013). 260--263. https://doi.org/bbkk

[3]

Ate. 2023. Ate - the Mindful Food Journal. https://youate.com/ Retrieved June 26, 2023 from

[4]

Amid Ayobi, Paul Marshall, and Anna L. Cox. 2020. Trackly: A Customisable and Pictorial Self-Tracking App to Support Agency in Multiple Sclerosis Self-Care. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2020). ACM, New York, NY, USA, 1--15. https://doi.org/10.1145/3313831.3376809

Digital Library

[5]

Elizabeth Barrett-Connor. 1991. Nutrition epidemiology: How do we know what they ate?. In American Journal of Clinical Nutrition, Vol. 54. 182--189. https://doi.org/10.1093/ajcn/54.1.182s

[6]

Eric P.S. Baumer, Sherri Jean Katz, Jill E. Freeman, Phil Adams, Amy L. Gonzales, John Pollak, Daniela Retelny, Jeff Niederdeppe, Christine M. Olson, and Geri K. Gay. 2012. Prescriptive persuasion and open-ended social awareness: Expanding the design space of mobile health. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW 2012). 475--484. https://doi.org/10.1145/2145204.2145279

Digital Library

[7]

Eric P.S. Baumer, Vera Khovanskaya, Mark Matthews, Lindsay Reynolds, Victoria Schwanda Sosik, and Geri Gay. 2014. Reviewing reflection: On the use of reflection in interactive system design. In Proceedings of the Conference on Designing Interactive Systems (DIS 2014). 93--102. https://doi.org/10.1145/2598510.2598598

Digital Library

[8]

Abdelkareem Bedri, Diana Li, Rushil Khurana, Kunal Bhuwalka, and Mayank Goel. 2020. FitByte: Automatic Diet Monitoring in Unconstrained Situations Using Multimodal Sensing on Eyeglasses. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2020), Vol. 20. ACM, New York, NY, USA, 1--12. https://doi.org/10.1145/3313831.3376869

Digital Library

[9]

Abdelkareem Bedri, Richard Li, Malcolm Haynes, Raj Prateek Kosaraju, Ishaan Grover, Temiloluwa Prioleau, Min Yan Beh, Mayank Goel, Thad Starner, and Gregory Abowd. 2017. EarBit: Using Wearable Sensors to Detect Eating Episodes in Unconstrained Environments. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 1, 3, Article 37 (sep 2017), 20 pages. https://doi.org/10.1145/3130902

Digital Library

[10]

Joan-Isaac Biel, Nathalie Martin, David Labbe, and Daniel Gatica-Perez. 2018. Bites?n'Bits: Inferring Eating Behavior from Contextual Mobile Data. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT 2018), Vol. 1, 4 (jan 2018), 1--33. https://doi.org/10.1145/3161161

Digital Library

[11]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology, Vol. 3, 2 (2006), 77--101. https://doi.org/10.1191/1478088706qp063oa

[12]

Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, Vol. 11, 4 (aug 2019), 589--597. https://doi.org/10.1080/2159676X.2019.1628806

[13]

Lora E. Burke, Molly B. Conroy, Susan M. Sereika, Okan U. Elci, Mindi A. Styn, Sushama D. Acharya, Mary A. Sevick, Linda J. Ewing, and Karen Glanz. 2011. The effect of electronic self-monitoring on weight loss and dietary intake: A randomized behavioral weight loss trial. Obesity, Vol. 19, 2 (feb 2011), 338--344. https://doi.org/10.1038/oby.2010.208

[14]

Michelle Clare Carter, Victoria Jane Burley, Camilla Nykjaer, and Janet Elizabeth Cade. 2013. Adherence to a smartphone application for weight loss compared to website and paper diary: Pilot randomized controlled trial. Journal of Medical Internet Research, Vol. 15, 4 (apr 2013), e32. https://doi.org/10.2196/jmir.2283

[15]

Pei-Yu (Peggy) Chi and Yang Li. 2015. Weave: Scripting Cross-Device Wearable Interaction. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI '15). ACM, New York, NY, USA, 3923--3932. https://doi.org/10.1145/2702123.2702451

Digital Library

[16]

Eun Kyoung Choe, Saeed Abdullah, Mashfiqui Rabbi, Edison Thomaz, Daniel A. Epstein, Felicia Cordeiro, Matthew Kay, Gregory D. Abowd, Tanzeem Choudhury, James Fogarty, Bongshin Lee, Mark Matthews, and Julie A. Kientz. 2017. Semi-Automated Tracking: A Balanced Approach for Self-Monitoring Applications. IEEE Pervasive Computing, Vol. 16, 1 (jan 2017), 74--84. https://doi.org/10.1109/MPRV.2017.18

Digital Library

[17]

Eun Kyoung Choe, Nicole B. Lee, Bongshin Lee, Wanda Pratt, and Julie A. Kientz. 2014. Understanding quantified-selfers' practices in collecting and exploring personal data. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1143--1152. https://doi.org/10.1145/2556288.2557372

Digital Library

[18]

Keum San Chun, Sarnab Bhattacharya, and Edison Thomaz. 2018. Detecting Eating Episodes by Tracking Jawbone Movements with a Non-Contact Wearable Sensor. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT 2018), Vol. 2, 1 (2018), 1--21. https://doi.org/10.1145/3191736

Digital Library

[19]

Chia-Fang Chung, Qiaosi Wang, Jessica Schroeder, Allison Cole, Jasmine Zia, James Fogarty, and Sean A. Munson. 2019. Identifying and Planning for Individualized Change. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT 2019), Vol. 3, 1 (2019), 1--27. https://doi.org/10.1145/3314394

Digital Library

[20]

James Clawson, Jessica A Pater, Andrew D Miller, Elizabeth D Mynatt, and Lena Mamykina. 2015. No longer wearing. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2015). ACM Press, New York, New York, USA, 647--658. https://doi.org/10.1145/2750858.2807554

Digital Library

[21]

Nathalie Colineau and Cé cile Paris. 2011. Motivating reflection about health within the family: The use of goal setting and tailored feedback. User Modeling and User-Adapted Interaction, Vol. 21, 4--5 (2011), 341--376. https://doi.org/10.1007/s11257-010--9089-x

[22]

Felicia Cordeiro, Elizabeth Bales, Erin Cherry, and James Fogarty. 2015a. Rethinking the Mobile Food Journal. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2015), Vol. 2015-April. ACM, New York, NY, USA, 3207--3216. https://doi.org/10.1145/2702123.2702154

Digital Library

[23]

Felicia Cordeiro, Daniel A. Epstein, Edison Thomaz, Elizabeth Bales, Arvind K. Jagannathan, Gregory D. Abowd, and James Fogarty. 2015b. Barriers and Negative Nudges: Exploring Challenges in Food Journaling. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2015), Vol. 2015-April. ACM, New York, NY, USA, 1159--1162. https://doi.org/10.1145/2702123.2702155

Digital Library

[24]

Mayara Costa Figueiredo, Clara Caldeira, Tera L. Reynolds, Sean Victory, Kai Zheng, and Yunan Chen. 2017. Self-Tracking for Fertility Care: Collaborative Support for a Highly Personalized Problem. Proc. ACM Hum.-Comput. Interact., Vol. 1, CSCW, Article 36 (dec 2017), 21 pages. https://doi.org/10.1145/3134671

Digital Library

[25]

Mary Ruth Craig, Alan R. Kristal, Carrie L. Cheney, and Ann L. Shattuck. 2000. The prevalence and impact of 'atypical' days in 4-day food records. Journal of the American Dietetic Association, Vol. 100, 4 (apr 2000), 421--427. https://doi.org/10.1016/S0002--8223(00)00130--9

[26]

Alaina Darby, Matthew W Strum, Erin Holmes, and Justin Gatwood. 2016. A Review of Nutritional Tracking Mobile Applications for Diabetes Patient Use. Diabetes Technology & Therapeutics, Vol. 18, 3 (mar 2016), 200--212. https://doi.org/10.1089/dia.2015.0299

[27]

Pooja M Desai, Elliot G Mitchell, Maria L Hwang, Matthew E Levine, David J Albers, and Lena Mamykina. 2019. Personal Health Oracle: Explorations of Personalized Predictions in Diabetes Self-Management. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2019). ACM, New York, NY, USA, 1--13. https://doi.org/10.1145/3290605.3300600

Digital Library

[28]

Linda Di Geronimo, Maria Husmann, and Moira C Norrie. 2016. Surveying personal device ecosystems with cross-device applications in mind. In PerDis 2016 - Proceedings of the 5th ACM International Symposium on Pervasive Displays. 220--227. https://doi.org/10.1145/2914920.2915028

Digital Library

[29]

Tao Dong, Elizabeth F Churchill, and Jeffrey Nichols. 2016. Understanding the Challenges of Designing and Developing Multi-Device Experiences. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems. ACM, New York, NY, USA, 62--72. https://doi.org/10.1145/2901790.2901851

Digital Library

[30]

Daniel A Epstein, Monica Caraway, Chuck Johnston, An Ping, James Fogarty, and Sean A Munson. 2016a. Beyond Abandonment to Next Steps. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2016). ACM, New York, NY, USA, 1109--1113. https://doi.org/10.1145/2858036.2858045

Digital Library

[31]

Daniel A. Epstein, Felicia Cordeiro, James Fogarty, Gary Hsieh, and Sean A. Munson. 2016b. Crumbs: Lightweight Daily Food Challenges to Promote Engagement and Mindfulness. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). New York, NY, USA, 5632--5644. https://doi.org/10.1145/2858036.2858044

Digital Library

[32]

Daniel A. Epstein, Bradley H. Jacobson, Elizabeth Bales, David W. McDonald, and Sean A. Munson. 2015a. From "nobody cares" to "way to go!". In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing - CSCW '15. New York, New York, USA, 1622--1636. https://doi.org/10.1145/2675133.2675135

Digital Library

[33]

Daniel A. Epstein, An Ping, James Fogarty, and Sean A. Munson. 2015b. A Lived Informatics Model of Personal Informatics. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2015). Association for Computing Machinery, Inc, New York, New York, USA, 731--742. https://doi.org/10.1145/2750858.2804250

Digital Library

[34]

Giannina Ferrara, Jenna Kim, Shuhao Lin, Jenna Hua, and Edmund Seto. 2019. A Focused Review of Smartphone Diet-Tracking Apps: Usability, Functionality, Coherence With Behavior Change Theory, and Comparative Validity of Nutrient Intake and Energy Estimates. JMIR mHealth and uHealth, Vol. 7, 5 (may 2019), e9232. https://doi.org/10.2196/mhealth.9232

[35]

Celia Framson, Alan R. Kristal, Jeannette M. Schenk, Alyson J. Littman, Steve Zeliadt, and Denise Benitez. 2009. Development and Validation of the Mindful Eating Questionnaire. Journal of the American Dietetic Association, Vol. 109, 8 (aug 2009), 1439--1444. https://doi.org/10.1016/j.jada.2009.05.006

[36]

Elliot G. Mitchell, Elizabeth M. Heitkemper, Marissa Burgermaster, Matthew E. Levine, Yishen Miao, Maria L. Hwang, Pooja M. Desai, Andrea Cassells, Jonathan N. Tobin, Esteban G. Tabak, David J. Albers, Arlene M. Smaldone, and Lena Mamykina. 2021. From Reflection to Action: Combining Machine Learning with Expert Knowledge for Nutrition Goal Recommendations. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI '21). ACM, New York, NY, USA, Article 206, 17 pages. https://doi.org/10.1145/3411764.3445555

Digital Library

[37]

Daniel Harrison, Paul Marshall, Nadia Bianchi-Berthouze, and Jon Bird. 2015. Activity tracking. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2015). ACM Press, New York, New York, USA, 617--621. https://doi.org/10.1145/2750858.2805832

Digital Library

[38]

William D. Heizer, Susannah Southern, and Susan McGovern. 2009. The Role of Diet in Symptoms of Irritable Bowel Syndrome in Adults: A Narrative Review. Journal of the American Dietetic Association, Vol. 109, 7 (jul 2009), 1204--1214. https://doi.org/10.1016/j.jada.2009.04.012

[39]

Jack F. Hollis, Christina M. Gullion, Victor J. Stevens, Phillip J. Brantley, Lawrence J. Appel, Jamy D. Ard, Catherine M. Champagne, Arlene Dalcin, Thomas P. Erlinger, Kristine Funk, Daniel Laferriere, Pao Hwa Lin, Catherine M. Loria, Carmen Samuel-Hodge, William M. Vollmer, and Laura P. Svetkey. 2008. Weight Loss During the Intensive Intervention Phase of the Weight-Loss Maintenance Trial. American Journal of Preventive Medicine, Vol. 35, 2 (aug 2008), 118--126. https://doi.org/10.1016/j.amepre.2008.04.013

[40]

Home Food Safety. 2011. Desktop Dining Survey: 2011 Results Americans' Food Safety Knowledge and Practice at Work. Technical Report. www.homefoodsafety.org

[41]

Jasmine K. Zia, Chia-Fang Chung, Jessica Schroeder, Sean A. Munson, Julie A. Kientz, James Fogarty, Elizabeth Bales, Jeanette M. Schenk and Margaret M. Heitkemper. 2017. The feasibility, usability, and clinical utility of traditional paper food and symptom journals for patients with irritable bowel syndrome J. Neurogastroenterology & Motility, Vol. 29, 2 (feb 2017), e12935. https://doi.org/10.1111/nmo.12935

[42]

Eunkyung Jo, Hyeonseok Bang, Myeonghan Ryu, Eun Jee Sung, Sungmook Leem, and Hwajung Hong. 2020. MAMAS: Supporting Parent--Child Mealtime Interactions Using Automated Tracking and Speech Recognition. Proceedings of the ACM on Human-Computer Interaction, Vol. 4, CSCW1 (may 2020), 1--32. https://doi.org/10.1145/3392876

Digital Library

[43]

Tero Jokela, Jarno Ojala, and Thomas Olsson. 2015. A Diary Study on Combining Multiple Information Devices in Everyday Activities and Tasks. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2015), Vol. 2015-April. ACM, New York, NY, USA, 3903--3912. https://doi.org/10.1145/2702123.2702211

Digital Library

[44]

David Jones. 2016. Interview: How the myfitnesspal app got 165 million users. https://www.linkedin.com/pulse/interview-how-myfitnesspal-app-got-165-million-users-david-jones

[45]

Jisu Jung, Lyndal Wellard-Cole, Colin Cai, Irena Koprinska, Kalina Yacef, Margaret Allman-Farinelli, and Judy Kay. 2020. Foundations for Systematic Evaluation and Benchmarking of a Mobile Food Logger in a Large-scale Nutrition Study. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT 2020), Vol. 4, 2 (jun 2020), 1--25. https://doi.org/10.1145/3397327

Digital Library

[46]

Ravi Karkar, Jessica Schroeder, Daniel A. Epstein, Laura R. Pina, Jeffrey Scofield, James Fogarty, Julie A. Kientz, Sean A. Munson, Roger Vilardaga, and Jasmine Zia. 2017. TummyTrials: A Feasibility Study of Using Self-Experimentation to Detect Individualized Food Triggers. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17). ACM, New York, NY, USA, 6850--6863. https://doi.org/10.1145/3025453.3025480

Digital Library

[47]

Fahim Kawsar and J. Bernheim Brush. 2013. Home computing unplugged: Why, where and when people use different connected devices at home. In UbiComp 2013 - Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 627--636. https://doi.org/10.1145/2493432.2493494

Digital Library

[48]

Da Jung Kim, Yeoreum Lee, Saeyeong Rho, and Youn Kyung Lim. 2016. Design opportunities in three stages of relationship development between users and self-tracking devices. In Conference on Human Factors in Computing Systems - Proceedings. ACM, 699--703. https://doi.org/10.1145/2858036.2858148

Digital Library

[49]

Young-Ho Kim, Jae Ho Jeon, Bongshin Lee, Eun Kyoung Choe, and Jinwook Seo. 2017. OmniTrack: A Flexible Self-Tracking Approach Leveraging Semi-Automated Tracking. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT 2017), Vol. 1, 3 (2017), 1--28. https://doi.org/10.1145/3130930

Digital Library

[50]

Young-Ho Kim, Bongshin Lee, Arjun Srinivasan, Eun Kyoung Choe, Young-Ho Kim, Bongshin Lee, Arjun Srinivasan, and Eun Kyoung Choe. 2021. Data@Hand: Fostering Visual Exploration of Personal Data on Smartphones Leveraging Speech and Touch Interaction. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2021) (CHI '21), Vol. 21 (jan 2021). https://doi.org/10.1145/3411764.3445421 arxiv: 2101.06283

Digital Library

[51]

Ahmet Baki Kocaballi, Emre Sezgin, Leigh Clark, John M Carroll, Yungui Huang, Jina Huh-Yoo, Junhan Kim, Rafal Kocielnik, Yi-Chieh Lee, Lena Mamykina, Elliot G Mitchell, Robert J Moore, Prasanth Murali, Elizabeth D Mynatt, Sun Young Park, Alessandro Pasta, Deborah Richards, Lucas M Silva, Diva Smriti, Brendan Spillane, Zhan Zhang, and Tamara Zubatiy. 2022. Design and Evaluation Challenges of Conversational Agents in Health Care and Well-being: Selective Review Study. J Med Internet Res, Vol. 24, 11 (15 Nov 2022), e38525. https://doi.org/10.2196/38525

[52]

Alexander Kunst. 2019. E-health application categories used by U.S. adults 2017. https://www.statista.com/statistics/378850/top-mobile-health-application-categories-used-by-us-consumers/

[53]

Liliana Laranjo, Adam G. Dunn, Huong Ly Tong, Ahmet Baki Kocaballi, Jessica Chen, Rabia Bashir, Didi Surian, Blanca Gallego, Farah Magrabi, Annie Y.S. Lau, and Enrico Coiera. 2018. Conversational agents in healthcare: A systematic review. Journal of the American Medical Informatics Association, Vol. 25, 9 (2018), 1248--1258. https://doi.org/10.1093/jamia/ocy072

[54]

Amanda Lazar, Christian Koehler, Joshua Tanenbaum, and David H Nguyen. 2015. Why we use and abandon smart devices. In UbiComp 2015 - Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 635--646. https://doi.org/10.1145/2750858.2804288

Digital Library

[55]

Jong Ho Lee, Jessica Schroeder, and Daniel A. Epstein. 2021. Understanding and Supporting Self-Tracking App Selection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT 2021), Vol. 5, 4 (dec 2021), 1--25. https://doi.org/10.1145/3494980

Digital Library

[56]

Uichin Lee, Kyungsik Han, Hyunsung Cho, Kyong-Mee Chung, Hwajung Hong, Sung-Ju Lee, Youngtae Noh, Sooyoung Park, and John M. Carroll. 2019. Intelligent positive computing with mobile, wearable, and IoT devices: Literature review and research directions. Ad Hoc Networks, Vol. 83 (feb 2019), 8--24. https://doi.org/10.1016/j.adhoc.2018.08.021

[57]

Ian Li, Anind Dey, and Jodi Forlizzi. 2010. A Stage-Based Model of Personal Informatics Systems. Conference on Human Factors in Computing Systems - Proceedings, Vol. 1 (2010), 557--566. https://doi.org/10.1145/1753326.1753409

Digital Library

[58]

Brian Y Lim, Xinni Chng, and Shengdong Zhao. 2017. Trade-off between Automation and Accuracy in Mobile Photo Recognition Food Logging. In Proceedings of the Fifth International Symposium of Chinese CHI on - Chinese CHI 2017, Vol. Part F1283. ACM Press, New York, New York, USA, 53--59. https://doi.org/10.1145/3080631.3080640

Digital Library

[59]

Xi Lu, Yunan Chen, and Daniel A. Epstein. 2021. How Cultural Norms Influence Persuasive Design: A Study on Chinese Food Journaling Apps. DIS 2021 - Proceedings of the 2021 ACM Designing Interactive Systems Conference: Nowhere and Everywhere (Jun 2021), 619--637. https://doi.org/10.1145/3461778.3462142

Digital Library

[60]

Xi Lu, Edison Thomaz, and Daniel A. Epstein. 2022. Understanding People's Perceptions of Approaches to Semi-Automated Dietary Monitoring. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, 3, Article 129 (Sep 2022), 27 pages. https://doi.org/10.1145/3550288

Digital Library

[61]

Ewa Luger and Abigail Sellen. 2016. Like Having a Really Bad PA. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2016). ACM, New York, NY, USA, 5286--5297. https://doi.org/10.1145/2858036.2858288

Digital Library

[62]

Kai Lukoff, Taoxi Li, Yuan Zhuang, and Brian Y. Lim. 2018. TableChat: Mobile Food Journaling to Facilitate Family Support for Healthy Eating. Proceedings of the ACM on Human-Computer Interaction, Vol. 2, CSCW (Nov 2018), 1--28. https://doi.org/10.1145/3274383

Digital Library

[63]

Yuhan Luo. 2021. Designing Multimodal Self-Tracking Technologies to Promote Data Capture and Self-Reflection. In Companion Publication of the 2021 ACM Designing Interactive Systems Conference (Virtual Event, USA) (DIS '21 Companion). New York, NY, USA, 11--15. https://doi.org/10.1145/3468002.3468232

Digital Library

[64]

Yuhan Luo, Young Ho Kim, Bongshin Lee, Naeemul Hassan, and Eun Kyoung Choe. 2021. FoodScrap: Promoting Rich Data Capture and Reflective Food Journaling through Speech Input. In DIS 2021 - Proceedings of the 2021 ACM Designing Interactive Systems Conference: Nowhere and Everywhere, Vol. 13. ACM, New York, NY, USA, 606--618. https://doi.org/10.1145/3461778.3462074

Digital Library

[65]

Yuhan Luo, Bongshin Lee, and Eun Kyoung Choe. 2020. TandemTrack: Shaping Consistent Exercise Experience by Complementing a Mobile App with a Smart Speaker. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2020). ACM, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376616

Digital Library

[66]

Lucas M. Silva and Daniel A. Epstein. 2021. Investigating Preferred Food Description Practices in Digital Food Journaling. In Proceedings of the ACM Designing Interactive Systems Conference: Nowhere and Everywhere (DIS 2021). ACM, New York, NY, USA, 589--605. https://doi.org/10.1145/3461778.3462145

Digital Library

[67]

Lena Mamykina, Elizabeth Mynatt, Patricia Davidson, and Daniel Greenblatt. 2008. MAHI: Investigation of Social Scaffolding for Reflective Thinking in Diabetes Management. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2008). ACM Press, New York, New York, USA, 477. https://doi.org/10.1145/1357054.1357131

Digital Library

[68]

Laura Maria, Christiane Attig, Thomas Franke, Britta Renner, Laura Maria Kö nig, Christiane Attig, Thomas Franke, and Britta Renner. 2021. Barriers to and Facilitators for Using Nutrition Apps: Systematic Review and Conceptual Framework. JMIR Mhealth Uhealth 2021;9(6):e20037 https://mhealth.jmir.org/2021/6/e20037, Vol. 9, 6 (Jun 2021), e20037. https://doi.org/10.2196/20037

[69]

Weiqing Min, Shuqiang Jiang, Linhu Liu, Yong Rui, and Ramesh Jain. 2019. A Survey on Food Computing. Comput. Surveys, Vol. 52, 5 (Oct 2019), 1--36. https://doi.org/10.1145/3329168

Digital Library

[70]

Manuel Mü ller, Morgan Harvey, David Elsweiler, and Stefanie Mika. 2012. Ingredient matching to determine the nutritional properties of internet-sourced recipes. In 2012 6th International Conference on Pervasive Computing Technologies for Healthcare and Workshops, PervasiveHealth 2012. https://doi.org/10.4108/icst.pervasivehealth.2012.248681

[71]

MyFitnessPal. 2023. MyFitnessPal: Calorie Counter, Diet & Exercise Journal. https://www.myfitnesspal.com/ Retrieved June 26, 2023 from

[72]

Michael Nebeling. 2017. XDBrowser 2.0: Semi-Automatic Generation of Cross-Device Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2017), Vol. 2017-May. ACM, New York, NY, USA, 4574--4584. https://doi.org/10.1145/3025453.3025547

Digital Library

[73]

Michael Nebeling and Anind K. Dey. 2016. XDBrowser: User-Defined Cross-Device Web Page Designs. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). Association for Computing Machinery, New York, NY, USA, 5494--5505. https://doi.org/10.1145/2858036.2858048

Digital Library

[74]

Jasmin Niess and Paweł W. Wo'z niak. 2018. Supporting Meaningful Personal Fitness: The Tracker Goal Evolution Model. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2018), Vol. 2018-April. ACM Press, New York, New York, USA, 1--12. https://doi.org/10.1145/3173574.3173745

Digital Library

[75]

Jon Noronha, Eric Hysen, Haoqi Zhang, and Krzysztof Z Gajos. 2011. PlateMate: Crowdsourcing Nutrition Analysis from Food Photographs. In Crowdsourcing Nutrition Analysis from Food Photographs. Proceedings of the ACM symposium on User interface software and technology (UIST 2011). ACM Press, New York, New York, USA, 1. https://doi.org/10.1145/2047196.2047198

Digital Library

[76]

NSDR. 2023. Nutrition Data System for Research (NSDR). http://www.ncc.umn.edu/products/ Retrieved June 26, 2023 from

[77]

Katie O'Leary, Tao Dong, Julia Katherine Haines, Michael Gilbert, Elizabeth F Churchill, and Jeffrey Nichols. 2017. The Moving Context Kit: Designing for Context Shifts in Multi-device Experiences. In DIS 2017 - Proceedings of the 2017 ACM Conference on Designing Interactive Systems. ACM, New York, NY, USA, 309--320. https://doi.org/10.1145/3064663.3064768

Digital Library

[78]

G. A. O'Reilly, L. Cook, D. Spruijt-Metz, and D. S. Black. 2014. Mindfulness-based interventions for obesity-related eating behaviours: A literature review. Obesity Reviews (2014). https://doi.org/10.1111/obr.12156

[79]

Laura Pina, Sang-Wha Sien, Clarissa Song, Teresa M. Ward, James Fogarty, Sean A. Munson, and Julie A. Kientz. 2020. DreamCatcher: Exploring How Parents and School-Age Children can Track and Review Sleep Information Together. Proceedings of the ACM on Human-Computer Interaction, Vol. 4, CSCW1 (may 2020), 1--25. https://doi.org/10.1145/3392882

Digital Library

[80]

Laura R. Pina, Sang Wha Sien, Teresa Ward, Jason C. Yip, Sean A. Munson, James Fogarty, and Julie A. Kientz. 2017. From Personal Informatics to Family Informatics: Understanding Family Practices around Health Monitoring. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW 2017). ACM, New York, New York, USA, 2300--2315. https://doi.org/10.1145/2998181.2998362

Digital Library

[81]

Martin Porcheron, Joel E. Fischer, Stuart Reeves, and Sarah Sharples. 2018. Voice Interfaces in Everyday Life. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2018), Vol. 2018-April. ACM, New York, NY, USA, 1--12. https://doi.org/10.1145/3173574.3174214

Digital Library

[82]

Mashfiqui Rabbi, Min Hane Aung, Mi Zhang, and Tanzeem Choudhury. 2015. MyBehavior. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing - UbiComp '15. ACM Press, New York, New York, USA, 707--718. https://doi.org/10.1145/2750858.2805840

Digital Library

[83]

Mirana Randriambelonoro. 2022. Assessing Activity of Daily Living through Technology-Enabled Tools: Mobility and Nutrition Assessment: MiranaBot: A Nutrition Assessment Use Case. Health Informatics (2022), 27--47. https://doi.org/10.1007/978--3-030--94212-0_2

[84]

Suzanne M. Rebro, Ruth E. Patterson, Alan R. Kristal, and Carrie L. Cheney. 1998. The Effect of Keeping Food Records on Eating Patterns. Journal of the American Dietetic Association, Vol. 98, 10 (oct 1998), 1163--1165. https://doi.org/10.1016/S0002--8223(98)00269--7

[85]

Recovery Record. 2023. https://www.recoveryrecord.de/ Retrieved June 26, 2023 from

[86]

John Rooksby, Mattias Rost, Alistair Morrison, and Matthew Chalmers. 2014. Personal tracking as lived informatics. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2014). ACM, New York, NY, USA, 1163--1172. https://doi.org/10.1145/2556288.2557039

Digital Library

[87]

Herman Saksono, Ashwini Ranade, Geeta Kamarthi, Carmen Castaneda-Sceppa, Jessica A Hoffman, Cathy Wirth, and Andrea G Parker. 2015. Spaceship Launch: Designing a Collaborative Exergame for Families. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, New York, NY, USA, 1776--1787. https://doi.org/10.1145/2675133.2675159

Digital Library

[88]

Jessica Schroeder, Jane Hoffswell, Chia Fang Chung, James Fogarty, Sean Munson, and Jasmine Zia. 2017. Supporting patient-provider collaboration to identify individual triggers using food and symptom journals. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW 2017). 1726--1739. https://doi.org/10.1145/2998181.2998276

Digital Library

[89]

Jessica Schroeder, Ravi Karkar, Natalia Murinova, James Fogarty, and Sean A. Munson. 2019. Examining Opportunities for Goal-Directed Self-Tracking to Support Chronic Condition Management. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT 2019), Vol. 3, 4 (dec 2019), 1--26. https://doi.org/10.1145/3369809

Digital Library

[90]

Scott H. Sicherer and Hugh A. Sampson. 2014. Food allergy: Epidemiology, pathogenesis, diagnosis, and treatment. Journal of Allergy and Clinical Immunology, Vol. 133, 2 (2014), 291--307.e5. https://doi.org/10.1016/j.jaci.2013.11.020

[91]

Katie A. Siek, Kay H. Connelly, Yvonne Rogers, Paul Rohwer, Desiree Lambert, and Janet L. Welch. 2006. When Do We Eat? An Evaluation of Food Items Input into an Electronic Food Monitoring Application. In 2006 Pervasive Health Conference and Workshops. IEEE, 1--10. https://doi.org/10.1109/PCTHEALTH.2006.361684

[92]

Lucas M. Silva, Franceli L. Cibrian, Daniel A. Epstein, Arpita Bhattacharya, Elizabeth A. Ankrah, Elissa Monteiro, Jesus A. Beltran, Sabrina E. Schuck, Kimberley D. Lakes, and Gillian R. Hayes. 2022. Adapting Multidevice Deployments During a Pandemic: Lessons Learned From Two Studies. IEEE Pervasive Computing, Vol. 21, 1 (jan 2022), 48--56. https://doi.org/10.1109/MPRV.2021.3104262

[93]

Emma Simpson, Jennifer Bradley, Ivan Poliakov, Dan Jackson, Patrick Olivier, Ashley J Adamson, and Emma Foster. 2017. Iterative Development of an Online Dietary Recall Tool: INTAKE24. Nutrients, Vol. 9, 2 (2017), 118. https://doi.org/10.3390/nu9020118

[94]

Edison Thomaz, Irfan Essa, and Gregory D. Abowd. 2015. A Practical Approach for Recognizing Eating Moments with Wrist-Mounted Inertial Sensing. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (Osaka, Japan) (UbiComp '15). ACM, New York, NY, USA, 1029--1040. https://doi.org/10.1145/2750858.2807545

Digital Library

[95]

Frances E. Thompson and Amy F. Subar. 2013. Dietary Assessment Methodology. In Nutrition in the Prevention and Treatment of Disease. Elsevier Inc., 5--46. https://doi.org/10.1016/B978-0--12--391884-0.00001--9

[96]

Christoph Trattner, David Elsweiler, and Simon Howard. 2017. Estimating the Healthiness of Internet Recipes: A Cross-sectional Study. Frontiers in Public Health, Vol. 5 (feb 2017). https://doi.org/10.3389/fpubh.2017.00016

[97]

Christopher C Tsai, Gunny Lee, Fred Raab, Gregory J Norman, Timothy Sohn, William G Griswold, and Kevin Patrick. 2007. Usability and Feasibility of PmEB: A Mobile Phone Application for Monitoring Real Time Caloric Balance. Mobile Networks and Applications, Vol. 12, 2--3 (jun 2007), 173--184. https://doi.org/10.1007/s11036-007-0014--4

[98]

Bert Vandenberghe, Jasper Vanhoof, Roos Voorend, David Geerts, and Fabienne Dobbels. 2018. The 'Self' as Barrier for Self-Management Technologies in Healthcare?. In Proceedings of the 12th EAI International Conference on Pervasive Computing Technologies for Healthcare. ACM, New York, NY, USA, 336--346. https://doi.org/10.1145/3240925.3240974

Digital Library

[99]

Shibo Zhang, Yuqi Zhao, Dzung Tri Nguyen, Runsheng Xu, Sougata Sen, Josiah Hester, and Nabil Alshurafa. 2020. NeckSense: A Multi-Sensor Necklace for Detecting Eating Activities in Free-Living Conditions. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, 2, Article 72 (Jun 2020), 26 pages. https://doi.org/10.1145/3397313

Digital Library

[100]

Jasmine Zia, Jessica Schroeder, Sean Munson, James Fogarty, Linda Nguyen, Pamela Barney, Margaret Heitkemper, and Uri Ladabaum. 2016. Feasibility and Usability Pilot Study of a Novel Irritable Bowel Syndrome Food and Gastrointestinal Symptom Journal Smartphone App. Clinical and Translational Gastroenterology, Vol. 7, 3 (Mar 2016), e147. https://doi.org/10.1038/ctg.2016.9

Cited By

Le HLakshminarayanan RLi JMishra VIntille S(2024)Collecting Self-reported Physical Activity and Posture Data Using Audio-based Ecological Momentary AssessmentProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785848:3(1-35)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678584

Index Terms

Exploring Opportunities for Multimodality and Multiple Devices in Food Journaling
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
  2. Ubiquitous and mobile computing
    1. Ubiquitous and mobile devices
      1. Mobile devices

Recommendations

Investigating Preferred Food Description Practices in Digital Food Journaling
DIS '21: Proceedings of the 2021 ACM Designing Interactive Systems Conference

Journaling of consumed foods through digital devices is a popular self-tracking strategy for weight loss and eating mindfulness. Research has explored modalities, like photos and open-ended text and voice descriptions, to make journaling less burdensome ...
FoodScrap: Promoting Rich Data Capture and Reflective Food Journaling Through Speech Input
DIS '21: Proceedings of the 2021 ACM Designing Interactive Systems Conference

The factors influencing people’s food decisions, such as one’s mood and eating environment, are important information to foster self-reflection and to develop personalized healthy diet. But, it is difficult to consistently collect them due to the heavy ...
Approaches to food journaling on mobile devices

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 7, Issue MHCI

MHCI

September 2023

1017 pages

EISSN:2573-0142

DOI:10.1145/3624512

Editor:
Jeff Nichols
Apple Inc., United States

Issue’s Table of Contents

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 September 2023

Published in PACMHCI Volume 7, Issue MHCI

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
669
Total Downloads

Downloads (Last 12 months)427
Downloads (Last 6 weeks)45

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Le HLakshminarayanan RLi JMishra VIntille S(2024)Collecting Self-reported Physical Activity and Posture Data Using Audio-based Ecological Momentary AssessmentProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785848:3(1-35)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678584

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents