Cited By
View all- Robinson RMekler E(2024)Reflections Towards More Thoughtful Engagement with Literature Reviews in HCIProceedings of the Halfway to the Future Symposium10.1145/3686169.3686183(1-4)Online publication date: 21-Oct-2024
What Do We Do? Lessons Learned from Conducting Systematic Reviews to Improve HCI Dissemination
Authors: 
Zaidat Ibrahim, 
Pegah Karimi, Aqueasha Martin-Hammond, Christina Harrington, 
Katie A. SiekAuthors Info & Claims
Article No.: 541, Pages 1 - 8
Abstract
Systematic reviews are essential in helping researchers address pre-defined research questions through explicit, methodical, and reproducible techniques for identifying studies and comprehensively synthesizing their findings. We highlight our experiences conducting two systematic review studies in HCI: (1) women’s reproductive health research in HCI and (2) the intersection of identity and older adults in health research. We identify patterns and lessons that can be applied to enhance the reporting and communication of our research. While these lessons may not be universally applicable, they provide HCI researchers with the opportunity for introspection regarding how we convey our findings to the broader research community. Additionally, these lessons contribute to upholding transparency and integrity in our work, rendering it more long-lasting and beneficial for secondary purposes, like literature reviews and study replication. We provide recommendations and, where feasible, good examples of how to effectively report participants’ demographics and study methodology in our HCI work.
Supplemental Material
- Download
- 12.28 MB
- Transcript
References
[1]
Adrian Bolesnikov, Karen Anne Cochrane, and Audrey Girouard. 2023. Wearable Identities: Understanding Wearables’ Potential for Supporting the Expression of Queer Identities. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 393, 19 pages. https://doi.org/10.1145/3544548.3581327
[2]
J Brender, Jan Talmon, Elske Ammenwerth, Jytte Brender, Nicolette de Keizer, Pirkko Nykänen, Michael Rigby, and E. Ammenwerth. 2013. STARE-HI–Statement on reporting of evaluation studies in Health Informatics. Applied Clinical Informatics 4 (01 2013), 331–358. https://doi.org/10.4338/ACI-2013-04-RA-0024
[3]
Kelly Caine. 2016. Local Standards for Sample Size at CHI. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 981–992. https://doi.org/10.1145/2858036.2858498
[4]
Iain Chalmers, Larry V Hedges, and Harris Cooper. 2002. A brief history of research synthesis. Evaluation & the health professions 25, 1 (2002), 12–37.
[5]
Mike Clarke. 2016. History of evidence synthesis to assess treatment effects: Personal reflections on something that is very much alive. Journal of the Royal Society of Medicine 109, 4 (2016), 154–163.
[6]
Munmun De Choudhury, Scott Counts, and Eric Horvitz. 2013. Major Life Changes and Behavioral Markers in Social Media: Case of Childbirth. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work (San Antonio, Texas, USA) (CSCW ’13). Association for Computing Machinery, New York, NY, USA, 1431–1442. https://doi.org/10.1145/2441776.2441937
[7]
Munmun De Choudhury, Scott Counts, and Eric Horvitz. 2013. Predicting Postpartum Changes in Emotion and Behavior via Social Media. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 3267–3276. https://doi.org/10.1145/2470654.2466447
[8]
Mari Ervasti, Minna Isomursu, and Igone Idigoras Leibar. 2011. Touch- and audio-based medication management service concept for vision impaired older people. In 2011 IEEE International Conference on RFID-Technologies and Applications. IEEE, 244–251. https://doi.org/10.1109/RFID-TA.2011.6068645
[9]
Pin Sym Foong, Shengdong Zhao, Felicia Tan, and Joseph Jay Williams. 2018. Harvesting Caregiving Knowledge: Design Considerations for Integrating Volunteer Input in Dementia Care. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18, Paper 79). Association for Computing Machinery, New York, NY, USA, 1–12.
[10]
R Stuart Geiger. 2019. The Rise and Fall of the Note: Changing Paper Lengths in ACM CSCW, 2000-2018. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 222 (nov 2019), 10 pages. https://doi.org/10.1145/3359324
[11]
Christina Harrington, Aqueasha Martin-Hammond, and Kirsten E Bray. 2022. Examining Identity as a Variable of Health Technology Research for Older Adults: A Systematic Review. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 265, 24 pages. https://doi.org/10.1145/3491102.3517621
[12]
Mayuko Iriguchi, Hiroki Koda, and Nobuo Masataka. 2018. Colour Perception Characteristics of Women in Menopause. In Proceedings of the 2018 International Joint Workshop on Multimedia Artworks Analysis and Attractiveness Computing in Multimedia (Yokohama, Japan) (MMArt&ACM’18). Association for Computing Machinery, New York, NY, USA, 20–25. https://doi.org/10.1145/3209693.3209694
[13]
Ida Kilias Svenningsen and Teresa Almeida. 2021. Embodying the Emotional Pregnancy: Design Explorations of a Prenatal Yoga Multi-Sensorial Environment. In Proceedings of the 32nd Australian Conference on Human-Computer Interaction (Sydney, NSW, Australia) (OzCHI ’20). Association for Computing Machinery, New York, NY, USA, 189–197. https://doi.org/10.1145/3441000.3441044
[14]
K. Cassie Kresnye, Mona Y. Alqassim, Briana Hollins, Lucia Guerra-Reyes, Maria K. Wolters, and Katie A. Siek. 2021. What to Expect When You Are No Longer Expecting: Information Needs of Women Who Experienced a Miscarriage. In Proceedings of the 14th EAI International Conference on Pervasive Computing Technologies for Healthcare (Atlanta, GA, USA) (PervasiveHealth ’20). Association for Computing Machinery, New York, NY, USA, 85–96. https://doi.org/10.1145/3421937.3421995
[15]
Sebastian Linxen, Christian Sturm, Florian Brühlmann, Vincent Cassau, Klaus Opwis, and Katharina Reinecke. 2021. How WEIRD is CHI?. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 143, 14 pages. https://doi.org/10.1145/3411764.3445488
[16]
Pamela J. Ludford, Reid Priedhorsky, Ken Reily, and Loren Terveen. 2007. Capturing, Sharing, and Using Local Place Information. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’07). Association for Computing Machinery, New York, NY, USA, 1235–1244. https://doi.org/10.1145/1240624.1240811
[17]
Elisa D. Mekler, Julia Ayumi Bopp, Alexandre N. Tuch, and Klaus Opwis. 2014. A Systematic Review of Quantitative Studies on the Enjoyment of Digital Entertainment Games. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14). Association for Computing Machinery, New York, NY, USA, 927–936. https://doi.org/10.1145/2556288.2557078
[18]
Novia Nurain and Chia-Fang Chung. 2023. “I Left My Legacy, Told My Story”: Understanding Older Adults’ Tracking Practices to Promote Active Aging. In Proceedings of the 2023 ACM Designing Interactive Systems Conference (Pittsburgh, PA, USA) (DIS ’23). Association for Computing Machinery, New York, NY, USA, 459–475. https://doi.org/10.1145/3563657.3596083
[19]
Jessica Pater, Amanda Coupe, Rachel Pfafman, Chanda Phelan, Tammy Toscos, and Maia Jacobs. 2021. Standardizing Reporting of Participant Compensation in HCI: A Systematic Literature Review and Recommendations for the Field. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 141, 16 pages. https://doi.org/10.1145/3411764.3445734
[20]
PRISMA. 2023. History & Development of PRISMA. Retrieved October 3, 2023 from http://prisma-statement.org/PRISMAStatement/HistoryAndDevelopment
[21]
Cassidy Pyle, Lee Roosevelt, Ashley Lacombe-Duncan, and Nazanin Andalibi. 2021. LGBTQ Persons’ Pregnancy Loss Disclosures to Known Ties on Social Media: Disclosure Decisions and Ideal Disclosure Environments. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–17.
[22]
Katja Rogers and Katie Seaborn. 2023. The Systematic Review-Lution: A Manifesto to Promote Rigour and Inclusivity in Research Synthesis. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI EA ’23). Association for Computing Machinery, New York, NY, USA, Article 424, 11 pages. https://doi.org/10.1145/3544549.3582733
[23]
Carolina Salvador, Arturo Nakasone, and Jose Antonio Pow-Sang. 2014. A Systematic Review of Usability Techniques in Agile Methodologies. In Proceedings of the 7th Euro American Conference on Telematics and Information Systems (Valparaiso, Chile) (EATIS ’14). Association for Computing Machinery, New York, NY, USA, Article 17, 6 pages. https://doi.org/10.1145/2590651.2590668
[24]
Shruti Sannon and Andrea Forte. 2022. Privacy Research with Marginalized Groups: What We Know, What’s Needed, and What’s Next. Proc. ACM Hum.-Comput. Interact. 6, CSCW2, Article 455 (nov 2022), 33 pages. https://doi.org/10.1145/3555556
[25]
Zahra Shakeri Hossein Abad, Craig Anslow, and Frank Maurer. 2014. Multi Surface Interactions with Geospatial Data: A Systematic Review. In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces (Dresden, Germany) (ITS ’14). Association for Computing Machinery, New York, NY, USA, 69–78. https://doi.org/10.1145/2669485.2669505
[26]
Supriya Singh, Anuja Cabraal, Catherine Demosthenous, Gunela Astbrink, and Michele Furlong. 2007. Password Sharing: Implications for Security Design Based on Social Practice. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’07). Association for Computing Machinery, New York, NY, USA, 895–904. https://doi.org/10.1145/1240624.1240759
[27]
Mark Starr, Iain Chalmers, Mike Clarke, and Andrew D Oxman. 2009. The origins, evolution, and future of The Cochrane Database of Systematic Reviews. International journal of technology assessment in health care 25, S1 (2009), 182–195.
[28]
Evropi Stefanidi, Marit Bentvelzen, Paweł W. Woźniak, Thomas Kosch, Mikołaj P. Woźniak, Thomas Mildner, Stefan Schneegass, Heiko Müller, and Jasmin Niess. 2023. Literature Reviews in HCI: A Review of Reviews. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 509, 24 pages. https://doi.org/10.1145/3544548.3581332
[29]
Yuko Takeda. 2010. Understanding the Life Stages of Women to Enhance Your Practice. Japan Medical Association Journal 53 (09 2010).
[30]
Hao Tan, Yi Zhu, and Jianghong Zhao. 2012. Development of an Automotive User Interface Design Knowledge System. In Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Portsmouth, New Hampshire) (AutomotiveUI ’12). Association for Computing Machinery, New York, NY, USA, 201–208. https://doi.org/10.1145/2390256.2390291
[31]
Anupriya Tuli, Shruti Dalvi, Neha Kumar, and Pushpendra Singh. 2019. “It’s a Girl Thing”: Examining Challenges and Opportunities around Menstrual Health Education in India. ACM Trans. Comput.-Hum. Interact. 26, 5, Article 29 (jul 2019), 24 pages. https://doi.org/10.1145/3325282
[32]
Joe Tullio, Anind K. Dey, Jason Chalecki, and James Fogarty. 2007. How It Works: A Field Study of Non-Technical Users Interacting with an Intelligent System. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’07). Association for Computing Machinery, New York, NY, USA, 31–40. https://doi.org/10.1145/1240624.1240630
[33]
Lauren Wilcox, Renee Shelby, Rajesh Veeraraghavan, Oliver L. Haimson, Gabriela Cruz Erickson, Michael Turken, and Rebecca Gulotta. 2023. Infrastructuring Care: How Trans and Non-Binary People Meet Health and Well-Being Needs through Technology. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 489, 17 pages. https://doi.org/10.1145/3544548.3581040
[34]
Yixuan Zhang, Joseph D Gaggiano, Nutchanon Yongsatianchot, Nurul M Suhaimi, Miso Kim, Yifan Sun, Jacqueline Griffin, and Andrea G Parker. 2023. What Do We Mean When We Talk about Trust in Social Media? A Systematic Review. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 670, 22 pages. https://doi.org/10.1145/3544548.3581019
Index Terms
- What Do We Do? Lessons Learned from Conducting Systematic Reviews to Improve HCI Dissemination
Recommendations
Refugees and HCI SIG: The Role of HCI in Responding to the Refugee Crisis
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing SystemsThe recent refugee crisis has escalated and expanded in to a global challenge. In their flight from hostility, refugee populations face challenges in accessing services that are often compounded by political and social tensions. The urgency and ...
HCI Knowledges and Situated Dissemination
DIS '17 Companion: Proceedings of the 2017 ACM Conference Companion Publication on Designing Interactive SystemsDrawing from STS, feminist STS and performance theories, my PhD interrogates the practice of research dissemination at an HCI workplace. By studying such practices at a micro level, I begin to construct a productive critique that addresses and questions ...
Situated Dissemination through an HCI Workplace
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing SystemsResearchers working in domains such as Research through Design and Feminist HCI have been questioning "dissemination practices" and their impact on our capacity to produce reflexive accounts of research in publications. This paper examines academic ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
May 2024
4761 pages
ISBN:9798400703317
DOI:10.1145/3613905
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 May 2024
Check for updates
Author Tags
Qualifiers
- Extended-abstract
- Research
- Refereed limited
Conference
Acceptance Rates
Overall Acceptance Rate 6,164 of 23,696 submissions, 26%
Upcoming Conference
CHI '25
- Sponsor:
- sigchi
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 270Total Downloads
- Downloads (Last 12 months)270
- Downloads (Last 6 weeks)37
Reflects downloads up to 21 Nov 2024
Other Metrics
Citations
Cited By
View all- Robinson RMekler E(2024)Reflections Towards More Thoughtful Engagement with Literature Reviews in HCIProceedings of the Halfway to the Future Symposium10.1145/3686169.3686183(1-4)Online publication date: 21-Oct-2024
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderFull Text
View this article in Full Text.
Full TextHTML Format
View this article in HTML Format.
HTML Format