research-article

Helping Helpers: Supporting Volunteers in Remote Sighted Assistance with Augmented Reality Maps

Authors:

Syed Masum Billah,

John M. CarrollAuthors Info & Claims

DIS '22: Proceedings of the 2022 ACM Designing Interactive Systems Conference

Pages 881 - 897

https://doi.org/10.1145/3532106.3533560

Published: 13 June 2022 Publication History

Abstract

Remote sighted assistance (RSA) has emerged as a conversational assistive service, where remote sighted workers, i.e., agents, provide real-time assistance to blind users via video-chat-like communication. Prior work identified several challenges for the agents to provide navigational assistance to users and proposed computer vision-mediated RSA service to address those challenges. We present an interactive system implementing a high-fidelity prototype of RSA service using augmented reality (AR) maps with localization and virtual elements placement capabilities. The paper also presents a confederate-based study design to evaluate the effects of AR maps with 13 untrained agents. The study revealed that, compared to baseline RSA, agents were significantly faster in providing indoor navigational assistance to a confederate playing the role of users, and agents’ mental workload was significantly reduced—all indicate the feasibility and scalability of AR maps in RSA services.

References

[1]

2022. 3d Scanner App™. Retrieved February 12, 2022 from https://apps.apple.com/us/app/3d-scanner-app/id1419913995

[2]

2022. ARCore. Retrieved February 12, 2022 from https://developers.google.com/ar

[3]

2022. Be My Eyes - Our story. Retrieved February 12, 2022 from https://www.bemyeyes.com/about

[4]

2022. Be My Eyes - See the world together. Retrieved February 12, 2022 from https://www.bemyeyes.com/

[5]

2022. Can I change my availability hours? - Be My Eyes Help Center. Retrieved February 12, 2022 from https://support.bemyeyes.com/hc/en-us/articles/360005892797-Can-I-change-my-availability-hours-

[6]

2022. Frequently Asked Questions - Aira. Retrieved February 12, 2022 from https://aira.io/frequently-asked-questions/

[7]

2022. Google Maps - Transit & Food. Retrieved February 12, 2022 from https://apps.apple.com/us/app/google-maps-transit-food/id585027354

[8]

2022. Home of the Blender project. Retrieved February 12, 2022 from https://www.blender.org

[9]

2022. Model I/O. Retrieved February 12, 2022 from https://developer.apple.com/documentation/modelio

[10]

2022. More to Explore with ARKit 5. Retrieved February 12, 2022 from https://developer.apple.com/augmented-reality/arkit/

[11]

2022. Multipeer Connectivity. Retrieved February 12, 2022 from https://developer.apple.com/documentation/multipeerconnectivity

[12]

2022. RealityKit. Retrieved February 12, 2022 from https://developer.apple.com/augmented-reality/realitykit

[13]

2022. SceneKit. Retrieved February 12, 2022 from https://developer.apple.com/scenekit

[14]

2022. TapTapSee. Retrieved February 12, 2022 from https://taptapseeapp.com/

[15]

2022. Visual Interpreting – Get Live, On-demand Access to Visual Information. Retrieved February 12, 2022 from https://aira.io/

[16]

Dragan Ahmetovic, Cole Gleason, Chengxiong Ruan, Kris Kitani, Hironobu Takagi, and Chieko Asakawa. 2016. NavCog: A Navigational Cognitive Assistant for the Blind. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services(Florence, Italy) (MobileHCI ’16). Association for Computing Machinery, New York, NY, USA, 90–99. https://doi.org/10.1145/2935334.2935361

Digital Library

[17]

Amaia Aizpurua, Myriam Arrue, Simon Harper, and Markel Vigo. 2014. Are users the gold standard for accessibility evaluation?. In Proceedings of the 11th Web for All Conference. 1–4. https://doi.org/10.1145/2596695.2596705

Digital Library

[18]

Mike Allen. 2017. The SAGE Encyclopedia of Communication Research Methods. 4 (2017). https://doi.org/10.4135/9781483381411

[19]

Richard Armitage and Laura B Nellums. 2020. The COVID-19 response must be disability inclusive. The Lancet Public Health 5, 5 (2020), e257. https://doi.org/10.1016/S2468-2667(20)30076-1

[20]

Mauro Avila, Katrin Wolf, Anke Brock, and Niels Henze. 2016. Remote assistance for blind users in daily life: A survey about Be My Eyes. In Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments. 1–2. https://doi.org/10.1145/2910674.2935839

Digital Library

[21]

Nikola Banovic, Rachel L Franz, Khai N Truong, Jennifer Mankoff, and Anind K Dey. 2013. Uncovering information needs for independent spatial learning for users who are visually impaired. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility. 1–8. https://doi.org/10.1145/2513383.2513445

Digital Library

[22]

Przemyslaw Baranski and Pawel Strumillo. 2015. Field trials of a teleassistance system for the visually impaired. In 2015 8th International Conference on Human System Interaction (HSI). IEEE, 173–179. https://doi.org/10.1109/HSI.2015.7170662

[23]

Cynthia L. Bennett and Daniela K. Rosner. 2019. The Promise of Empathy: Design, Disability, and Knowing the ”Other”. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300528

Digital Library

[24]

Jeffrey P Bigham, Chandrika Jayant, Andrew Miller, Brandyn White, and Tom Yeh. 2010. VizWiz:: LocateIt-enabling blind people to locate objects in their environment. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops. IEEE, 65–72. https://doi.org/10.1109/CVPRW.2010.5543821

[25]

Erin Brady and Jeffrey P Bigham. 2015. Crowdsourcing accessibility: Human-powered access technologies. Foundations and Trends® in Human–Computer Interaction 8, 4(2015), 273–372. https://doi.org/10.1561/1100000050

[26]

Erin L. Brady, Meredith Ringel Morris, Yu Zhong, Samuel White, and Jeffrey P. Bigham. 2013. Visual challenges in the everyday lives of blind people. In 2013 ACM SIGCHI Conference on Human Factors in Computing Systems, Wendy E. Mackay, Stephen A. Brewster, and Susanne Bødker (Eds.). ACM, 2117–2126. https://doi.org/10.1145/2470654.2481291

Digital Library

[27]

Steve Branson, Catherine Wah, Florian Schroff, Boris Babenko, Peter Welinder, Pietro Perona, and Serge J. Belongie. 2010. Visual Recognition with Humans in the Loop. In 11th European Conference on Computer Vision. 438–451. https://doi.org/10.1007/978-3-642-15561-1_32

[28]

Komang Candra Brata and Deron Liang. 2020. Comparative study of user experience on mobile pedestrian navigation between digital map interface and location-based augmented reality.International Journal of Electrical & Computer Engineering (2088-8708) 10(2020). https://doi.org/10.11591/ijece.v10i2.pp2037-2044

[29]

Virginia Braun and Victoria Clarke. 2006. Using Thematic Analysis in Psychology. Qualitative Research in Psychology 3, 2 (2006), 77–101. https://doi.org/10.1191/1478088706qp063oa

[30]

Andrius Budrionis, Darius Plikynas, Povilas Daniušis, and Audrius Indrulionis. 2020. Smartphone-based computer vision travelling aids for blind and visually impaired individuals: A systematic review. Assistive Technology(2020), 1–17. https://doi.org/10.1080/10400435.2020.1743381

[31]

M Bujacz, P Baranski, M Moranski, P Strumillo, and A Materka. 2008. Remote guidance for the blind—A proposed teleassistance system and navigation trials. In 2008 Conference on Human System Interactions. IEEE, 888–892. https://doi.org/10.1109/HSI.2008.4581561

[32]

Michele A Burton, Erin Brady, Robin Brewer, Callie Neylan, Jeffrey P Bigham, and Amy Hurst. 2012. Crowdsourcing subjective fashion advice using VizWiz: challenges and opportunities. In Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility. ACM, 135–142. https://doi.org/10.1145/2384916.2384941

Digital Library

[33]

John M. Carroll, Sooyeon Lee, Madison Reddie, Jordan Beck, and Mary Beth Rosson. 2020. Human-Computer Synergies in Prosthetic Interactions. IxD&A 44(2020), 29–52. https://doi.org/10.55612/s-5002-044-002

[34]

John M Carroll, Patrick C Shih, Kyungsk Han, and Jess Kropczynski. 2017. Coordinating community cooperation: Integrating timebanks and nonprofit volunteering by design. International Journal of Design 11, 1 (2017).

[35]

Babar Chaudary, Iikka Paajala, Eliud Keino, and Petri Pulli. 2017. Tele-guidance based navigation system for the visually impaired and blind persons. In eHealth 360. Springer, 9–16. https://doi.org/10.1007/978-3-319-49655-9_2

[36]

Babar Chaudary, Sami Pohjolainen, Saima Aziz, Leena Arhippainen, and Petri Pulli. 2021. Teleguidance-based remote navigation assistance for visually impaired and blind people—usability and user experience. Virtual Reality (2021), 1–18. https://doi.org/10.1007/s10055-021-00536-z

Digital Library

[37]

Roger M. Downs and David Stea. 2011. Cognitive Maps and Spatial Behaviour: Process and Products., 312–317 pages. https://doi.org/10.1002/9780470979587.ch41

[38]

Mostafa Elgendy, Miklós Herperger, Tibor Guzsvinecz, and Cecilia Sik Lanyi. 2019. Indoor Navigation for People with Visual Impairment using Augmented Reality Markers. In The 10th IEEE International Conference on Cognitive Infocommunications (CogInfoCom). IEEE, 425–430. https://doi.org/10.1109/CogInfoCom47531.2019.9089960

[39]

Giovanni Fusco and James M Coughlan. 2020. Indoor localization for visually impaired travelers using computer vision on a smartphone. In Proceedings of the 17th International Web for All Conference(W4A ’20). Article 8, 11 pages. https://doi.org/10.1145/3371300.3383345

Digital Library

[40]

Jan C Galvin and Marcia J Scherer. 1996. Evaluating, Selecting, and Using Appropriate Assistive Technology.Aspen Publishers, Inc.

[41]

Aura Ganz, James M Schafer, Yang Tao, Carole Wilson, and Meg Robertson. 2014. PERCEPT-II: Smartphone based indoor navigation system for the blind. In 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 3662–3665. https://doi.org/10.1109/EMBC.2014.6944417

[42]

Vanja Garaj, Rommanee Jirawimut, Piotr Ptasinski, Franjo Cecelja, and Wamadeva Balachandran. 2003. A system for remote sighted guidance of visually impaired pedestrians. British Journal of Visual Impairment 21, 2 (2003), 55–63. https://doi.org/10.1177/026461960302100204

[43]

João Guerreiro, Dragan Ahmetovic, Daisuke Sato, Kris Kitani, and Chieko Asakawa. 2019. Airport accessibility and navigation assistance for people with visual impairments. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–14. https://doi.org/10.1145/3290605.3300246

Digital Library

[44]

João Guerreiro, Daisuke Sato, Saki Asakawa, Huixu Dong, Kris M Kitani, and Chieko Asakawa. 2019. CaBot: Designing and evaluating an autonomous navigation robot for blind people. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility. 68–82. https://doi.org/10.1145/3308561.3353771

Digital Library

[45]

Nicole Holmes and Kelly Prentice. 2017. iPhone video link FaceTime as an orientation tool: Remote O&M for people with vision impairment. International Journal of Orientation & Mobility 7, 1 (2017), 60–67. https://doi.org/10.21307/ijom-2017-057

[46]

Bill Holton. 2015. Crowdviz: Remote video assistance on your iphone. AFB AccessWorld Magazine(2015).

[47]

Bill Holton. 2016. BeSpecular: A new remote assistant service. Access World Magazine 17, 7 (2016).

[48]

Alice M Isen and Paula F Levin. 1972. Effect of feeling good on helping: cookies and kindness.Journal of personality and social psychology 21, 3(1972), 384. https://doi.org/10.1037/h0032317

[49]

Rabia Jafri, Syed Abid Ali, Hamid R. Arabnia, and Shameem Fatima. 2014. Computer vision-based object recognition for the visually impaired in an indoors environment: a survey. The Visual Computer 30, 11 (2014), 1197–1222. https://doi.org/10.1007/s00371-013-0886-1

Digital Library

[50]

Vaishnav Kameswaran, Alexander J. Fiannaca, Melanie Kneisel, Amy Karlson, Edward Cutrell, and Meredith Ringel Morris. 2020. Understanding in-situ use of commonly available navigation technologies by people with visual impairments. In The 22nd international ACM SIGACCESS conference on computers and accessibility. 1–12. https://doi.org/10.1145/3373625.3416995

Digital Library

[51]

Rie Kamikubo, Naoya Kato, Keita Higuchi, Ryo Yonetani, and Yoichi Sato. 2020. Support strategies for remote guides in assisting people with visual impairments for effective indoor navigation. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12. https://doi.org/10.1145/3313831.3376823

Digital Library

[52]

Seita Kayukawa, Keita Higuchi, João Guerreiro, Shigeo Morishima, Yoichi Sato, Kris Kitani, and Chieko Asakawa. 2019. Bbeep: A sonic collision avoidance system for blind travellers and nearby pedestrians. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12. https://doi.org/10.1145/3290605.3300282

Digital Library

[53]

Anja Kintsch and Rogerio DePaula. 2002. A framework for the adoption of assistive technology. SWAAAC 2002: Supporting learning through assistive technology (2002), 1–10.

[54]

Jesper Kjeldskov and Mikael B Skov. 2014. Was it worth the hassle? Ten years of mobile HCI research discussions on lab and field evaluations. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services. 43–52. https://doi.org/10.1145/2628363.2628398

Digital Library

[55]

Jesper Kjeldskov, Mikael B Skov, Benedikte S Als, and Rune T Høegh. 2004. Is it worth the hassle? Exploring the added value of evaluating the usability of context-aware mobile systems in the field. In International Conference on Mobile Human-Computer Interaction. Springer, 61–73. https://doi.org/10.1007/978-3-540-28637-0_6

[56]

Eunjeong Ko and Eun Yi Kim. 2017. A vision-based wayfinding system for visually impaired people using situation awareness and activity-based instructions. Sensors 17, 8 (2017), 1882. https://doi.org/10.3390/s17081882

[57]

Adarsh Kowdle, Yao-Jen Chang, Andrew C. Gallagher, and Tsuhan Chen. 2011. Active learning for piecewise planar 3D reconstruction. In The 24th IEEE Conference on Computer Vision and Pattern Recognition. 929–936. https://doi.org/10.1109/CVPR.2011.5995638

Digital Library

[58]

Aliasgar Kutiyanawala, Vladimir Kulyukin, and John Nicholson. 2011. Teleassistance in accessible shopping for the blind. In Proceedings on the International Conference on Internet Computing (ICOMP). Citeseer, 1.

[59]

Walter S. Lasecki, Kyle I. Murray, Samuel White, Robert C. Miller, and Jeffrey P. Bigham. 2011. Real-Time Crowd Control of Existing Interfaces. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (Santa Barbara, California, USA) (UIST ’11). Association for Computing Machinery, New York, NY, USA, 23–32. https://doi.org/10.1145/2047196.2047200

Digital Library

[60]

Walter S Lasecki, Rachel Wesley, Jeffrey Nichols, Anand Kulkarni, James F Allen, and Jeffrey P Bigham. 2013. Chorus: a crowd-powered conversational assistant. In Proceedings of the 26th annual ACM symposium on User interface software and technology. ACM, 151–162. https://doi.org/10.1145/2501988.2502057

Digital Library

[61]

Sooyeon Lee, Madison Reddie, Krish Gurdasani, Xiying Wang, Jordan Beck, Mary Beth Rosson, and John M. Carroll. 2018. Conversations for Vision: Remote Sighted Assistants Helping People with Visual Impairments. arxiv:1812.00148 [cs.HC]

[62]

Sooyeon Lee, Madison Reddie, Chun-Hua Tsai, Jordan Beck, Mary Beth Rosson, and John M Carroll. 2020. The emerging professional practice of remote sighted assistance for people with visual impairments. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12. https://doi.org/10.1145/3313831.3376591

Digital Library

[63]

Sooyeon Lee, Rui Yu, Jingyi Xie, Syed Masum Billah, and John M. Carroll. 2022. Opportunities for Human-AI Collaboration in Remote Sighted Assistance. In 27th International Conference on Intelligent User Interfaces (Helsinki, Finland) (IUI ’22). Association for Computing Machinery, New York, NY, USA, 63–78. https://doi.org/10.1145/3490099.3511113

Digital Library

[64]

Gordon E Legge, Paul J Beckmann, Bosco S Tjan, Gary Havey, Kevin Kramer, David Rolkosky, Rachel Gage, Muzi Chen, Sravan Puchakayala, and Aravindhan Rangarajan. 2013. Indoor navigation by people with visual impairment using a digital sign system. PloS one 8, 10 (2013). https://doi.org/10.1371/journal.pone.0076783

[65]

Ki-Joune Li and Jiyeong Lee. 2010. Indoor spatial awareness initiative and standard for indoor spatial data. In Proceedings of IROS 2010 Workshop on Standardization for Service Robot, Vol. 18.

[66]

Yang Liu, Noelle RB Stiles, and Markus Meister. 2018. Augmented reality powers a cognitive assistant for the blind. ELife 7(2018), e37841. https://doi.org/10.7554/eLife.37841

[67]

Jack M Loomis, Roberta L Klatzky, and Reginald G Golledge. 2001. Navigating without vision: basic and applied research. Optometry and vision science 78, 5 (may 2001), 282–289. https://doi.org/10.1097/00006324-200105000-00011

[68]

Roberto Manduchi, Sri Kurniawan, and Homayoun Bagherinia. 2010. Blind guidance using mobile computer vision: A usability study. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility. 241–242. https://doi.org/10.1145/1878803.1878851

Digital Library

[69]

Bob Martin and Lisa McCormack. 1999. Issues surrounding Assistive Technology use and abandonment in an emerging technological culture. Assistive technology on the threshold of the new millennium 6 (1999), 413.

[70]

Troy McDaniel, Kanav Kahol, Daniel Villanueva, and Sethuraman Panchanathan. 2008. Integration of RFID and computer vision for remote object perception for individuals who are blind. In Proceedings of the First International Conference on Ambient Media and Systems. ICST. https://doi.org/10.4108/icst.ambisys2008.2902

[71]

Brian J Nguyen, Yeji Kim, Kathryn Park, Allison J Chen, Scarlett Chen, Donald Van Fossan, and Daniel L Chao. 2018. Improvement in patient-reported quality of life outcomes in severely visually impaired individuals using the Aira assistive technology system. Translational Vision Science & Technology 7, 5 (2018), 30–30. https://doi.org/10.1167/tvst.7.5.30

[72]

B Pentland. 1998. Living in a State of Stuck: How technology impacts the lives of people with disabilities. Spinal Cord 36, 3 (mar 1998), 219–219. https://doi.org/10.1038/sj.sc.3100518

[73]

Helen Petrie, Stefan Carmien, and Andrew Lewis. 2018. Assistive technology abandonment: research realities and potentials. In International conference on computers helping people with special needs. Springer, 532–540. https://doi.org/10.1007/978-3-319-94274-2_77

[74]

Helen Petrie, Valerie Johnson, Thomas Strothotte, Andreas Raab, Rainer Michel, Lars Reichert, and Axel Schalt. 1997. MoBIC: An aid to increase the independent mobility of blind travellers. British Journal of Visual Impairment 15, 2 (1997), 63–66. https://doi.org/10.1177/026461969701500205

[75]

Betsy Phillips and Hongxin Zhao. 1993. Predictors of assistive technology abandonment. Assistive technology 5, 1 (1993), 36–45. https://doi.org/10.1080/10400435.1993.10132205

[76]

Paul E Ponchillia and Susan Kay Vlahas Ponchillia. 1996. Foundations of rehabilitation teaching with persons who are blind or visually impaired. American Foundation for the Blind.

[77]

Giorgio Presti, Dragan Ahmetovic, Mattia Ducci, Cristian Bernareggi, Luca Ludovico, Adriano Baratè, Federico Avanzini, and Sergio Mascetti. 2019. WatchOut: Obstacle sonification for people with visual impairment or blindness. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility. 402–413. https://doi.org/10.1145/3308561.3353779

Digital Library

[78]

Paymon Rafian and Gordon E Legge. 2017. Remote sighted assistants for indoor location sensing of visually impaired pedestrians. ACM Transactions on Applied Perception (TAP) 14, 3 (2017), 19. https://doi.org/10.1145/3047408

Digital Library

[79]

Marti L Riemer-Reiss and Robbyn R Wacker. 1999. Assistive Technology Use and Abandonment among College Students with Disabilities. IEJLL: International Electronic Journal for Leadership in Learning (1999).

[80]

Sebastião Rocha and Arminda Lopes. 2020. Navigation based application with augmented reality and accessibility. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3334480.3383004

Digital Library

[81]

Ranga Rodrigo, Mehrnaz Zouqi, Zhenhe Chen, and Jagath Samarabandu. 2009. Robust and efficient feature tracking for indoor navigation. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 39, 3(2009), 658–671. https://doi.org/10.1109/TSMCB.2008.2008196

Digital Library

[82]

[82] LP Rosenblum, P Chanes-Mora, CR McBride, J Flewellen, N Nagarajan, R Nave Stawaz, and B Swenor.2020. Retrieved May 15, 2022 from https://www.afb.org/research-and-initiatives/covid-19-research/flatten-inaccessibility

[83]

Manaswi Saha, Alexander J Fiannaca, Melanie Kneisel, Edward Cutrell, and Meredith Ringel Morris. 2019. Closing the Gap: Designing for the Last-Few-Meters Wayfinding Problem for People with Visual Impairments. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility. 222–235. https://doi.org/10.1145/3308561.3353776

Digital Library

[84]

Stefano Scheggi, A Talarico, and Domenico Prattichizzo. 2014. A remote guidance system for blind and visually impaired people via vibrotactile haptic feedback. In 22nd Mediterranean Conference on Control and Automation. IEEE, 20–23. https://doi.org/10.1109/MED.2014.6961320

[85]

Richard Simpson. 2011. Challenges to effective evaluation of assistive technology. In Design and Use of Assistive Technology. Springer, 51–56. https://doi.org/10.1007/978-1-4419-7031-2_6

[86]

Sudipta N. Sinha, Drew Steedly, Richard Szeliski, Maneesh Agrawala, and Marc Pollefeys. 2008. Interactive 3D architectural modeling from unordered photo collections. ACM Trans. Graph. 27, 5 (2008), 159. https://doi.org/10.1145/1409060.1409112

Digital Library

[87]

Robert D Stevens and Alistair DN Edwards. 1996. An approach to the evaluation of assistive technology. In Proceedings of the second annual ACM conference on Assistive technologies. 64–71. https://doi.org/10.1145/228347.228359

Digital Library

[88]

Ender Tekin and James M. Coughlan. 2010. A mobile phone application enabling visually impaired users to find and read product barcodes. In Computers Helping People with Special Needs, Klaus Miesenberger, Joachim Klaus, Wolfgang Zagler, and Arthur Karshmer (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 290–295. https://doi.org/10.1007/978-3-642-14100-3_43

[89]

Nelson Daniel Troncoso Aldas, Sooyeon Lee, Chonghan Lee, Mary Beth Rosson, John M Carroll, and Vijaykrishnan Narayanan. 2020. AIGuide: An Augmented Reality Hand Guidance Application for People with Visual Impairments. In The 22nd International ACM SIGACCESS Conference on Computers and Accessibility. 1–13. https://doi.org/10.1145/3373625.3417028

Digital Library

[90]

Prashant Verma, Kushal Agrawal, and V. Sarasvathi. 2020. Indoor Navigation Using Augmented Reality. In Proceedings of the 2020 4th International Conference on Virtual and Augmented Reality Simulations (Sydney, NSW, Australia) (ICVARS 2020). Association for Computing Machinery, New York, NY, USA, 58–63. https://doi.org/10.1145/3385378.3385387

Digital Library

[91]

Jingyi Xie, Madison Reddie, Sooyeon Lee, Syed Masum Billah, Zihan Zhou, Chun-Hua Tsai, and John M. Carroll. 2022. Iterative Design and Prototyping of Computer Vision Mediated Remote Sighted Assistance. ACM Trans. Comput.-Hum. Interact. 29, 4, Article 36 (mar 2022), 40 pages. https://doi.org/10.1145/3501298

Digital Library

[92]

Chris Yoon, Ryan Louie, Jeremy Ryan, MinhKhang Vu, Hyegi Bang, William Derksen, and Paul Ruvolo. 2019. Leveraging augmented reality to create apps for people with visual disabilities: A case study in indoor navigation. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility. 210–221. https://doi.org/10.1145/3308561.3353788

Digital Library

[93]

Yu Zhong, Walter S Lasecki, Erin Brady, and Jeffrey P Bigham. 2015. Regionspeak: Quick comprehensive spatial descriptions of complex images for blind users. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 2353–2362. https://doi.org/10.1145/2702123.2702437

Digital Library

[94]

Peter A. Zientara, Sooyeon Lee, Gus H. Smith, Rorry Brenner, Laurent Itti, Mary B. Rosson, John M. Carroll, Kevin M. Irick, and Vijaykrishnan Narayanan. 2017. Third Eye: A shopping assistant for the visually impaired. Computer 50, 02 (feb 2017), 16–24. https://doi.org/10.1109/MC.2017.36

Digital Library

Cited By

Yu RLee SXie JBillah SCarroll J(2024)Human–AI Collaboration for Remote Sighted Assistance: Perspectives from the LLM EraFuture Internet10.3390/fi1607025416:7(254)Online publication date: 18-Jul-2024
https://doi.org/10.3390/fi16070254
Teng YCourtien CRios DTseng YGibson JAziz MReyna AVaish RSmith B(2024)Help Supporters: Exploring the Design Space of Assistive Technologies to Support Face-to-Face Help Between Blind and Sighted StrangersProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642816(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642816
Lyu YCai JCallis ACotter KCarroll J(2024)"I Got Flagged for Supposed Bullying, Even Though It Was in Response to Someone Harassing Me About My Disability.": A Study of Blind TikTokers’ Content Moderation ExperiencesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642148(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642148
Show More Cited By

Recommendations

Are Two Heads Better than One? Investigating Remote Sighted Assistance with Paired Volunteers
DIS '23: Proceedings of the 2023 ACM Designing Interactive Systems Conference

Remote Sighted Assistance (RSA) is a popular smartphone-mediated aid for people with blindness, where a sighted individual converses with a blind individual in a one-on-one (1:1) session. Since sighted assistants outnumber blind individuals (13:1), this ...
Opportunities for Human-AI Collaboration in Remote Sighted Assistance
IUI '22: Proceedings of the 27th International Conference on Intelligent User Interfaces

Remote sighted assistance (RSA) has emerged as a conversational assistive technology for people with visual impairments (VI), where remote sighted agents provide realtime navigational assistance to users with visual impairments via video-chat-like ...
The Emerging Professional Practice of Remote Sighted Assistance for People with Visual Impairments
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

People with visual impairments (PVI) must interact with a world they cannot see. Remote sighted assistance (RSA) has emerged as a conversational assistive technology. We interviewed RSA assistants ("agents") who provide assistance to PVI via a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

DIS '22: Proceedings of the 2022 ACM Designing Interactive Systems Conference

June 2022

1947 pages

ISBN:9781450393584

DOI:10.1145/3532106

Editors:
Florian `Floyd' Mueller
Monash University, Melbourne, Australia
,
Stefan Greuter
Deakin University, Melbourne, Australia
,
Rohit Ashok Khot
RMIT University, Melbourne, Australia
,
Penny Sweetser
The Australian National University, Canberra, Australia
,
Marianna Obrist
University College London, UK

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 June 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Institutes of Health, National Library of Medicine

Conference

DIS '22

Sponsor:

SIGCHI

DIS '22: Designing Interactive Systems Conference

June 13 - 17, 2022

Virtual Event, Australia

Acceptance Rates

Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
322
Total Downloads

Downloads (Last 12 months)102
Downloads (Last 6 weeks)15

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yu RLee SXie JBillah SCarroll J(2024)Human–AI Collaboration for Remote Sighted Assistance: Perspectives from the LLM EraFuture Internet10.3390/fi1607025416:7(254)Online publication date: 18-Jul-2024
https://doi.org/10.3390/fi16070254
Teng YCourtien CRios DTseng YGibson JAziz MReyna AVaish RSmith B(2024)Help Supporters: Exploring the Design Space of Assistive Technologies to Support Face-to-Face Help Between Blind and Sighted StrangersProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642816(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642816
Lyu YCai JCallis ACotter KCarroll J(2024)"I Got Flagged for Supposed Bullying, Even Though It Was in Response to Someone Harassing Me About My Disability.": A Study of Blind TikTokers’ Content Moderation ExperiencesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642148(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642148
Yu RLiu JZhou ZHuang S(2024)NeRF-Enhanced Outpainting for Faithful Field-of-View Extrapolation2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10611328(16826-16833)Online publication date: 13-May-2024
https://doi.org/10.1109/ICRA57147.2024.10611328
Kocaballi ASezgin EClark LCarroll JHuang YHuh-Yoo JKim JKocielnik RLee YMamykina LMitchell EMoore RMurali PMynatt EPark SPasta ARichards DSilva LSmriti DSpillane BZhang ZZubatiy T(2022)Design and Evaluation Challenges of Conversational Agents in Health Care and Well-being: Selective Review StudyJournal of Medical Internet Research10.2196/3852524:11(e38525)Online publication date: 15-Nov-2022
https://doi.org/10.2196/38525
Xie JLi NLee SCarroll J(2022)YouTube Videos as Data: Seeing Daily Challenges for People with Visual Impairments During COVID-19Proceedings of the 2022 ACM Conference on Information Technology for Social Good10.1145/3524458.3547224(218-224)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1145/3524458.3547224

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents