abstract

Hybrid Wizard of Oz: Concept Testing a Recommender System

Authors:

Sruthi Viswanathan,

Behrooz Omidvar-Tehrani,

Adrien Bruyat,

Frédéric Roulland,

Antonietta Maria GrassoAuthors Info & Claims

CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 7

https://doi.org/10.1145/3334480.3383097

Published: 25 April 2020 Publication History

Get Access

Abstract

In this paper, we report the results from a Hybrid Wizard of Oz experiment consisting of a critical incident interview and an in-situ simulation. The study aimed at validating the need for a contextualised and personalised Point-Of-Interest (POI) recommender and understanding the detailed user needs for it. Our key findings include: feeling bored as a key trigger to search for POIs, trust issues with the existing recommendation sources, intent to find free activities, information needs on areas-of-interest beyond points-of-interest, support for socialising, and language barriers. With this study, we also exemplify a cost- and time-effective approach for design of intelligent systems.

References

[1]

Carlos Lamsfus, Dan Wang, Aurkene AlzuaSorzabal, and Zheng Xiang. 2015. Going Mobile: Defining Context for On-the-Go Travelers. Journal of Travel Research 54, 6: 691--701. https://doi.org/10.1177/0047287514538839

Crossref

Google Scholar

[2]

David Maulsby, Saul Greenberg, and Richard Mander. 1993. Prototyping an intelligent agent through Wizard of Oz. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '93, 277--284. https://doi.org/10.1145/169059.169215

Digital Library

Google Scholar

[3]

Julie Anne Séguin, Alec Scharff, and Kyle Pedersen. 2019. Triptech: A Method for Evaluating Early Design Concepts. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems - CHI EA '19, 1--8. https://doi.org/10.1145/3290607.3299061

Digital Library

Google Scholar

[4]

Simone Stumpf, Vidya Rajaram, Lida Li, Margaret Burnett, Thomas Dietterich, Erin Sullivan, Russell Drummond, and Jonathan Herlocker. 2007. Toward harnessing user feedback for machine learning. In Proceedings of the 12th international conference on Intelligent user interfaces - IUI '07, 82. https://doi.org/10.1145/1216295.1216316

Digital Library

Google Scholar

[5]

Jacob T. Browne. 2019. Wizard of Oz Prototyping for Machine Learning Experiences. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems - CHI EA '19, 1--6. https://doi.org/10.1145/3290607.3312877

Digital Library

Google Scholar

[6]

Jed R Wood and Larry E Wood. 2008. Card Sorting: Current Practices and Beyond. 4, 1: 6. https://doi.org/10.5555/2835577.2835578

Digital Library

Google Scholar

[7]

Dwayne D. Gremler. 2004. The Critical Incident Technique in Service Research. Journal of Service Research 7, 1: 65--89. https://doi.org/10.1177/1094670504266138

Crossref

Google Scholar

[8]

Barkhuus, L., Wohn, D.Y. Making the city my own: uses and practices of mobile location technologies for exploration of a new city. Pers Ubiquit Comput 23, 269--278 (2019) https://doi.org/10.1007/s00779-018-01191-z

Digital Library

Google Scholar

[9]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2: 77--101. https://doi.org/10.1191/1478088706qp063oa

Crossref

Google Scholar

[10]

Sruthi Viswanathan, Cecile Boulard, and Antonietta Maria Grasso. 2019. Ageing Clouds: Novel yet Natural Support for Urban Exploration. In Companion Publication of the 2019 on Designing Interactive Systems Conference 2019 Companion - DIS '19 Companion, 313--317. https://doi.org/10.1145/3301019.3323885

Digital Library

Google Scholar

[11]

Sihem Amer-Yahia, Ria M. Borromeo, Shady Elbassuoni, Behrooz Omidvar-Tehrani, and Sruthi Viswanathan. 2020. Interactive Generation and Cus-tomization of Travel Packages for Individuals and Groups. In25th Interna-tional Conference on Intelligent User Interfaces Companion (IUI '20 Compan-ion), March 17--20, 2020, Cagliari, Italy.ACM, New York, NY, USA, 2 pages. https://doi.org/10.1145/3379336.3381456

Digital Library

Google Scholar

[12]

Behrooz Omidvar-Tehrani, Sruthi Viswanathan, Frederic Roulland, Jean- Michel Renders. 2020. SAGE: Interactive State-aware Point-of-Interest Rec- ommendation. In WSDM Workshop SUM'20: State-based User Modelling, Feb- ruary 7, 2020, Houston, Texas, TX, USA. ACM, New York, NY, USA, 6 pages. https://www.k4all.org/wpcontent/uploads/2020/01/WSDMSUM20_paper_SA GE_Interactive_State_awarePoint_of_Interest_Recommendation.pdf

Google Scholar

[13]

Jie Bao, Yu Zheng, David Wilkie, and Mohamed Mokbel. Recommendations in location-based social networks: a survey. GeoInformatica, 2015 https://doi.org/10.1007/s10707-014-0220--8

Digital Library

Google Scholar

Cited By

View all

Gautam SZhang XRosson MFarzan RLópez CCardoso Llach DQuercia DMustafa MNiu SWong-Villacrés M(2024)Towards Dynamic Learning: A Framework for Simulating Adaptive Learning SystemsCompanion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing10.1145/3678884.3681913(603-608)Online publication date: 11-Nov-2024
https://dl.acm.org/doi/10.1145/3678884.3681913
Jansen AColombo S(2022)Wizard of Errors: Introducing and Evaluating Machine Learning Errors in Wizard of Oz StudiesExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519684(1-7)Online publication date: 27-Apr-2022
https://dl.acm.org/doi/10.1145/3491101.3519684
Omidvar-Tehrani BViswanathan SRenders J(2020)Interactive and Explainable Point-of-Interest Recommendation using Look-alike GroupsProceedings of the 28th International Conference on Advances in Geographic Information Systems10.1145/3397536.3422238(389-392)Online publication date: 3-Nov-2020
https://dl.acm.org/doi/10.1145/3397536.3422238
Show More Cited By

Index Terms

Hybrid Wizard of Oz: Concept Testing a Recommender System
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Prototyping an intelligent agent through Wizard of Oz
CHI '93: Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems

Turvy is a simulated prototype of an instructible agent. The user teaches it by demonstrating actions and pointing at or talking about relevant data. We formalized our assumptions about what could be implemented, then used the Wizard of Oz to flesh out ...
Designing Ambient Wanderer: Mobile Recommendations for Urban Exploration
DIS '20: Proceedings of the 2020 ACM Designing Interactive Systems Conference

Recommender systems are widely integrated into our everyday activities. These intelligent systems succeed in learning the user's profile to recommend movies, music, news and more. However, for designing context-aware recommendations, new challenges ...
Wizard of Oz experiments and companion dialogues
BCS '10: Proceedings of the 24th BCS Interaction Specialist Group Conference

Novel speech systems such as the conversational agents being developed by the Companions Project (www.companions-project.org) can be simulated using the Wizard of Oz methodology. In this approach technologies that are not yet ready for testing by people ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

4474 pages

ISBN:9781450368193

DOI:10.1145/3334480

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2020

Check for updates

Author Tags

Qualifiers

Abstract

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
498
Total Downloads

Downloads (Last 12 months)50
Downloads (Last 6 weeks)3

Reflects downloads up to 24 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Gautam SZhang XRosson MFarzan RLópez CCardoso Llach DQuercia DMustafa MNiu SWong-Villacrés M(2024)Towards Dynamic Learning: A Framework for Simulating Adaptive Learning SystemsCompanion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing10.1145/3678884.3681913(603-608)Online publication date: 11-Nov-2024
https://dl.acm.org/doi/10.1145/3678884.3681913
Jansen AColombo S(2022)Wizard of Errors: Introducing and Evaluating Machine Learning Errors in Wizard of Oz StudiesExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519684(1-7)Online publication date: 27-Apr-2022
https://dl.acm.org/doi/10.1145/3491101.3519684
Omidvar-Tehrani BViswanathan SRenders J(2020)Interactive and Explainable Point-of-Interest Recommendation using Look-alike GroupsProceedings of the 28th International Conference on Advances in Geographic Information Systems10.1145/3397536.3422238(389-392)Online publication date: 3-Nov-2020
https://dl.acm.org/doi/10.1145/3397536.3422238
Viswanathan SOmidvar-Tehrani BBruyat ARoulland FGrasso AWakkary RAndersen KOdom WDesjardins APetersen M(2020)Designing Ambient WandererProceedings of the 2020 ACM Designing Interactive Systems Conference10.1145/3357236.3395518(1405-1418)Online publication date: 3-Jul-2020
https://dl.acm.org/doi/10.1145/3357236.3395518

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Abstract

References

Cited By

Index Terms

Recommendations

Prototyping an intelligent agent through Wizard of Oz

Designing Ambient Wanderer: Mobile Recommendations for Urban Exploration

Wizard of Oz experiments and companion dialogues

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

HTML Format

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations