research-article

Effect of Sense of Direction on Open Card Sorts for Websites

Authors:

Georgia Zafeiriou,

Christos Katsanos,

Alexandros LiapisAuthors Info & Claims

CHI Greece 2021: CHI Greece 2021: 1st International Conference of the ACM Greek SIGCHI Chapter

Article No.: 6, Pages 1 - 8

https://doi.org/10.1145/3489410.3489416

Published: 25 November 2021 Publication History

Abstract

Card sorting is one widely used method to design website information architectures based on an understanding of their users’ mental models. This paper studies the effect of participants’ sense of direction on card sorting results. To this end, a two-phase study for an eshop was conducted. In the first phase, 40 participants rated their sense of direction on the standardized Santa Barbara Sense of Direction scale (SBSOD) and were then involved in an open card sort study. Then, we analyzed the card sort data following a median-split approach based on participants’ self-reported SBSOD score. This resulted in two proposed information architectures for the eshop: one from the SBSOD-low participants and one from the SBSOD-high participants. In the second phase, we first developed two prototypes for the eshop, one for each proposed information architecture. Next, 30 participants were involved in a within-subjects user testing study. Results showed that users interacting with the eshop produced by the SBSOD-high participants were significantly more effective, more efficient and rated the tasks as easier compared to users interacting with the eshop produced by the SBSOD-low participants. No significant difference was observed for users’ perceived usability.

References

[1]

George Adamides, Georgios Christou, Christos Katsanos, Michalis Xenos, and Thanasis Hadzilacos. 2015. Usability guidelines for the design of robot teleoperation: A taxonomy. IEEE Transactions on Human-Machine Systems 45 (2), 256–262.

[2]

William Albert and Thomas S. Tullis. 2013. Measuring the user experience: Collecting, analyzing, and presenting usability metrics (2nd ed.). Morgan Kaufmann.

Digital Library

[3]

Aaron Bangor, Philip Kortum, and James Miller. 2008. An empirical evaluation of the System Usability Scale. International Journal of Human-Computer Interaction 24 (6), 574–594.

[4]

C. L Borgman. 1984. Psychological research in human-computer interaction. Annual review of information science and technology 19, 33–64.

[5]

John Brooke. 1996. SUS: a “quick and dirty” usability scale. In Usability Evaluation in Industry, P. W. Jordan, B Thomas, B. A. Weerdmeester and A. L. McClelland (eds.). Taylor and Francis, London.

[6]

Stefano Bussolon, Barbara Russi, and Fabio Del Missier. 2006. Online card sorting: as good as the paper version. In Proceedings of the 13th European conference on Cognitive ergonomics: trust and control in complex socio-technical systems (ECCE ’06), Association for Computing Machinery, New York, NY, USA, 113–114.

Digital Library

[7]

Miranda G. Capra. 2005. Factor analysis of card sort data: an alternative to hierarchical cluster analysis. In Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting, HFES, Santa Monica, CA, 691–695.

[8]

Brendan Cassidy, Dipti Saurabh Antani, and Janet C C. Read. 2013. Using an open card sort with children to categorize games in a mobile phone application store. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’13), ACM, New York, NY, USA, 2287–2290.

Digital Library

[9]

Barbara S. Chaparro, Veronica D. Hinkle, and Shannon K. Riley. 2008. The usability of computerized card sorting: a comparison of three applications by researchers and end users. Journal of Usability Studies 4 (1), 31–48.

Digital Library

[10]

Jianming Dong, Shirley Martin, and Paul Waldo. 2001. A user input and analysis tool for information architecture. In CHI ’01 extended abstracts on Human factors in computing systems, ACM, Seattle, Washington, 23–24.

Digital Library

[11]

Brian Dorn and Mark Guzdial. 2010. Learning on the job: characterizing the programming knowledge and learning strategies of web designers. In Proceedings of the 28th international conference on Human factors in computing systems (CHI ’10), Atlanta, Georgia, USA, 703–712.

Digital Library

[12]

Ricard E. Downing, Joi L. Moore, and Steven W. Brown. 2005. The effects and interaction of spatial visualization and domain expertise on information seeking. Computers in Human Behavior 21 (2), 195–209.

[13]

Andy P Field. 2009. Discovering statistics using SPSS. SAGE, Los Angeles, [Calif.]; London.

Digital Library

[14]

Michelle E. Harper, Florian Jentsch, Lori Rhodenizer Van Duyne, Kimberly Smith-Jentsch, and Alicia D. Sanchez. 2002. Computerized card sort training tool: Is it comparable to manual card sorting? In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, SAGE Publications Inc, 2049–2053.

[15]

Mary Hegarty, Anthony E Richardson, Daniel R Montello, Kristin Lovelace, and Ilavanil Subbiah. 2002. Development of a self-report measure of environmental spatial ability. Intelligence 30 (5), 425–447.

[16]

William Hudson. 2013. Card sorting. In The Encyclopedia of Human-Computer Interaction, 2nd Ed., Mads Soegaard and Rikke Friis Dam (eds.). Interaction Design Foundation. Retrieved from https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed

[17]

Rebecca Jeong and Sonia Chiasson. 2020. “Lime”, “open lock”, and “blocked”: children's perception of colors, symbols, and words in cybersecurity warnings. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20), ACM, New York, NY, USA, 14 pages.

Digital Library

[18]

Karin Kappel, Martin Tomitsch, Thomas Költringer, and Thomas Grechenig. 2006. Developing user interface guidelines for DVD menus. In CHI ’06 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’06), ACM, New York, NY, USA, 177–182.

Digital Library

[19]

Christos Katsanos, Nikolaos Tselios, and Nikolaos Avouris. 2008. AutoCardSorter: designing the information architecture of a web site using latent semantic analysis. In Proceedings of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, CHI 2008, ACM, Florence, Italy, 875–878.

Digital Library

[20]

Christos Katsanos, Nikolaos Tselios, and Nikolaos Avouris. 2008. Automated semantic elaboration of web site information architecture. Interacting with Computers 20 (6), 535–544.

Digital Library

[21]

Christos Katsanos, Nikolaos Tselios, Nikolaos Avouris, Stavros Demetriadis, Ioannis Stamelos, and Lefteris Angelis. 2019. Cross-study reliability of the open card sorting method. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA ’19), ACM, New York, NY, USA, LBW2718:1-LBW2718:6.

Digital Library

[22]

Christos Katsanos, Nikolaos Tselios, Jorge Goncalves, Tomi Juntunen, and Vassilis Kostakos. 2014. Multipurpose public displays: Can automated grouping of applications and services enhance user experience? International Journal of Human-Computer Interaction 30 (3), 237–249.

[23]

Christos Katsanos, Nikolaos Tselios, and Michalis Xenos. 2012. Perceived usability evaluation of learning management systems: a first step towards standardization of the System Usability Scale in Greek. In 2012 16th Panhellenic Conference on Informatics, 302–307.

Digital Library

[24]

Christina Kelley, Bongshin Lee, and Lauren Wilcox. 2017. Self-tracking for mental wellness: understanding expert perspectives and student experiences. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ’17), ACM, New York, NY, USA, 629–641.

Digital Library

[25]

Lynn T. Kozlowski and Kendall J. Bryant. 1977. Sense of direction, spatial orientation, and cognitive maps. Journal of Experimental Psychology: Human Perception and Performance 3 (4), 590–598.

[26]

Georgios Melissourgos and Christos Katsanos. 2020. CardSorter: Towards an open source tool for online card sorts. In Proceedings of the 24th Pan-Hellenic Conference on Informatics (PCI 2020), ACM, New York, NY, USA, 77–81.

Digital Library

[27]

Ather Nawaz. 2012. A Comparison of card-sorting analysis methods. In Proceedings of the 10th Asia Pacific Conference on Computer-Human Interaction, APCHI ’12, ACM Press, 583–592.

[28]

Jacob Nielsen. 2004. Card Sorting: How many users to test. Alertbox. Retrieved from http://www.useit.com/alertbox/20040719.html

[29]

Konstantina Orfanou, Nikolaos Tselios, and Christos Katsanos. 2015. Perceived usability evaluation of learning management systems: Empirical evaluation of the System Usability Scale. The International Review of Research in Open and Distributed Learning 16 (2), 227–246.

[30]

Sione Paea and Ross Baird. 2018. Information Architecture (IA): Using multidimensional scaling (MDS) and K-Means clustering algorithm for analysis of card sorting data. Journal of Usability Studies 13 (3), 138–157.

Digital Library

[31]

Styliani Pampoukidou and Christos Katsanos. 2021. Test-retest reliability of the open card sorting method. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA ’21), Association for Computing Machinery, New York, NY, USA, Article330:1-Article330:7.

Digital Library

[32]

Celeste Paul. 2008. A modified Delphi approach to a new card sorting methodology. Journal of Usability Studies 4 (1), 7–30.

Digital Library

[33]

Celeste Paul. 2014. Analyzing card-sorting data using graph visualization. Journal of Usability Studies 9 (3), 87–104.

Digital Library

[34]

Helen Petrie, Christopher Power, Paul Cairns, and Cagla Seneler. 2011. Using card sorts for understanding website information architectures: technological, methodological and cultural Issues. In Human-Computer Interaction – INTERACT 2011 (Lecture Notes in Computer Science), Springer Berlin Heidelberg, 309–322.

Digital Library

[35]

Carol Righi, Janice James, Michael Beasley, Donald L. Day, Jean E. Fox, Jennifer Gieber, Chris Howe, and Laconya Ruby. 2013. Card sort analysis best practices. Journal of Usability Studies 8 (3), 69–89.

Digital Library

[36]

Louis Rosenfeld, Peter Morville, and Jorge Arango. 2015. Information architecture: For the Web and beyond (4 edition ed.). O'Reilly Media, Sebastopol, CA.

Digital Library

[37]

Jeff Sauro and Joseph S. Dumas. 2009. Comparison of three one-question, post-task usability questionnaires. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’09), ACM, New York, NY, USA, 1599–1608.

Digital Library

[38]

Hasti Seifi, Michael Oppermann, Julia Bullard, Karon E. MacLean, and Katherine J. Kuchenbecker. 2020. Capturing experts’ mental models to organize a collection of haptic devices: affordances outweigh attributes. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20), ACM, New York, NY, USA, 13 pages.

Digital Library

[39]

Donna Spencer. 2009. Card sorting: designing usable categories. Rosenfeld Media, Brooklyn, N.Y.

[40]

Ikuo Suzuki. 2012. Effects of sense of direction on Internet skill and cognitive maps of the Web. Computers in Human Behavior 28 (1), 120–128.

Digital Library

[41]

Thomas Tullis and J Stetson. 2004. A comparison of questionnaires for assessing website usability. In Usability Professionals Association (UPA) 2004 Conference, 7–11.

[42]

Thomas Tullis and Larry Wood. 2004. How many users are enough for a card-sorting study? In Usability Professionals Association (UPA) 2004 Conference, Minneapolis, MN.

[43]

Thomas Tullis and Larry Wood. 2004. How can you do a card-sorting study with LOTS of cards? In Usability Professionals Association (UPA) 2004 Conference, Minneapolis, MN.

[44]

Jed Wood and Larry Wood. 2008. Card sorting: current practices and beyond. Journal of Usability Studies 4 (1), 1–6.

Digital Library

[45]

Panayiotis Zaphiris, Mariya Ghiawadwala, and Shabana Mughal. 2005. Age-centered research-based web design guidelines. In CHI ’05 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’05), ACM, New York, NY, USA, 1897–1900.

Digital Library

Cited By

Pechlevanoudis CZilidis GKatsanos C(2023)How Many Participants Do You Need for an Open Card Sort? A Case Study of E-commerce WebsitesHuman-Computer Interaction – INTERACT 202310.1007/978-3-031-42293-5_7(80-89)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1007/978-3-031-42293-5_7

Index Terms

Effect of Sense of Direction on Open Card Sorts for Websites
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
    2. Interaction paradigms
  2. Interaction design
    1. Interaction design process and methods
2. Information systems

Index terms have been assigned to the content through auto-classification.

Recommendations

Test-Retest Reliability of the Open Card Sorting Method
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

Open card sorting is a popular method for designing information architectures. However, there is a lack of empirical evidence on the validity and reliability of the method. This paper explores the test-retest reliability of open card sorting. To this end,...
CardSorter: Towards an Open Source Tool for Online Card Sorts
PCI '20: Proceedings of the 24th Pan-Hellenic Conference on Informatics

Structural design of shared information environments is rather challenging. Card sorting constitutes a valuable method towards this end. However, there is a lack of free software tools for conducting card sorts, especially from a distance, and analysing ...
Validity of the Open Card Sorting Method for Producing Website Information Structures
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

When it comes to creating website information structures, open card sorting is the main approach used. However, scientific research for the method's validity is lacking. This paper explores the validity of open card sorting for website structural ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

CHI Greece 2021: CHI Greece 2021: 1st International Conference of the ACM Greek SIGCHI Chapter

November 2021

172 pages

ISBN:9781450385787

DOI:10.1145/3489410

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 November 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CHI Greece 2021

CHI Greece 2021: 1st International Conference of the ACM Greek SIGCHI Chapter

November 25 - 27, 2021

Online (Athens, Greece), Greece

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
84
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pechlevanoudis CZilidis GKatsanos C(2023)How Many Participants Do You Need for an Open Card Sort? A Case Study of E-commerce WebsitesHuman-Computer Interaction – INTERACT 202310.1007/978-3-031-42293-5_7(80-89)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1007/978-3-031-42293-5_7

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents