research-article

Open access

Modelling Learning of New Keyboard Layouts

Authors:

Jussi P. P. Jokinen,

Antti Oulasvirta,

Chaklam Silpasuwanchai,

Xiangshi RenAuthors Info & Claims

CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

Pages 4203 - 4215

https://doi.org/10.1145/3025453.3025580

Published: 02 May 2017 Publication History

Abstract

Predicting how users learn new or changed interfaces is a long-standing objective in HCI research. This paper contributes to understanding of visual search and learning in text entry. With a goal of explaining variance in novices' typing performance that is attributable to visual search, a model was designed to predict how users learn to locate keys on a keyboard: initially relying on visual short-term memory but then transitioning to recall-based search. This allows predicting search times and visual search patterns for completely and partially new layouts. The model complements models of motor performance and learning in text entry by predicting change in visual search patterns over time. Practitioners can use it for estimating how long it takes to reach the desired level of performance with a given layout.

Supplementary Material

MP4 File (p4203-jokinen.mp4)

Download
182.17 MB

References

[1]

John R Anderson. 1976. Language, memory and thought Hillsdale. NJ: LEA (1976).

[2]

John R Anderson. 1995. Learning and memory. Wiley, New York.

[3]

John R Anderson. 2007. How Can the Human Mind Occur in the Physical Universe? Oxford University Press, New York.

[4]

John R Anderson. 2013. The architecture of cognition. Psychology Press.

[5]

John R Anderson, D Bothell, C Lebiere, and M Matessa. 1998. An integrated theory of list memory. Journal Of Memory And Language 38, 4 (1998), 341--380.

[6]

Alan Baddeley. 2012. Working memory: theories, models, and controversies. Annual review of psychology 63 (2012), 1--29.

[7]

Gilles Bailly, Antti Oulasvirta, Duncan P Brumby, and Andrew Howes. 2014. Model of visual search and selection time in linear menus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 3865--3874.

Digital Library

[8]

Gilles Bailly, Antti Oulasvirta, Timo Kötzing, and Sabrina Hoppe. 2013. Menuoptimizer: Interactive optimization of menu systems. In Proceedings of the ACM symposium on User interface software and technology. ACM, 331--342.

Digital Library

[9]

Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2015. Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software 67, 1 (2015), 1--48.

[10]

Xiaojun Bi, Barton A. Smith, and Shumin Zhai. 2010. Quasi-qwerty Soft Keyboard Optimization. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 283--286.

Digital Library

[11]

Xiaojun Bi and Shumin Zhai. 2016. IJQwerty: What Difference Does One Key Change Make? Gesture Typing Keyboard Optimization Bounded by One Key Position Change from Qwerty. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 49--58.

Digital Library

[12]

Michael D Byrne. 2001. ACT-R/PM and menu selection: Applying a cognitive architecture to HCI. International Journal of Human-Computer Studies 55, 1 (2001), 41--84.

Digital Library

[13]

Stuart K Card, Thomas P Moran, and Allen Newell. 1980. Computer text-editing: An information-processing analysis of a routine cognitive skill. Cognitive psychology 12, 1 (1980), 32--74.

[14]

Stuart K Card, Allen Newell, and Thomas P Moran. 1983. The psychology of human-computer interaction. (1983).

[15]

Andy Cockburn, Carl Gutwin, and Saul Greenberg. 2007. A predictive model of menu performance. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 627--636.

Digital Library

[16]

Andy Cockburn, Carl Gutwin, Joey Scarr, and Sylvain Malacria. 2015. Supporting novice to expert transitions in user interfaces. ACM Computing Surveys (CSUR) 47, 2 (2015), 31:1--31:36.

[17]

Nelson Cowan. 2012. Working memory capacity. Psychology press.

[18]

Justin Cuaresma and I Scott MacKenzie. 2013. A study of variations of Qwerty soft keyboards for mobile phones. In Proceedings of the International Conference on Multimedia and Human-Computer Interaction-MHCI. 126.1--126.8.

[19]

Arindam Das and Wolfgang Stuerzlinger. 2012. Comparing Cognitive Effort in Spatial Learning of Text Entry Keyboards and ShapeWriters. In Proceedings of the International Working Conference on Advanced Visual Interfaces. ACM, New York, NY, USA, 649--652.

Digital Library

[20]

Paul A David. 1985. Clio and the Economics of QWERTY. The American economic review 75, 2 (1985), 332--337.

[21]

Mark Dunlop and John Levine. 2012. Multidimensional pareto optimization of touchscreen keyboards for speed, familiarity and improved spell checking. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2669--2678.

Digital Library

[22]

August Dvorak. 1943. There is a better typewriter keyboard. National Business Education Quarterly 11, 51--58 (1943), 66.

[23]

Brian D Ehret. 2002. Learning where to look: Location learning in graphical user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 211--218.

Digital Library

[24]

Janine F. Hay and Larry L. Jacoby. 1999. Separating habit and recollection in young and older adults: effects of elaborative processing and distinctiveness. Psychology and aging 14, 1 (1999), 122--134.

[25]

William E Hick. 1952. On the rate of gain of information. Quarterly Journal of Experimental Psychology 4, 1 (1952), 11--26.

[26]

Joop Hox. 2010. Multilevel analysis: Techniques and Applications (2nd ed.). Routledge, Hove.

[27]

Johan Hulleman. 2009. No need for inhibitory tagging of locations in visual search. Psychonomic Bulletin & Review 16, 1 (2009), 116--120.

[28]

Ray Hyman. 1953. Stimulus information as a determinant of reaction time. Journal of experimental psychology 45, 3 (1953), 188--196.

[29]

Christian P. Janssen and Wayne D. Gray. 2012. When, What, and How Much to Reward in Reinforcement Learning-Based Models of Cognition. Cognitive Science 36, 2 (2012), 333--358.

[30]

Bonnie E John, Konstantine Prevas, Dario D Salvucci, and Ken Koedinger. 2004. Predictive human performance modeling made easy. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 455--462.

Digital Library

[31]

Marcel A Just and Patricia A Carpenter. 1980. A theory of reading: from eye fixations to comprehension. Psychological review 87, 4 (1980), 329--354.

[32]

S. W. Keele. 1986. Motor Control. In Handbook of perception and human performance, & J. P. Thomas K. R. Boff, L. Kaufman (Ed.). Wiley, New York, 30.1--30.60.

[33]

Nina Keith and K Anders Ericsson. 2007. A deliberate practice account of typing proficiency in everyday typists. Journal of Experimental Psychology: Applied 13, 3 (2007), 135--145.

[34]

David Kieras. 2011. The persistent visual store as the locus of fixation memory in visual search tasks. Cognitive Systems Research 12, 2 (2011), 102--112.

Digital Library

[35]

David E Kieras and Anthony J Hornof. 2014. Towards accurate and practical predictive models of active-vision-based visual search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 3875--3884.

Digital Library

[36]

David E Kieras and David E Meyer. 1997. An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Human-computer interaction 12, 4 (1997), 391--438.

Digital Library

[37]

Tuomo Kujala and Dario D Salvucci. 2015. Modeling visual sampling on in-car displays: The challenge of predicting safety-critical lapses of control. International Journal of Human-Computer Studies 79 (2015), 66--78.

Digital Library

[38]

Paul Ung-Joon Lee and Shumin Zhai. 2004. Top-down learning strategies: can they facilitate stylus keyboard learning? International journal of human-computer studies 60, 5 (2004), 585--598.

[39]

Blake MacDonald, Pritam Ranjan, Hugh Chipman, and others. 2015. GPfit: An R package for ?tting a gaussian process model to deterministic simulator outputs. Journal of Statistical Software 64, 12 (2015), 1--23.

[40]

I. Scott Mackenzie. 2002. A note on calculating text entry speed. (2002). Unpublished work. Available online at http://www.yorku.ca/mack/RN-TextEntrySpeed.html.

[41]

I. Scott MacKenzie and Shawn X. Zhang. 1999. The design and evaluation of a high-performance soft keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 25--31.

[42]

I. Scott MacKenzie and Shawn X Zhang. 2001. An empirical investigation of the novice experience with soft keyboards. Behaviour & Information Technology 20, 6 (2001), 411--418.

[43]

Laurent Magnien, Jean Leon Bouraoui, and Nadine Vigouroux. 2004. Mobile Devices: Soft Keyboard Text-entry Enhanced by Visual Cues. In Proceedings of the 1st French-speaking Conference on Mobility and Ubiquity Computing. ACM, New York, NY, USA, 158--165.

Digital Library

[44]

Akira Miyake and Priti Shah. 1999. Models of working memory: Mechanisms of active maintenance and executive control. Cambridge University Press.

[45]

Allen Newell. 1990. Unified theories of cognition. Harvard University Press, Cambridge, Mass.

Digital Library

[46]

Allen Newell and Paul S Rosenbloom. 1981. Mechanisms of skill acquisition and the law of practice. Cognitive skills and their acquisition 1 (1981), 1--55.

[47]

Marketta Niemelä and Pertti Saariluoma. 2003. Layout attributes and recall. Behaviour & information technology 22, 5 (2003), 353--363.

[48]

Antti Oulasvirta, Lari Kärkkäinen, and Jari Laarni. 2005. Expectations and memory in link search. Computers in Human Behavior 21, 5 (2005), 773--789.

[49]

Antti Oulasvirta, Anna Reichel, Wenbin Li, Yan Zhang, and Myroslav Bachynskyi. 2013. Improving Two Thumb Text Entry on Touchscreen Devices. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2013), 2765--2774. http://10.1145/2470654.2481383

Digital Library

[50]

Andriy Pavlovych and Wolfgang Stuerzlinger. 2004. Model for non-expert text entry speed on 12-button phone keypads. In Proceedings of the Conference on Human Factors in Computing Systems. 351--358.

Digital Library

[51]

David Peebles and Corinna Jones. A model of object location memory. In Proceedings of the Annual Conference of the Cognitive Science Society.

[52]

Robert A Rescorla and Allan R Wagner. 1972. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. Classical conditioning II: Current research and theory 2 (1972), 64--99.

[53]

Ladislao Salmerón, José J Cañas, and Inmaculada Fajardo. 2005. Are expert users always better searchers' Interaction of expertise and semantic grouping in hypertext search tasks. Behaviour & information technology 24, 6 (2005), 471--475.

[54]

Timothy A Salthouse. 1984. Effects of age and skill in typing. Journal of Experimental Psychology: General 113, 3 (1984), 345--371.

[55]

Dario D Salvucci. 2001. An integrated model of eye movements and visual encoding. Cognitive Systems Research 1, 4 (2001), 201--220.

Digital Library

[56]

Dario D Salvucci and Kristen L Macuga. 2002. Predicting the Effects of Cell-Phone Dialing on Driver Performance. Cognitive Systems Research 3, 1 (2002), 95--102.

[57]

Andrew Sears, Julie A Jacko, Josey Chu, and Francisco Moro. 2001. The role of visual search in the design of effective soft keyboards. Behaviour & Information Technology 20, 3 (2001), 159--166.

[58]

Amanda L. Smith and B. S. Chaparro. 2015. Smartphone Text Input Method Performance, Usability, and Preference With Younger and Older Adults. Human Factors: The Journal of the Human Factors and Ergonomics Society 57, 6 (2015), 1015--1028.

[59]

Leong-hwee Teo, Bonnie E John, and Marilyn Hughes Blackmon. 2012. CogTool-Explorer: A Model of Goal-Directed User Exploration that Considers Information Layout. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2012), 2479--2488.

Digital Library

[60]

Kashyap Todi, Daryl Weir, and Antti Oulasvirta. 2016. Sketchplore: Sketch and Explore with a Layout Optimiser. In Proceedings of the ACM Conference on Designing Interactive Systems. ACM, 543--555.

Digital Library

[61]

Vladislav D. Veksler, Christopher W. Myers, and Kevin A. Gluck. 2014. SAwSu: An Integrated Model of Associative and Reinforcement Learning. Cognitive Science 38, 3 (2014), 580--598.

[62]

Viswanath Venkatesh, James YL Thong, and Xin Xu. 2016. Unified Theory of Acceptance and Use of Technology: A Synthesis and the Road Ahead. Journal of the Association for Information Systems 17, 5 (2016), 328.

[63]

R William Soukoreff and I. Scott Scott Mackenzie. 1995. Theoretical upper and lower bounds on typing speed using a stylus and a soft keyboard. Behaviour & Information Technology 14, 6 (1995), 370--379.

[64]

Eldad Yechiam, Ido Erev, Vered Yehene, and Daniel Gopher. 2003. Melioration and the transition from touch-typing training to everyday use. Human Factors: The Journal of the Human Factors and Ergonomics Society 45, 4 (2003), 671--684.

[65]

Shumin Zhai, Michael Hunter, and Barton A Smith. 2002a. Performance optimization of virtual keyboards. Human--Computer Interaction 17, 2--3 (2002), 229--269.

[66]

Shumin Zhai, Alison Sue, and Johnny Accot. 2002b. Movement model, hits distribution and learning in virtual keyboarding. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2002), 17--24.

Digital Library

Cited By

Shi DZhu YJokinen JAcharya APutkonen AZhai SOulasvirta A(2024)CRTypist: Simulating Touchscreen Typing Behavior via Computational RationalityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642918(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642918
Yu DCibulskis MMortensen EChristensen MBergström J(2024)Metrics of Motor Learning for Analyzing Movement Mapping in Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642354(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642354
Flores GNahapetian A(2024)Keyboardless Keyboard: Smart Phone Gyroscope for Improved User Interface2024 IEEE 21st Consumer Communications & Networking Conference (CCNC)10.1109/CCNC51664.2024.10454832(472-477)Online publication date: 6-Jan-2024
https://doi.org/10.1109/CCNC51664.2024.10454832
Show More Cited By

Index Terms

Modelling Learning of New Keyboard Layouts
1. Human-centered computing

Recommendations

Individualising Graphical Layouts with Predictive Visual Search Models
Special Issue on IUI 2018

In domains where users are exposed to large variations in visuo-spatial features among designs, they often spend excess time searching for common elements (features) on an interface. This article contributes individualised predictive models of visual ...
Influence of Visual Salience on Webpage Product Searches

Visual salience can increase search efficiency in complex displays but does that influence persist when completing a specific search? In two experiments, participants were asked to search webpages for the prices of specific products. Those products were ...
Improvements to keyboard optimization with integer programming
UIST '14: Proceedings of the 27th annual ACM symposium on User interface software and technology

Keyboard optimization is concerned with the design of keyboards for different terminals, languages, user groups, and tasks. Previous work in HCI has used random search based methods, such as simulated annealing. These "black box" approaches are ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

May 2017

7138 pages

ISBN:9781450346559

DOI:10.1145/3025453

General Chairs:
Gloria Mark
University of California Irvine
,
Susan Fussell
Cornell University
,
Program Chairs:
Cliff Lampe
University of Michigan
,
m.c. schraefel
University of Southampton
,
Juan Pablo Hourcade
University of Iowa
,
Caroline Appert
Université Paris-Sud
,
Daniel Wigdor
University of Toronto

Copyright © 2017 Owner/Author.

This work is licensed under a Creative Commons Attribution-ShareAlike International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Best Paper

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

CHI '17

Sponsor:

SIGCHI

CHI '17: CHI Conference on Human Factors in Computing Systems

May 6 - 11, 2017

Colorado, Denver, USA

Acceptance Rates

CHI '17 Paper Acceptance Rate 600 of 2,400 submissions, 25%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

40
Total Citations
View Citations
2,772
Total Downloads

Downloads (Last 12 months)402
Downloads (Last 6 weeks)30

Reflects downloads up to 26 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Shi DZhu YJokinen JAcharya APutkonen AZhai SOulasvirta A(2024)CRTypist: Simulating Touchscreen Typing Behavior via Computational RationalityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642918(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642918
Yu DCibulskis MMortensen EChristensen MBergström J(2024)Metrics of Motor Learning for Analyzing Movement Mapping in Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642354(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642354
Flores GNahapetian A(2024)Keyboardless Keyboard: Smart Phone Gyroscope for Improved User Interface2024 IEEE 21st Consumer Communications & Networking Conference (CCNC)10.1109/CCNC51664.2024.10454832(472-477)Online publication date: 6-Jan-2024
https://doi.org/10.1109/CCNC51664.2024.10454832
Mutasim ABatmaz AHudhud Mughrabi MStuerzlinger W(2024)The Guided Evaluation MethodInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103317190:COnline publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1016/j.ijhcs.2024.103317
Ahteck A(2023)Gridfinite Possibilities: the Illuminated Button Matrix as Input and Output DeviceProceedings of the 15th Conference on Creativity and Cognition10.1145/3591196.3596615(378-379)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3591196.3596615
Komninos A(2023)RoboType: Realistic Mobile Text Entry Evaluations with Synthetic UsersProceedings of the 25th International Conference on Mobile Human-Computer Interaction10.1145/3565066.3608693(1-7)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1145/3565066.3608693
Mutasim ABatmaz AHudhud Mughrabi MStuerzlinger W(2023)Does Repeatedly Typing the Same Phrase Provide a Good Estimate of Expert Text Entry Performance?Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585647(1-8)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585647
Moon HOulasvirta ALee B(2023)Amortized Inference with User SimulationsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581439(1-20)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581439
Yamanaka S(2022)Utility of Crowdsourced User Experiments for Measuring the Central Tendency of User Performance: A Case of Error-Rate Model Evaluation in a Pointing TaskFrontiers in Artificial Intelligence10.3389/frai.2022.7988925Online publication date: 17-Mar-2022
https://doi.org/10.3389/frai.2022.798892
Murray-Smith ROulasvirta AHowes AMüller JIkkala ABachinski MFleig AFischer FKlar M(2022)What simulation can do for HCI researchInteractions10.1145/356403829:6(48-53)Online publication date: 3-Nov-2022
https://dl.acm.org/doi/10.1145/3564038
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents