research-article

Taming wild behavior: the input observer for obtaining text entry and mouse pointing measures from everyday computer use

Authors:

Jacob WobbrockAuthors Info & Claims

CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 1947 - 1956

https://doi.org/10.1145/2207676.2208338

Published: 05 May 2012 Publication History

Abstract

We present the Input Observer, a tool that can run quietly in the background of users' computers and measure their text entry and mouse pointing performance from everyday use. In lab studies, participants are presented with prescribed tasks, enabling easy identification of speeds and errors. In everyday use, no such prescriptions exist. We devised novel algorithms to segment text entry and mouse pointing input streams into "trials". We are the first to measure errors for unprescribed text entry and mouse pointing. To measure errors, we utilize web search engines, adaptive offline dictionaries, an Automation API, and crowdsourcing. Capturing errors allows us to employ Crossman's (1957) speed-accuracy normalization when calculating Fitts' law throughputs. To validate the Input Observer, we compared its measures from 12 participants over a week of computer use to the same participants' results from a lab study. Overall, in the lab and field, average text entry speeds were 74.47 WPM and 80.59 WPM, respectively. Average uncorrected error rates were near zero, at 0.12% and 0.28%. For mouse pointing, average movement times were 971 ms and 870 ms. Average pointing error rates were 4.42% and 4.66%. Average throughputs were 3.48 bits/s and 3.45 bits/s. Device makers, researchers, and assistive technology specialists may benefit from measures of everyday use.

References

[1]

Anderson, T. W. and Darling, D. A. (1954). A test of goodness of fit. J. American Statistical Association 49 (268), 765--769.

[2]

Arthur, D. and Vassilvitskii, S. (2007). k-means++: The advantages of careful seeding. Proc. SODA '07. Philadelphia: Society for Industrial and Applied Mathematics, 1027--1035.

Digital Library

[3]

Balota, D. A., Yap, M. J., Cortese, M. J., Hutchison, K. A., Kessler, B., Loftis, B., Neely, J. H., Nelson, D. L., Simpson, G. B. and Treiman, R. (2007). The English Lexicon Project. Behavior Research Methods 39 (3), 445--459.

[4]

Bradski, G. R. and Pisarevsky, V. (2000). Intel's Computer Vision Library: Applications in calibration, stereo, segmentation, tracking, gesture, face and object recognition. Proc. CVPR '00. Washington, D.C.: IEEE Computer Society, 796--797.

[5]

Brown, B., Reeves, S. and Sherwood, S. (2011). Into the wild: Challenges and opportunities for field trial methods. Proc. CHI '11. New York: ACM Press, 1657--1666.

Digital Library

[6]

Carter, S., Mankoff, J., Klemmer, S. R. and Matthews, T. (2008). Exiting the cleanroom: On ecological validity and ubiquitous computing. Human-Computer Interaction 23 (1), 47--99.

[7]

Chapuis, O., Blanch, R. and Beaudouin-Lafon, M. (2007). Fitts' law in the wild: A field study of aimed movements. LRI Technical Report Number 1480. Laboratorie de Recherche en Informatique. Orsay, France: Universite de Paris Sud.

[8]

Choudhury, T., Borriello, G., Consolvo, S., Haehnel, D., Harrison, B., Hemingway, B., Hightower, J., Klasnja, P., Koscher, K., LaMarca, A., Landay, J. A., LeGrand, L., Lester, J., Rahimi, A., Rea, A. and Wyatt, D. (2008). The Mobile Sensing Platform: An embedded activity recognition system. IEEE Pervasive Computing 7 (2), 32--41.

Digital Library

[9]

Crossman, E. R. F. W. (1957). The speed and accuracy of simple hand movements. In The Nature and Acquisition of Industrial Skills, Crossman and Seymour (eds.). University of Birmingham: Final Report to the M.R.C./D.S.I.R. Joint Committee on Individual Efficiency in Industry.

[10]

D'Agostino, R. B. (1986). Tests for the normal distribution. In Goodness-of-Fit Techniques, D'Agostino and Stephens (eds.). New York: Marcel Dekker, 367--420.

[11]

Dixon, M. and Fogarty, J. A. (2010). Prefab: Implementing advanced behaviors using pixel-based reverse engineering of interface structure. Proc. CHI '10. New York: ACM Press, 1525--1534.

Digital Library

[12]

Dumont, C., Vincent, C. and Mazer, B. (2002). Development of a standardized instrument to assess computer task performance. American J. Occupational Therapy 56 (1), 60--68.

[13]

Evans, A. and Wobbrock, J. O. (2011). Input Observer: Measuring text entry and pointing performance from naturalistic everyday computer use. Extended Abstracts CHI '11. New York: ACM Press, 1879--1884.

Digital Library

[14]

Fisher, R., Perkins, S., Walker, A. and Wolfart, E. (2003). Gaussian smoothing. Hypermedia Image Processing Reference. Available at http://homepages.inf.ed.ac.uk/rbf/HIPR2/gsmooth.htm

[15]

Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. J. Experimental Psychology 47 (6), 381--391.

[16]

Fitts, P. M. and Radford, B. K. (1966). Information capacity of discrete motor responses under different cognitive sets. J. Experimental Psychology 71 (4), 475482.

[17]

Froehlich, J., Chen, M. Y., Consolvo, S., Harrison, B. and Landay, J. A. (2007). MyExperience: A system for in situ tracing and capturing of user feedback on mobile phones. Proc. MobiSYS '07. New York: ACM Press, 57--70.

Digital Library

[18]

Gajos, K. Z., Weld, D. S. and Wobbrock, J. O. (2010). Automatically generating personalized user interfaces with SUPPLE. Artificial Intelligence 174 (12-13), 910--950.

Digital Library

[19]

Hamerly, G. and Elkan, C. (2004). Learning the k in kmeans. In Advances in Neural Information Processing Systems 16, Thrun, Saul and Schölkopf (eds.). Cambridge, MA: The M.I.T. Press, 281--288.

[20]

Hourcade, J. P. (2006). Learning from preschool children's pointing sub-movements. Proc. IDC '06. New York: ACM Press, 65--72.

Digital Library

[21]

Hurst, A., Hudson, S. E. and Mankoff, J. (2007). Dynamic detection of novice vs. skilled use without a task model. Proc. CHI '07. New York: ACM Press, 271--280.

Digital Library

[22]

Hurst, A., Mankoff, J. and Hudson, S. E. (2008). Understanding pointing problems in real world computing environments. Proc. ASSETS '08. New York: ACM Press, 43--50.

Digital Library

[23]

Hurst, A., Hudson, S. E. and Mankoff, J. (2010). Automatically identifying targets users interact with during real world tasks. Proc. IUI '10. New York: ACM Press, 11--20.

Digital Library

[24]

Hwang, F., Keates, S., Langdon, P. and Clarkson, P. J. (2004). Mouse movements of motion-impaired users: A submovement analysis. Proc. ASSETS '04. New York: ACM Press, 102--109.

Digital Library

[25]

Jansen, A., Findlater, L. and Wobbrock, J. O. (2011). From the lab to the world: Lessons from extending a pointing technique for real-world use. Extended Abstracts CHI '11. New York: ACM Press, 1867--1872.

Digital Library

[26]

Lloyd, S. P. (1982). Least squares quantization in PCM. IEEE Trans. Information Theory 28 (2), 129--137.

Digital Library

[27]

MacKenzie, I. S. (1992). Fitts' law as a research and design tool in human-computer interaction. Human-Computer Interaction 7 (1), 91--139.

Digital Library

[28]

MacKenzie, I. S., Kauppinen, T. and Silfverberg, M. (2001). Accuracy measures for evaluating computer pointing devices. Proc. CHI '01. New York: ACM Press, 9--16.

Digital Library

[29]

MacKenzie, I. S. and Soukoreff, R. W. (2002). Text entry for mobile computing: Models and methods, theory and practice. Human-Computer Interaction 17 (2), 147--198.

[30]

MacKenzie, I. S. and Soukoreff, R. W. (2003). Phrase sets for evaluating text entry techniques. Extended Abstracts CHI '03. New York: ACM Press, 754--755.

Digital Library

[31]

MacKenzie, I. S. and Soukoreff, R. W. (2003). Card, English, and Burr (1978) - 25 years later. Extended Abstracts CHI '03. New York: ACM Press, 760--761.

Digital Library

[32]

MacKenzie, I. S. and Isokoski, P. (2008). Fitts' throughput and the speed-accuracy tradeoff. Proc. CHI '08. New York: ACM Press, 1633--1636.

Digital Library

[33]

Meyer, D. E., Abrams, R. A., Kornblum, S., Wright, C. E. and Smith, J. E. K. (1988). Optimality in human motor performance: Ideal control of rapid aimed movements. Psychological Review 95 (3), 340--370.

[34]

Mithal, A. K. and Douglas, S. A. (1996). Differences in movement microstructure of the mouse and the finger-controlled isometric joystick. Proc. CHI '96. New York: ACM Press, 300--307.

Digital Library

[35]

Murata, A. (1996). Empirical evaluation of performance models of pointing accuracy and speed with a PC mouse. Int'l J. Human-Computer Interaction 8 (4), 457--469.

Digital Library

[36]

Palen, L. and Salzman, M. (2002). Voice-mail diary studies for naturalistic data capture under mobile conditions. Proc. CSCW '02. New York: ACM Press, 87--95.

Digital Library

[37]

Rogers, Y. (2011). Interaction design gone wild: Striving for wild theory. interactions 18 (4), 58--62.

Digital Library

[38]

Rosenbaum, D. A. (1991). Aiming. In Human Motor Control. San Diego, CA: Academic Press, 205--216.

[39]

Sears, A., Lin, M., Jacko, J. and Xiao, Y. (2003). When computers fade... Pervasive computing and situationally-induced impairments and disabilities. Proc. HCI Int'l '03. Mahwah, NJ: Lawrence Erlbaum, 1298--1302.

[40]

Smith, R. (2007). An overview of the Tesseract OCR engine. Proc. ICDAR '07. Washington, D.C.: IEEE Computer Society, 629--633.

Digital Library

[41]

Soukoreff, R. W. and MacKenzie, I. S. (2001). Measuring errors in text entry tasks: An application of the Levenshtein string distance statistic. Extended Abstracts CHI '01. New York: ACM Press, 319--320.

Digital Library

[42]

Soukoreff, R. W. and MacKenzie, I. S. (2003). Metrics for text entry research: An evaluation of MSD and KSPC, and a new unified error metric. Proc. CHI '03. New York: ACM Press, 113--120.

Digital Library

[43]

Soukoreff, R. W. and MacKenzie, I. S. (2004). Towards a standard for pointing device evaluation, perspectives on 27 years of Fitts' law research in HCI. Int'l J. HumanComputer Studies 61 (6), 751--789.

Digital Library

[44]

Stephens, M. A. (1972). EDF statistics for goodness of fit and some comparisons. J. American Statistical Association 69 (347), 730--737.

[45]

Walker, N., Philbin, D. A. and Fisk, A. D. (1997). Agerelated differences in movement control: Adjusting submovement structure to optimize performance. J. Gerontology: Psychological Sciences 52B (1), 40--52.

[46]

Wobbrock, J. O. and Myers, B. A. (2006). Analyzing the input stream for character-level errors in unconstrained text entry evaluations. ACM Trans. Computer-Human Interaction 13 (4), 458--489.

Digital Library

[47]

Wobbrock, J. O. (2007). Measures of text entry performance. In Text Entry Systems, MacKenzie and Tanaka-Ishii (eds.). San Francisco: Morgan Kaufmann, 47--74.

[48]

Wobbrock, J. O., Kane, S. K., Gajos, K. Z., Harada, S. and Froehlich, J. (2011). Ability-based design: Concept, principles, and examples. ACM Trans. Accessible Computing 3 (3), 9:1--27.

Digital Library

[49]

Wobbrock, J. O., Shinohara, K. and Jansen, A. (2011). The effects of task dimensionality, endpoint deviation, throughput calculation, and experiment design on pointing measures and models. Proc. CHI '11. New York: ACM Press, 1639--1648.

Digital Library

[50]

Zhai, S. (2004). Characterizing computer input with Fitts' law parameters - the information and non-information aspects of pointing. Int'l J. Human-Computer Studies 61 (6), 791--809.

Digital Library

[51]

Zhai, S., Kong, J. and Ren, X. (2004). Speed-accuracy tradeoff in Fitts' law tasks - on the equivalency of actual and nominal pointing precision. Int'l J. Human-Computer Studies 61 (6), 823--856.

Digital Library

Cited By

Komninos AFeit ALeiva LLehmann FSimou IMinas DFotopoulos AXenos M(2024)An LLM-driven Transcription Task for Mobile Text Entry StudiesProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701586(264-279)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1145/3701571.3701586
Pandey VKhan NGupta AGajos K(2023)Accuracy and Reliability of At-Home Quantification of Motor Impairments Using a Computer-Based Pointing Task with Children with Ataxia-TelangiectasiaACM Transactions on Accessible Computing10.1145/358179016:1(1-25)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3581790
Akpinar EYeşilada YKaragöz P(2023)Effect of Context on Smartphone Users’ Typing Performance in the WildACM Transactions on Computer-Human Interaction10.1145/357701330:3(1-44)Online publication date: 10-Jun-2023
https://dl.acm.org/doi/10.1145/3577013
Show More Cited By

Index Terms

Taming wild behavior: the input observer for obtaining text entry and mouse pointing measures from everyday computer use
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Input observer: measuring text entry and pointing performance from naturalistic everyday computer use
CHI EA '11: CHI '11 Extended Abstracts on Human Factors in Computing Systems

In this paper we describe the Input Observer, a background application that will be capable of measuring a user's text entry and pointing abilities from everyday computer use "in the wild." The application runs quietly in the background of the user's ...
Trackball text entry for people with motor impairments
CHI '06: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

We present a new gestural text entry method for trackballs. The method uses the mouse cursor and relies on crossing instead of pointing. A user writes in fluid Roman-like unistrokes by ""pulsing"" the trackball in desired letter patterns. We examine ...
Stylus based text input using expanding CIRRIN
AVI '06: Proceedings of the working conference on Advanced visual interfaces

CIRRIN [3] is a stylus based text input technique for mobile devices with a touch sensitive display. In this paper we explore the benefit of expanding the letters of CIRRIN to reduce the overall difficulty of selecting a letter. We adapted the existing ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

May 2012

3276 pages

ISBN:9781450310154

DOI:10.1145/2207676

General Chair:
Joseph A. Konstan
University of Minnesota
,
Program Chairs:
Ed H. Chi
Google
,
Kristina Höök
Mobile Life at KTH

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 May 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '12

Sponsor:

SIGCHI

CHI '12: CHI Conference on Human Factors in Computing Systems

May 5 - 10, 2012

Texas, Austin, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

52
Total Citations
View Citations
558
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)5

Reflects downloads up to 17 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Komninos AFeit ALeiva LLehmann FSimou IMinas DFotopoulos AXenos M(2024)An LLM-driven Transcription Task for Mobile Text Entry StudiesProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701586(264-279)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1145/3701571.3701586
Pandey VKhan NGupta AGajos K(2023)Accuracy and Reliability of At-Home Quantification of Motor Impairments Using a Computer-Based Pointing Task with Children with Ataxia-TelangiectasiaACM Transactions on Accessible Computing10.1145/358179016:1(1-25)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3581790
Akpinar EYeşilada YKaragöz P(2023)Effect of Context on Smartphone Users’ Typing Performance in the WildACM Transactions on Computer-Human Interaction10.1145/357701330:3(1-44)Online publication date: 10-Jun-2023
https://dl.acm.org/doi/10.1145/3577013
Wang TMonacelli ERabreau OBoulesteix CVarillon SGastal ARiman CPeralta H(2023)How does the dedicated software PLEIA provide computer access assessment for people with physical disabilities?Universal Access in the Information Society10.1007/s10209-023-01005-x23:4(1779-1794)Online publication date: 17-Jun-2023
https://doi.org/10.1007/s10209-023-01005-x
Priya KJoshi A(2023)Fitts’ Throughput Vs Empirical Throughput: A Comparative StudyHuman-Computer Interaction – INTERACT 202310.1007/978-3-031-42280-5_28(436-455)Online publication date: 25-Aug-2023
https://doi.org/10.1007/978-3-031-42280-5_28
Kong JZhong MFogarty JWobbrock J(2022)Quantifying Touch: New Metrics for Characterizing What Happens During a TouchProceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3517428.3544804(1-13)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1145/3517428.3544804
Rodrigues ANicolau HSantos ABranco DRainey JVerweij DSmeddinck JMontague KGuerreiro T(2022)Investigating the Tradeoffs of Everyday Text-Entry Collection MethodsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501908(1-15)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501908
Rodrigues ASantos AMontague KNicolau HGuerreiro T(2021)WildKey: A Privacy-Aware Keyboard Toolkit for Data Collection In-The-WildAdjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers10.1145/3460418.3482872(542-545)Online publication date: 21-Sep-2021
https://dl.acm.org/doi/10.1145/3460418.3482872
Schmid PMalacria SCockburn ANancel MIqbal SMacLean KChevalier FMueller S(2020)Interaction InterferencesProceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology10.1145/3379337.3415883(516-528)Online publication date: 20-Oct-2020
https://dl.acm.org/doi/10.1145/3379337.3415883
Lee BNancel MKim SOulasvirta ABernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)AutoGain: Gain Function Adaptation with Submovement Efficiency OptimizationProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376244(1-12)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376244
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents