research-article

Representatively memorable: sampling the right phrase set to get the text entry experiment right

Authors:

Luis A. Leiva,

Germán Sanchis-TrillesAuthors Info & Claims

CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 1709 - 1712

https://doi.org/10.1145/2556288.2557024

Published: 26 April 2014 Publication History

Get Access

Abstract

In text entry experiments, memorability is a desired property of the phrases used as stimuli. Unfortunately, to date there is no automated method to achieve this effect. As a result, researchers have to use either manually curated English-only phrase sets or sampling procedures that do not guarantee phrases being memorable. In response to this need, we present a novel sampling method based on two core ideas: a multiple regression model over language-independent features, and the statistical analysis of the corpus from which phrases will be drawn. Our results show that researchers can finally use a method to successfully curate their own stimuli targeting potentially any language or domain. The source code as well as our phrase sets are publicly available.

Supplementary Material

MP4 File (p1709-sidebyside.mp4)

Download
92.44 MB

References

[1]

Coleman, M., and Liau, T. L. A computer readability formula designed for machine scoring. Journal of Applied Psychology 60, 1 (1975).

Crossref

Google Scholar

[2]

Danescu-Niculescu-Mizil, C., Cheng, J., Kleinberg, J., and Lee, L. You had me at hello: How phrasing affects memorability. In Proc. ACL (2012).

Digital Library

Google Scholar

[3]

Genzel, D., and Charniak, E. Entropy rate constancy in text. In Proc. ACL (2002).

Digital Library

Google Scholar

[4]

Just, M. A., and Carpenter, P. A. A theory of reading: From eye fixations to comprehension. Psychological Review 87, 1 (1980).

Crossref

Google Scholar

[5]

Keller, F. The entropy rate principle as a predictor of processing effort: An evaluation against eye-tracking data. In Proc. EMNLP (2004).

Google Scholar

[6]

Kristensson, P. O., and Vertanen, K. Performance comparisons of phrase sets and presentation styles for text entry evaluations. In Proc. IUI (2012).

Digital Library

Google Scholar

[7]

MacKenzie, I. S., and Soukoreff, R. W. Text entry for mobile computing: Models and methods, theory and practice. Human-Computer Interaction 17, 2 (2002).

Crossref

Google Scholar

[8]

MacKenzie, I. S., and Soukoreff, R. W. Phrase sets for evaluating text entry techniques. In Proc. CHI EA (2003).

Digital Library

Google Scholar

[9]

Nelder, J. A., and Weddrburn, R. W. M. Generalized linear models. J. R. Statist. Soc. 135, 3 (1972).

Google Scholar

[10]

Paek, T., and Hsu, B.-J. P. Sampling representative phrase sets for text entry experiments: a procedure and public resource. In Proc. CHI (2011).

Digital Library

Google Scholar

[11]

Schwarz, G. E. Estimating the dimension of a model. Annals of Statistics 6, 2 (1978).

Crossref

Google Scholar

[12]

Senter, R. J., and Smith, E. A. Automated readability index. Tech. Rep. AMRL-TR-6620, Wright-Patterson Air Force Base, 1967.

Google Scholar

[13]

Vertanen, K., and Kristensson, P. O. A versatile dataset for text entry evaluations based on genuine mobile emails. In Proc. MobileHCI (2011).

Digital Library

Google Scholar

Cited By

View all

Simou IKomninos A(2023)Modelling human behavior during text entryProceedings of the 25th International Conference on Mobile Human-Computer Interaction10.1145/3565066.3609739(1-3)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1145/3565066.3609739
Komninos ATsiouma AGogoulou GGarofalakis J(2022)Don’t Look Up: The Cost of Attention to Stimulus Phrases in Mobile Text Entry EvaluationsProceedings of the 26th Pan-Hellenic Conference on Informatics10.1145/3575879.3576015(341-346)Online publication date: 25-Nov-2022
https://dl.acm.org/doi/10.1145/3575879.3576015
Wimmer CSchlögl RKappel KGrechenig TPaternò FJacucci GRohs MSantoro C(2019)Measuring mobile text entry performance and behaviour in the wild with a serious gameProceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia10.1145/3365610.3365633(1-11)Online publication date: 26-Nov-2019
https://dl.acm.org/doi/10.1145/3365610.3365633
Show More Cited By

Index Terms

Representatively memorable: sampling the right phrase set to get the text entry experiment right
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Sampling representative phrase sets for text entry experiments: a procedure and public resource
CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Text entry experiments evaluating the effectiveness of various input techniques often employ a procedure whereby users are prompted with natural language phrases which they are instructed to enter as stimuli. For experimental validity, it is desirable ...
A systematic comparison of 3 phrase sampling methods for text entry experiments in 10 languages
MobileHCI '14: Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services

Today's reference datasets for conducting text entry experiments are only available in English, which may lead to misleading results when testing text entry methods with non-native English speakers. We compared 3 automated phrase sampling methods ...
Performance comparisons of phrase sets and presentation styles for text entry evaluations
IUI '12: Proceedings of the 2012 ACM international conference on Intelligent User Interfaces

We empirically compare five different publicly-available phrase sets in two large-scale (N = 225 and N = 150) crowdsourced text entry experiments. We also investigate the impact of asking participants to memorize phrases before writing them versus ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2014

4206 pages

ISBN:9781450324731

DOI:10.1145/2556288

General Chairs:
Matt Jones
Swansea University, Wales, UK
,
Philippe Palanque
Université Paul Sabatier, France
,
Program Chairs:
Albrecht Schmidt
University of Stuttgart, Germany
,
Tovi Grossman
Autodesk Research, Canada

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 April 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '14

Sponsor:

SIGCHI

CHI '14: CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2014

Ontario, Toronto, Canada

Acceptance Rates

CHI '14 Paper Acceptance Rate 465 of 2,043 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
348
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)1

Reflects downloads up to 29 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Simou IKomninos A(2023)Modelling human behavior during text entryProceedings of the 25th International Conference on Mobile Human-Computer Interaction10.1145/3565066.3609739(1-3)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1145/3565066.3609739
Komninos ATsiouma AGogoulou GGarofalakis J(2022)Don’t Look Up: The Cost of Attention to Stimulus Phrases in Mobile Text Entry EvaluationsProceedings of the 26th Pan-Hellenic Conference on Informatics10.1145/3575879.3576015(341-346)Online publication date: 25-Nov-2022
https://dl.acm.org/doi/10.1145/3575879.3576015
Wimmer CSchlögl RKappel KGrechenig TPaternò FJacucci GRohs MSantoro C(2019)Measuring mobile text entry performance and behaviour in the wild with a serious gameProceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia10.1145/3365610.3365633(1-11)Online publication date: 26-Nov-2019
https://dl.acm.org/doi/10.1145/3365610.3365633
Schlögl RWimmer CGrechenig TBrewster SFitzpatrick GCox AKostakos V(2019)Hyper TyperExtended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290607.3313035(1-6)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290607.3313035
Vertanen KGaines DFletcher CStanage AWatling RKristensson PBrewster SFitzpatrick GCox AKostakos V(2019)VelociWatchProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300821(1-14)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300821
Yi XYu CShi WBi XShi YMark GFussell SLampe Cschraefel mHourcade JAppert CWigdor D(2017)Word Clarity as a Metric in Sampling Keyboard Test SetsProceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3025701(4216-4228)Online publication date: 2-May-2017
https://dl.acm.org/doi/10.1145/3025453.3025701
Leiva LSahami ACatala AHenze NSchmidt ABegole BKim JInkpen KWoo W(2015)Text Entry on Tiny QWERTY Soft KeyboardsProceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems10.1145/2702123.2702388(669-678)Online publication date: 18-Apr-2015
https://dl.acm.org/doi/10.1145/2702123.2702388
Joshi ADalvi GJoshi MKumar JIlavarasan PDalvi G(2014)Corpus of Marathi Word Frequencies from Touch-Screen Devices Using Swarachakra Android KeyboardProceedings of the 6th Indian Conference on Human-Computer Interaction10.1145/2676702.2676705(74-79)Online publication date: 7-Dec-2014
https://dl.acm.org/doi/10.1145/2676702.2676705
Sanchis-Trilles GLeiva LQuigley ADiamond SIrani PSubramanian S(2014)A systematic comparison of 3 phrase sampling methods for text entry experiments in 10 languagesProceedings of the 16th international conference on Human-computer interaction with mobile devices & services10.1145/2628363.2634229(537-542)Online publication date: 23-Sep-2014
https://dl.acm.org/doi/10.1145/2628363.2634229

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Sampling representative phrase sets for text entry experiments: a procedure and public resource

A systematic comparison of 3 phrase sampling methods for text entry experiments in 10 languages

Performance comparisons of phrase sets and presentation styles for text entry evaluations