research-article

Pre-design empiricism for information visualization: scenarios, methods, and challenges

Authors:

Matthew Brehmer,

Sheelagh Carpendale,

Melanie ToryAuthors Info & Claims

BELIV '14: Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization

Pages 147 - 151

https://doi.org/10.1145/2669557.2669564

Published: 10 November 2014 Publication History

Abstract

Empirical study can inform visualization design, both directly and indirectly. Pre-design empirical methods can be used to characterize work practices and their associated problems in a specific domain, directly motivating design choices during the subsequent development of a specific application or technique. They can also be used to understand how individuals, existing tools, data, and contextual factors interact, indirectly informing later research in our community. Contexts for empirical study vary and practitioners should carefully consider finding the most appropriate methods for any given situation. This paper discusses some of the challenges associated with conducting pre-design studies by way of four illustrative scenarios, highlighting the methods as well as the challenges unique to the visualization domain. We encourage researchers and practitioners to conduct more pre-design empirical studies and describe in greater detail their use of empirical methods for informing design.

References

[1]

Brehmer, M., Ingram, S., Stray, J., and Munzner, T. Overview: The design, adoption, and analysis of a visual document mining system for investigative journalists. To Appear in IEEE TVCG (Proc. InfoVis), (2014).

[2]

Brehmer, M. and Munzner, T. A multi-level typology of abstract visualization tasks. IEEE TVCG (Proc. InfoVis) 19, 12 (2013), 2376--2385.

Digital Library

[3]

Brehmer, M., Sedlmair, M., Ingram, S., and Munzner, T. Visualizing dimensionally-reduced data: Interviews with analysts and a characterization of task sequences. To appear in Proc. ACM BELIV Workshop, (2014).

Digital Library

[4]

Brewer, I., MacEachren, A. M., Abdo, H., Gundrum, J., and Otto, G. Collaborative geographic visualization: Anabling shared understanding of environmental processes. Proc. IEEE Symp. InfoVis, (2000), 137--141.

Digital Library

[5]

Buxton, W. Sketching user experiences: Getting the design right and the right design. Morgan Kaufmann, 2007.

Digital Library

[6]

Carpendale, S. Evaluating information visualizations. In Information Visualization: Human-Centered Issues and Perspectives. Springer, 2008, 19--45.

Digital Library

[7]

Collins, C. Interactive Visualizations of Natural Language. Doctoral Thesis, University of Toronto, 2010.

Digital Library

[8]

Erbacher, R. F., Frincke, D. A., Chung Wong, P., Moody, S., and Fink, G. A multi-phase network situational awareness cognitive task analysis. Information Visualization 9, 3 (2010), 204--219.

Digital Library

[9]

Furniss, D., Blandford, A., Curzon, P., and Mary, Q. Confessions from a grounded theory PhD: Experiences and lessons learnt. Proc. ACM CHI, (2011), 113--122.

Digital Library

[10]

Gaver, B., Dunne, T., and Pacenti, E. Design: cultural probes. interactions 6, 1 (1999), 21--29.

Digital Library

[11]

Grammel, L., Tory, M., and Storey, M.-A. How information visualization novices construct visualizations. IEEE TVCG (Proc. InfoVis) 16, 6 (2010), 943--952.

Digital Library

[12]

Huang, D., Tory, M., and Bartram, L. Data in everyday life: Visualizing time-varying data on a calendar. To appear in Proc. Poster Compendium IEEE VIS, (2014).

[13]

IDEO. Method Cards App. 2010. http://www.ideo.com/work/ideo-method-card-app.

[14]

Isenberg, P., Tang, A., and Carpendale, S. An exploratory study of visual information analysis. Proc. ACM CHI, (2008), 1217--1226.

Digital Library

[15]

Isenberg, P., Zuk, T., Collins, C., and Carpendale, S. Grounded evaluation of information visualizations. Proc. ACM BELIV Workshop, (2008).

Digital Library

[16]

Kandel, S., Paepcke, A., Hellerstein, J. M., and Heer, J. Enterprise data analysis and visualization: An interview study. IEEE TVCG (Proc. VAST) 18, 12 (2012), 2917--2926.

[17]

Lam, H., Bertini, E., Isenberg, P., Plaisant, C., and Carpendale, S. Empirical studies in information visualization: Seven scenarios. IEEE TVCG 18, 9 (2012), 1520--1536.

Digital Library

[18]

McGrath, J. E. Methodology matters: Doing research in the behavioral and social sciences. In Readings in Human-Computer Interaction: Towards the Year 2000. Morgan Kaufmann, 1995, 152--169.

Digital Library

[19]

McKenna, S., Mazur, D., Agutter, J., and Meyer, M. Design activity framework for visualization design. To Appear in IEEE TVCG (Proc. InfoVis), (2014).

[20]

Metoyer, R., Lee, B., Riche, N. H., and Czerwinski, M. Understanding the verbal language and structure of end-user descriptions of data visualizations. Proc. ACM CHI, (2012), 1659--1662.

Digital Library

[21]

Morris, M. R., Danielescu, A., Drucker, S., Fisher, D., Lee, B., schraefel, M. C., and Wobbrock, J. O. Reducing legacy bias in gesture elicitation studies. Interactions 21, 3 (2014), 40--45.

Digital Library

[22]

Munzner, T. A nested model for visualization design and validation. IEEE TVCG (Proc. InfoVis) 15, 6 (2009), 921--928.

Digital Library

[23]

Nielsen, J. Noncommand user interfaces. Comm. ACM 36, 4 (1993), 89--99.

Digital Library

[24]

Plaisant, C. The challenge of information visualization evaluation. Proc. ACM AVI, (2004), 109--117.

Digital Library

[25]

Pretorius, A. J. and van Wijk, J. J. Visual inspection of multivariate graphs. Computer Graphics Forum (Proc. EuroVis) 27, 3 (2008), 967--974.

Digital Library

[26]

Rodgers, J. and Bartram, L. Exploring ambient and artistic visualization for residential energy use feedback. IEEE TVCG (Proc. InfoVis) 17, 12 (2011), 2489--2497.

Digital Library

[27]

Sedlmair, M., Isenberg, P., Baur, D., and Butz, A. Information visualization evaluation in large companies: Challenges, experiences and recommendations. Information Visualization 10, 3 (2011), 248--266.

Digital Library

[28]

Sedlmair, M., Meyer, M., and Munzner, T. Design study methodology: Reflections from the trenches and the stacks. IEEE TVCG (Proc. InfoVis) 18, 12 (2012), 2431--2440.

[29]

Sprague, D. and Tory, M. Exploring how and why people use visualizations in casual contexts: Modeling user goals and regulated motivations. Information Visualization 11, 2 (2012), 106--123.

Digital Library

[30]

Tory, M. and Staub-French, S. Qualitative analysis of visualization: A building design field study. Proc. ACM BE, (2008).

Digital Library

[31]

Vicente, K. J. Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work. CRC Press, 1999.

[32]

Walny, J., Lee, B., Johns, P., Riche, N. H., and Carpendale, S. Understanding pen and touch interaction for data exploration on interactive whiteboards. IEEE TVCG (Proc. InfoVis) 18, 12 (2012), 2779--2788.

[33]

Xie, S., Kraemer, E., Stirewalt, R. E. K., Dillon, L. K., and Fleming, S. D. Design and evaluation of extensions to UML sequence diagrams for modeling multithreaded interactions. Information Visualization 8, 2 (2009), 120--136.

Digital Library

[34]

Zuk, T. Visualizing Uncertainty. Doctoral Thesis, University of Calgary, 2008.

Digital Library

Cited By

Waseem KSturdee MWang ZLewis RTonelli MCarpendale S(2024)Exploring Using Personalised Comics for Healthcare Communication for Patients Living With HemodialysisProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661506(859-873)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661506
Crisan AShang MBrochu E(2024)Eliciting Model Steering Interactions From Users via Data and Visual Design ProbesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332289830:9(6005-6019)Online publication date: Sep-2024
https://doi.org/10.1109/TVCG.2023.3322898
Romero-Organvidez DHorcas JGalindo JBenavides D(2024)Data visualization guidance using a software product line approachJournal of Systems and Software10.1016/j.jss.2024.112029213:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.jss.2024.112029
Show More Cited By

Index Terms

Pre-design empiricism for information visualization: scenarios, methods, and challenges
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

Action Design Research and Visualization Design
BELIV '16: Proceedings of the Sixth Workshop on Beyond Time and Errors on Novel Evaluation Methods for Visualization

In applied visualization research, artifacts are shaped by a series of small design decisions, many of which are evaluated quickly and informally via methods that often go unreported and unverified. Such design decisions are influenced not only by ...
Casual Information Visualization: Depictions of Data in Everyday Life

Information visualization has often focused on providing deep insight for expert user populations and on techniques for amplifying cognition through complicated interactive visual models. This paper proposes a new subdomain for infovis research that ...
Software Design Patterns for Information Visualization

Despite a diversity of software architectures supporting information visualization, it is often difficult to identify, evaluate, and re-apply the design solutions implemented within such frameworks. One popular and effective approach for addressing such ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

BELIV '14: Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization

November 2014

184 pages

ISBN:9781450332095

DOI:10.1145/2669557

Editors:
Heidi Lam
Google Inc.
,
Petra Isenberg
INRIA, France
,
Tobias Isenberg
INRIA, France
,
Michael Sedlmair
University of Vienna, Austria

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 November 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

BELIV '14

BELIV '14: Novel Evaluation Methods For Visualization 2014

November 10, 2014

Paris, France

Acceptance Rates

BELIV '14 Paper Acceptance Rate 23 of 30 submissions, 77%;

Overall Acceptance Rate 45 of 64 submissions, 70%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
450
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)1

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Waseem KSturdee MWang ZLewis RTonelli MCarpendale S(2024)Exploring Using Personalised Comics for Healthcare Communication for Patients Living With HemodialysisProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661506(859-873)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661506
Crisan AShang MBrochu E(2024)Eliciting Model Steering Interactions From Users via Data and Visual Design ProbesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332289830:9(6005-6019)Online publication date: Sep-2024
https://doi.org/10.1109/TVCG.2023.3322898
Romero-Organvidez DHorcas JGalindo JBenavides D(2024)Data visualization guidance using a software product line approachJournal of Systems and Software10.1016/j.jss.2024.112029213:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.jss.2024.112029
Horcas JGalindo JBenavides DFelfernig AFuentes L(2022)Variability in data visualizationProceedings of the 26th ACM International Systems and Software Product Line Conference - Volume A10.1145/3546932.3546993(55-66)Online publication date: 12-Sep-2022
https://dl.acm.org/doi/10.1145/3546932.3546993
Coenen JMoere A(2021)Public Data Visualization: Analyzing Local Running Statistics on Situated DisplaysComputer Graphics Forum10.1111/cgf.1429740:3(159-171)Online publication date: 29-Jun-2021
https://doi.org/10.1111/cgf.14297
Reynolds SMertz TArzt SKohlhammer J(2021)User-Centered Design of Visualizations for Software Vulnerability Reports2021 IEEE Symposium on Visualization for Cyber Security (VizSec)10.1109/VizSec53666.2021.00013(68-78)Online publication date: Oct-2021
https://doi.org/10.1109/VizSec53666.2021.00013
Hall KBradley AHinrichs UHuron SWood JCollins CCarpendale S(2020)Design by Immersion: A Transdisciplinary Approach to Problem-Driven VisualizationsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2019.293479026:1(109-118)Online publication date: Jan-2020
https://doi.org/10.1109/TVCG.2019.2934790
Crisan AMunzner T(2019)Uncovering Data Landscapes through Data Reconnaissance and Task Wrangling2019 IEEE Visualization Conference (VIS)10.1109/VISUAL.2019.8933542(46-50)Online publication date: Oct-2019
https://doi.org/10.1109/VISUAL.2019.8933542
Theis SRasche PBröhl CWille MMertens A(2018)Task-Data Taxonomy for Health Data Visualizations: Web-Based Survey With Experts and Older AdultsJMIR Medical Informatics10.2196/medinform.93946:3(e39)Online publication date: 9-Jul-2018
https://doi.org/10.2196/medinform.9394
Borgo RMicallef LBach BMcGee FLee B(2018)Information Visualization Evaluation Using CrowdsourcingComputer Graphics Forum10.1111/cgf.1344437:3(573-595)Online publication date: 10-Jul-2018
https://doi.org/10.1111/cgf.13444
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents