Computer-Assisted Heuristic Evaluation of Data Visualization
Pages 408 - 420
Abstract
Heuristic evaluation has been an important part of data visualization. Many heuristic rules and guidelines for evaluating data visualization have been proposed and reviewed. However, applying heuristic evaluation in practice is not trivial. First, the heuristic rules are discussed in different publications across different disciplines. There is no central repository of heuristic rules for data visualization. There are no consistent guidelines on how to apply them. Second, it is difficult to find multiple experts who are knowledgeable about the heuristic rules, their pitfalls, and counterpoints. To address this issue, we present a computer-assisted heuristic evaluation method for data visualization. Based on this method, we developed a Python-based tool for evaluating plots created by the visualization tool Plotly. Recent advances in declarative data visualization libraries have made it feasible to create such a tool. By providing advice, critiques, and recommendations, this tool serves as a knowledgeable virtual assistant to help data visualization developers evaluate their visualizations as they code.
References
[1]
Altair: Declarative visualization in python. https://altair-viz.github.io/, Accessed 07 Aug 2021
[2]
d3.js. https://d3js.org/, Accessed 07 Aug 2021
[3]
Plotly graphing libraries. https://plotly.com/python/, Accessed 07 Aug 2021
[4]
Aigner W, Hoffmann S, and Rind A Evalbench: a software library for visualization evaluation Comput. Graph. Forum 2013 32 41-50
[5]
Amar, R., Stasko, J.: A knowledge task-based framework for design and evaluation of information visualizations. In: Proceedings of the IEEE Symposium on Information Visualization, pp. 143–149 (2004)
[6]
Barcellos, R., Viterbo, J., Bernardini, F., Trevisan, D.: An instrument for evaluating the quality of data visualizations. In: 2018 22nd International Conference Information Visualisation (IV), pp. 169–174 (2018)
[7]
Bertin J Semiology of Graphics 2010 1 Noida ESRI Press
[8]
Borland D and Taylor RM II Rainbow color map (still) considered harmful IEEE Comput. Graph. Appl. 2007 27 02 14-17
[9]
Bostock M, Ogievetsky V, and Heer J D3 data-driven documents IEEE Trans. Vis. Comput. Graph. 2011 17 2301-2309
[10]
Brath, R., Banissi, E.: Evaluation of visualization by critiques. In: Proceedings of the Sixth Workshop on Beyond Time and Errors on Novel Evaluation Methods for Visualization (BELIV), pp. 19–26. ACM (2016)
[11]
Carpendale S Kerren A, Stasko JT, Fekete J-D, and North C Evaluating information visualizations Information Visualization 2008 Heidelberg Springer 19-45
[12]
Chambers JM, Cleveland WS, Tukey PA, and Kleiner B Graphical Methods for Data Analysis 1983 Pacific Grove Duxbury Press
[13]
Cleveland WS and McGill R Graphical perception: theory, experimentation, and application to the development of graphical methods J. Am. Stat. Assoc. 1984 79 531-554
[14]
Forsell, C.: Evaluation in information visualization: Heuristic evaluation. In: Proceedings of the International Conference on Information Visualisation, pp. 136–142. IEEE (2012)
[15]
Forsell, C., Johansson, J.: An heuristic set for evaluation in information visualization. In: Proceedings of the International Conference on Advanced Visual Interfaces, pp. 199–206. ACM (2010)
[16]
Heer J and Bostock M Declarative language design for interactive visualization IEEE Trans. Vis. Comput. Graph. 2010 16 6 1149-1156
[17]
Hullman J, Adar E, and Shah P Benefitting infovis with visual difficulties IEEE Trans. Vis. Comput. Graph. 2011 17 2213-2222
[18]
Isenberg T, Isenberg P, Chen J, Sedlmair M, and Moller T A systematic review on the practice of evaluating visualization IEEE Trans. Vis. Comput. Graph. 2013 19 2818-2827
[19]
Kosara, R.: An empire built on sand: reexamining what we think we know about visualization. In: Proceedings of the Sixth Workshop on Beyond Time and Errors on Novel Evaluation Methods in Visualization, pp. 162–168. ACM (2016)
[20]
Kosara R, Drury F, Holmquist LE, and Laidlaw DH Visualization criticism IEEE Comput. Graph. Appl 2008 28 13-15
[21]
Lam H, Bertini E, Isenberg P, Plaisant C, and Carpendale S Empirical studies in information visualization: seven scenarios IEEE Trans. Vis. Comput. Graph. 2012 18 1520-1536
[22]
Larkin JH and Simon HA Why a diagram is (sometimes) worth ten thousand words Cogn. Sci. 1987 11 65-100
[23]
Lücke-Tieke, H., Beuth, M., Schader, P., May, T., Bernard, J., Kohlhammer, J.: Lowering the barrier for successful replication and evaluation. In: 2018 IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV), pp. 60–68 (2018)
[24]
Munzner T A nested model for visualization design and validation IEEE Trans. Vis. Comput. Graph. 2009 15 921-928
[25]
Nielsen, J.: Heuristic evaluation. In: Nielsen, J., Mack, R.L. (eds.) Usability Inspection Methods. Wiley, Hoboken (1994)
[26]
Nielsen, J.: Finding usability problems through heuristic evaluation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 373–380. ACM (1992)
[27]
Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 249–256. ACM (1990)
[28]
Parish CM and Edmondson PD Data visualization heuristics for the physical sciences Mater. Des. 2019 179 107868
[29]
Rougier NP, Droettboom M, and Bourne PE Ten simple rules for better figures PLOS Comput. Biol. 2014 10 9 1-7
[30]
Santos BS, Ferreira BQ, and Dias P Kurosu M Heuristic evaluation in information visualization using three sets of heuristics: an exploratory study Human-Computer Interaction: Design and Evaluation 2015 Cham Springer 259-270
[31]
Santos BS, Ferreira BQ, and Dias P Using heuristic evaluation to foster visualization analysis and design skills IEEE Comput. Graph. Appl 2016 36 86-90
[32]
Santos, B.S., Silva, S., Dias, P.: Heuristic evaluation in visualization: an empirical study. In: 2018 IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV), pp. 78–85 (2018)
[33]
Satyanarayan A, Moritz D, Wongsuphasawat K, and Heer J Vega-lite: a grammar of interactive graphics IEEE Trans. Vis. Comput. Graph. 2017 23 1 341-350
[34]
Satyanarayan A, Russell R, Hoffswell J, and Heer J Reactive vega: a streaming dataflow architecture for declarative interactive visualization IEEE Trans. Vis. Comput. Graph. 2016 22 659-668
[35]
Satyanarayan, A., Wongsuphasawat, K., Heer, J.: Declarative interaction design for data visualization. In: Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, pp. 669–678. ACM (2014)
[36]
Scholtz J Developing guidelines for assessing visual analytics environments Inf. Vis. 2011 10 212-231
[37]
Senay, H., Ignatius, E.: Rules and principles of scientific visualization. Technical report, Department of Electrical Engineering and Computer Science, George Washington University (1990). http://www6.uniovi.es/hypvis/percept/visrules.htm, Accessed 07 Aug 2021
[38]
Shneiderman, B.: The eyes have it: a task by data type taxonomy for information visualizations. In: Proceedings of IEEE Symposium on Visual Languages, pp. 336–343 (1996)
[39]
Tarrell, A., Forsell, C., Fruhling, A., Grinstein, G., Borgo, R., Scholtz, J.: Toward visualization-specific heuristic evaluation. In: Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization. ACM (2014)
[40]
Tory M and Möller T Evaluating visualizations: do expert reviews work? IEEE Comput. Graph. Appl. 2005 25 8-11
[41]
Tufte ER Visual Explanations: Images and Quantities, Evidence and Narrative 1997 Cheshire Graphics Press
[42]
Tufte ER The Visual Display of Quantitative Information 2011 2 Cheshire Graphics Press
[43]
Tversky B, Morrison JB, and Betrancourt M Animation: can it facilitate? Int. J. Hum.-Comput. Stud. 2002 57 247-262
[44]
Väätäjä, H., et al.: Information visualization heuristics in practical expert evaluation. In: Proceedings of the Sixth Workshop on Beyond Time and Errors on Novel Evaluation Methods for Visualization, pp. 36–43. ACM (2016)
[45]
Wall E A heuristic approach to value-driven evaluation of visualizations IEEE Trans. Vis. Comput. Graph. 2019 25 491-500
[46]
Williams R, Scholtz J, Blaha LM, Franklin L, and Huang Z Kurosu M Evaluation of visualization heuristics Human-Computer Interaction. Theories, Methods, and Human Issues 2018 Cham Springer 208-224
[47]
Zhou, M.X., Feiner, S.K.: Top-down hierarchical planning of coherent visual discourse. In: Proceedings of the International Conference on Intelligent User Interface (IUI), pp. 129–136. ACM (1997)
[48]
Zuk, T., Schlesier, L., Neumann, P., Hancock, M., Carpendale, S.: Heuristics for information visualization evaluation. In: Proceedings of the 2006 AVI Workshop on Beyond Time and Errors: Novel Evaluation Methods for Information Visualization, pp. 1–6 (2006)
Index Terms
- Computer-Assisted Heuristic Evaluation of Data Visualization
Index terms have been assigned to the content through auto-classification.
Recommendations
A divide-and-conquer local search heuristic for data visualization
Data visualization techniques have become important tools for analyzing large multidimensional data sets and providing insights with respect to scientific, economic, and engineering applications. Typically, these visualization applications are modeled ...
TimeLine and visualization of multiple-data sets and the visualization querying challenge
Data in its raw form can potentially contain valuable information, but much of that value is lost if it cannot be presented to a user in a way that is useful and meaningful. Data visualization techniques offer a solution to this issue. Such methods are ...
Effective Big Data Visualization
IDEAS '17: Proceedings of the 21st International Database Engineering & Applications SymposiumIn the last several years, big data analytics has found an increasing role in our everyday lives. Data visualization has long been accepted as an integral part of data analytics. However, data visualization systems are not equipped to handle the ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Oct 2021
554 pages
ISBN:978-3-030-90435-7
DOI:10.1007/978-3-030-90436-4
© Springer Nature Switzerland AG 2021.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 04 October 2021
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 30 Nov 2024
Other Metrics
Citations
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in