research-article

Scalability in Visualization

Authors:

Gaëlle Richer,

Moataz Abdelaal,

Jean-Daniel Fekete,

Michael Sedlmair,

Daniel WeiskopfAuthors Info & Claims

IEEE Transactions on Visualization and Computer Graphics, Volume 30, Issue 7

Pages 3314 - 3330

https://doi.org/10.1109/TVCG.2022.3231230

Published: 29 December 2022 Publication History

Abstract

We introduce a conceptual model for scalability designed for visualization research. With this model, we systematically analyze over 120 visualization publications from 1990 to 2020 to characterize the different notions of scalability in these works. While many article have addressed scalability issues, our survey identifies a lack of consistency in the use of the term in the visualization research community. We address this issue by introducing a consistent terminology meant to help visualization researchers better characterize the scalability aspects in their research. It also helps in providing multiple methods for supporting the claim that a work is “scalable.” Our model is centered around an effort function with inputs and outputs. The inputs are the problem size and resources, whereas the outputs are the actual efforts, for instance, in terms of computational run time or visual clutter. We select representative examples to illustrate different approaches and facets of what scalability can mean in visualization literature. Finally, targeting the diverse crowd of visualization researchers without a scalability tradition, we provide a set of recommendations for how scalability can be presented in a clear and consistent way to improve fair comparison between visualization techniques and systems and foster reproducibility.

References

[1]

C. R. Johnson, “Top scientific visualization research problems,” IEEE Comput. Graph. Appl., vol. 24, no. 4, pp. 13–17, Jul./Aug. 2004.

Digital Library

[2]

P. C. Wong, H.-W. Shen, C. R. Johnson, C. Chen, and R. B. Ross, “The top 10 challenges in extreme-scale visual analytics,” IEEE Comput. Graph. Appl., vol. 32, no. 4, pp. 63–67, Jul./Aug. 2012.

Digital Library

[3]

J. J. Thomas and K. A. Cook, Eds., Illuminating the Path: The Research and Development Agenda for Visual Analytics. Richland, WA, USA: National Visualization and Analytics Ctr, 2005.

[4]

D. A. Keim, J. Kohlhammer, G. P. Ellis, and F. Mansmann, Eds., Mastering the Information Age – Solving Problems with Visual Analytics. Goslar, Germany: Eurographics Association, 2010.

[5]

C. B. Weinstock and J. B. Goodenough, “On system scalability,” Softw. Eng. Inst., Carnegie Mellon Univ., Pittsburgh, PA, USA, Tech. Rep. CMU/SEI-2006-TN-012, 2006.

[6]

A. B. Bondi, “Characteristics of scalability and their impact on performance,” in Proc. Workshop. Softw. Perform., 2000, pp. 195–203.

[7]

M. M. Michael, J. E. Moreira, D. Shiloach, and R. W. Wisniewski, “Scale-up x scale-out: A case study using Nutch/Lucene,” in Proc. IEEE Int. Parallel Distrib. Process. Symp., 2007, pp. 1–8.

[8]

M. D. Hill, “What is scalability?,” ACM SIGARCH Comput. Archit. News, vol. 18, no. 4, pp. 18–21, 1990.

Digital Library

[9]

L. Duboc, D. S. Rosenblum, and T. Wicks, “A framework for modelling and analysis of software systems scalability,” in Proc. IEEE/ACM Int. Conf. Softw. Eng., 2006, pp. 949–952.

[10]

G. G. Robertson, D. S. Ebert, S. G. Eick, D. A. Keim, and K. Joy, “Scale and complexity in visual analytics,” Inf. Vis., vol. 8, no. 4, pp. 247–253, 2009.

Digital Library

[11]

B. Yost and C. North, “The perceptual scalability of visualization,” IEEE Trans. Vis. Comput. Graph., vol. 12, no. 5, pp. 837–844, Sep./Oct. 2006.

Digital Library

[12]

S. G. Eick and A. F. Karr, “Visual scalability,” J. Comput. Graphical Statist., vol. 11, no. 1, pp. 22–43, 2002.

[13]

H. Lam, E. Bertini, P. Isenberg, C. Plaisant, and S. Carpendale, “Empirical studies in information visualization: Seven scenarios,” IEEE Trans. Vis. Comput. Graph., vol. 18, no. 9, pp. 1520–1536, Sep. 2012.

Digital Library

[14]

S. K. Card, T. P. Moran, and A. Newell, The Psychology of Human-Computer Interaction. Mahwah, NJ, USA: Erlbaum, 1983.

Digital Library

[15]

D. A. Szafir, S. Haroz, M. Gleicher, and S. Franconeri, “Four types of ensemble coding in data visualizations,” J. Vis., vol. 16, no. 5, pp. 11–11, 2016.

[16]

A. Treisman, “Preattentive processing in vision,” Comput. Vis. Graph. Image Process., vol. 31, no. 2, pp. 156–177, 1985.

Digital Library

[17]

C. G. Healey and J. T. Enns, “Large datasets at a glance: Combining textures and colors in scientific visualization,” IEEE Trans. Vis. Comput. Graph., vol. 5, no. 2, pp. 145–167, Second Quarter 1999.

Digital Library

[18]

P. M. Fitts, “The information capacity of the human motor system in controlling the amplitude of movement,” J. Exp. Psychol., vol. 47, no. 6, pp. 381–391, 1954.

[19]

Y. Guiard, F. Bourgeois, D. Mottet, and M. Beaudouin-Lafon, “Beyond the 10-bit barrier: Fitts’ law in multi-scale electronic worlds,” in People and Computers XV—Interaction Without Frontiers. London, UK: Springer, 2001, pp. 573–587.

[20]

W. E. Hick, “On the rate of gain of information,” Quart. J. Exp. Psychol., vol. 4, no. 1, pp. 11–26, 1952.

[21]

A. Cockburn, C. Gutwin, and S. Greenberg, “A predictive model of menu performance,” in Proc. SIGCHI Conf. Hum. Factors Comput. Syst., 2007, pp. 627–636.

[22]

R. Budiu, “Scaling user interfaces: An information-processing approach to multi-device design,” 2014. Accessed: Mar. 29, 2021. [Online]. Available: https://www.nngroup.com/articles/scaling-user-interfaces/

[23]

B. Brown, S. Bødker, and K. Höök, “Does HCI scale? Scale hacking and the relevance of HCI,” Interactions, vol. 24, no. 5, pp. 28–33, 2017.

Digital Library

[24]

J. R. Ruthruff, S. G. Elbaum, and G. Rothermel, “Experimental program analysis: A new program analysis paradigm,” in Proc. Int. Symp. Softw. Testing Anal., 2006, pp. 49–60.

[25]

A. Perrot, R. Bourqui, N. Hanusse, F. Lalanne, and D. Auber, “Large interactive visualization of density functions on big data infrastructure,” in Proc. IEEE Symp. Large Data Anal. Vis., 2015, pp. 99–106.

[26]

H. Chen, R. Sukthankar, G. Wallace, and K. Li, “Scalable alignment of large-format multi-projector displays using camera homography trees,” in Proc. IEEE Vis. Conf., 2002, pp. 339–346.

[27]

M. R. Jakobsen and K. Hornbæk, “Sizing up visualizations: Effects of display size in focus+context, overview+detail, and zooming interfaces,” in Proc. SIGCHI Conf. Hum. Factors Comput. Syst., 2011, pp. 1451–1460.

[28]

M. Khoury, Y. Hu, S. Krishnan, and C. E. Scheidegger, “Drawing large graphs by low-rank stress majorization,” Comput. Graph. Forum, vol. 31, no. 3, pp. 975–984, 2012.

Digital Library

[29]

M. Falk and D. Weiskopf, “Output-sensitive 3D line integral convolution,” IEEE Trans. Vis. Comput. Graph., vol. 14, no. 4, pp. 820–834, Jul./Aug. 2008.

Digital Library

[30]

J. L. Gustafson, “Reevaluating Amdahl's law,” Commun. ACM, vol. 31, no. 5, pp. 532–533, 1988.

Digital Library

[31]

M. Howison, E. W. Bethel, and H. Childs, “Hybrid parallelism for volume rendering on large-, multi-, and many-core systems,” IEEE Trans. Vis. Comput. Graph., vol. 18, no. 1, pp. 17–29, Jan. 2012.

Digital Library

[32]

Y. Wang et al., “Structure-aware fisheye views for efficient large graph exploration,” IEEE Trans. Vis. Comput. Graph., vol. 25, no. 1, pp. 566–575, Jan. 2019.

Digital Library

[33]

M. Ghoniem, J.-D. Fekete, and P. Castagliola, “A comparison of the readability of graphs using node-link and matrix-based representations,” in Proc. IEEE Symp. Inf. Vis., 2004, pp. 17–24.

[34]

Y. Glémarec et al., “A scalability benchmark for a virtual audience perception model in virtual reality,” in Proc. ACM Symp. Virtual Reality Softw. Technol., 2019, Art. no.

[35]

J. Deng, O. Russakovsky, J. Krause, M. S. Bernstein, A. Berg, and L. Fei-Fei, “Scalable multi-label annotation,” in Proc. SIGCHI Conf. Hum. Factors Comput. Syst., 2014, pp. 3099–3102.

[36]

L. D. Lins, J. T. Klosowski, and C. E. Scheidegger, “Nanocubes for real-time exploration of spatiotemporal datasets,” IEEE Trans. Vis. Comput. Graph., vol. 19, no. 12, pp. 2456–2465, Dec. 2013.

Digital Library

[37]

D. Moritz, B. Howe, and J. Heer, “Falcon: Balancing interactive latency and resolution sensitivity for scalable linked visualizations,” in Proc. SIGCHI Conf. Hum. Factors Comput. Syst., 2019, Art. no.

[38]

J. Abello, F. van Ham, and N. Krishnan, “ASK-GraphView: A large scale graph visualization system,” IEEE Trans. Vis. Comput. Graph., vol. 12, no. 5, pp. 669–676, Sep./Oct. 2006.

Digital Library

[39]

C. A. L. Pahins, S. A. Stephens, C. Scheidegger, and J. L. D. Comba, “Hashedcubes: Simple, low memory, real-time visual exploration of big data,” IEEE Trans. Vis. Comput. Graph., vol. 23, no. 1, pp. 671–680, Jan. 2017.

Digital Library

[40]

K. Charmaz, Constructing Grounded Theory. London, UK: Sage, 2014.

[41]

T. Isenberg, P. Isenberg, J. Chen, M. Sedlmair, and T. Möller, “A systematic review on the practice of evaluating visualization,” IEEE Trans. Vis. Comput. Graph., vol. 19, no. 12, pp. 2818–2827, Dec. 2013.

Digital Library

[42]

D. Sacha et al., “Visual interaction with dimensionality reduction: A structured literature analysis,” IEEE Trans. Vis. Comput. Graph., vol. 23, no. 1, pp. 241–250, Jan. 2017.

Digital Library

[43]

M. Sedlmair, C. Heinzl, S. Bruckner, H. Piringer, and T. Möller, “Visual parameter space analysis: A conceptual framework,” IEEE Trans. Vis. Comput. Graph., vol. 20, no. 12, pp. 2161–2170, Dec. 2014.

[44]

P. Isenberg et al., “vispubdata.org: A metadata collection about IEEE Visualization (VIS) publications,” IEEE Trans. Vis. Comput. Graph., vol. 23, no. 9, pp. 2199–2206, Sep. 2017.

Digital Library

[45]

E. M. Bennett, R. Alpert, and A. C. Goldstein, “Communications through limited-response questioning,” Public Opin. Quart., vol. 18, no. 3, pp. 303–308, 1954.

[46]

L. McInnes and J. Healy, “UMAP: Uniform manifold approximation and projection for dimension reduction,” 2018,.

[47]

T. Munzner, “A nested process model for visualization design and validation,” IEEE Trans. Vis. Comput. Graph., vol. 15, no. 6, pp. 921–928, Nov./Dec. 2009.

Digital Library

[48]

J. Jo, F. Vernier, P. Dragicevic, and J.-D. Fekete, “A declarative rendering model for multiclass density maps,” IEEE Trans. Vis. Comput. Graph., vol. 25, no. 1, pp. 470–480, Jan. 2019.

Digital Library

[49]

A. Lex, N. Gehlenborg, H. Strobelt, R. Vuillemot, and H. Pfister, “UpSet: Visualization of intersecting sets,” IEEE Trans. Vis. Comput. Graph., vol. 20, no. 12, pp. 1983–1992, Dec. 2014.

[50]

J. Tierny, A. Gyulassy, E. Simon, and V. Pascucci, “Loop surgery for volumetric meshes: Reeb graphs reduced to contour trees,” IEEE Trans. Vis. Comput. Graph., vol. 15, no. 6, pp. 1177–1184, Nov./Dec. 2009.

Digital Library

[51]

M. Hadwiger, J. Beyer, W. Jeong, and H. Pfister, “Interactive volume exploration of petascale microscopy data streams using a visualization-driven virtual memory approach,” IEEE Trans. Vis. Comput. Graph., vol. 18, no. 12, pp. 2285–2294, Dec. 2012.

Digital Library

[52]

J. Beyer, A. K. Al-Awami, N. Kasthuri, J. W. Lichtman, H. Pfister, and M. Hadwiger, “ConnectomeExplorer: Query-guided visual analysis of large volumetric neuroscience data,” IEEE Trans. Vis. Comput. Graph., vol. 19, no. 12, pp. 2868–2877, Dec. 2013.

Digital Library

[53]

C. Y. Ip and A. Varshney, “Saliency-assisted navigation of very large landscape images,” IEEE Trans. Vis. Comput. Graph., vol. 17, no. 12, pp. 1737–1746, Dec. 2011.

Digital Library

[54]

K. Kurzhals, M. Hlawatsch, F. Heimerl, M. Burch, T. Ertl, and D. Weiskopf, “Gaze stripes: Image-based visualization of eye tracking data,” IEEE Trans. Vis. Comput. Graph., vol. 22, no. 1, pp. 1005–1014, Jan. 2016.

Digital Library

[55]

W. Dou, X. Wang, R. Chang, and W. Ribarsky, “ParallelTopics: A probabilistic approach to exploring document collections,” in Proc. IEEE Conf. Vis. Analytics Sci. Technol., 2011, pp. 231–240.

[56]

C. Papadopoulos, I. Gutenko, and A. E. Kaufman, “VEEVVIE: Visual explorer for empirical visualization, VR and interaction experiments,” IEEE Trans. Vis. Comput. Graph., vol. 22, no. 1, pp. 111–120, Jan. 2016.

Digital Library

[57]

J. Georgii and R. Westermann, “A generic and scalable pipeline for GPU tetrahedral grid rendering,” IEEE Trans. Vis. Comput. Graph., vol. 12, no. 5, pp. 1345–1352, Sep./Oct. 2006.

Digital Library

[58]

W. Qiao, M. McLennan, R. Kennell, D. S. Ebert, and G. Klimeck, “Hub-based simulation and graphics hardware accelerated visualization for nanotechnology applications,” IEEE Trans. Vis. Comput. Graph., vol. 12, no. 5, pp. 1061–1068, Sep./Oct. 2006.

Digital Library

[59]

H. Chen, R. Sukthankar, G. Wallace, and K. Li, “Scalable alignment of large-format multi-projector displays using camera homography trees,” in Proc. IEEE Vis. Conf., 2002, pp. 339–346.

[60]

E. T. Brown, J. Liu, C. E. Brodley, and R. Chang, “Dis-function: Learning distance functions interactively,” in Proc. IEEE Conf. Vis. Analytics Sci. Technol., 2012, pp. 83–92.

[61]

J. Jo, J. Huh, J. Park, B. H. Kim, and J. Seo, “LiveGantt: Interactively visualizing a large manufacturing schedule,” IEEE Trans. Vis. Comput. Graph., vol. 20, no. 12, pp. 2329–2338, Dec. 2014.

[62]

J. Heer and M. Bostock, “Declarative language design for interactive visualization,” IEEE Trans. Vis. Comput. Graph., vol. 16, no. 6, pp. 1149–1156, Nov./Dec. 2010.

Digital Library

[63]

V. Yoghourdjian, Y. Yang, T. Dwyer, L. Lawrence, M. Wybrow, and K. Marriott, “Scalability of network visualisation from a cognitive load perspective,” IEEE Trans. Vis. Comput. Graph., vol. 27, no. 2, pp. 1677–1687, Feb. 2021.

[64]

H. Strobelt et al., “Vials: Visualizing alternative splicing of genes,” IEEE Trans. Vis. Comput. Graph., vol. 22, no. 1, pp. 399–408, Jan. 2016.

Digital Library

[65]

N. Pezzotti et al., “GPGPU linear complexity t-SNE optimization,” IEEE Trans. Vis. Comput. Graph., vol. 26, no. 1, pp. 1172–1181, Jan. 2020.

[66]

R. Veras and C. Collins, “Discriminability tests for visualization effectiveness and scalability,” IEEE Trans. Vis. Comput. Graph., vol. 26, no. 1, pp. 749–758, Jan. 2020.

[67]

W. Aigner, S. Hoffmann, and A. Rind, “EvalBench: A software library for visualization evaluation,” Comput. Graph. Forum, vol. 32, no. 3, pp. 41–50, 2013.

[68]

A. Eiselmayer, C. Wacharamanotham, M. Beaudouin-Lafon, and W. E. Mackay, “Touchstone2: An interactive environment for exploring trade-offs in HCI experiment design,” in Proc. SIGCHI Conf. Hum. Factors Comput. Syst., 2019, pp. 1–11.

[69]

C. Schulz et al., “Generative data models for validation and evaluation of visualization techniques,” in Proc. 6th Workshop Beyond Time Errors Novel Eval. Methods Vis., 2016, pp. 112–124.

[70]

F. Schreiber and D. Weiskopf, “Quantitative visual computing,” Inf. Technol., vol. 64, no. 4/5, pp. 119–120, 2022.

[71]

J.-D. Fekete and J. Freire, “Exploring reproducibility in visualization,” IEEE Comput. Graph. Appl., vol. 40, no. 5, pp. 108–119, Sep./Oct. 2020.

Digital Library

[72]

C. Plaisant, J.-D. Fekete, and G. Grinstein, “Promoting insight-based evaluation of visualizations: From contest to benchmark repository,” IEEE Trans. Vis. Comput. Graph., vol. 14, no. 1, pp. 120–134, Jan./Feb. 2008.

Digital Library

[73]

L. Battle et al., “Database benchmarking for supporting real-time interactive querying of large data,” in Proc. ACM SIGMOD Int. Conf. Manage. Data, 2020, pp. 1571–1587.

[74]

K.-L. Ma, “In situ visualization at extreme scale: Challenges and opportunities,” IEEE Comput. Graph. Appl., vol. 29, no. 6, pp. 14–19, Nov./Dec. 2009.

[75]

J.-D. Fekete, D. Fisher, A. Nandi, and M. Sedlmair, “Progressive data analysis and visualization (Dagstuhl seminar 18411),” Dagstuhl Rep., vol. 8, no. 10, pp. 1–40, 2019.

[76]

L. Wilkinson and G. Wills, “Scagnostics distributions,” J. Comput. Graphical Statist., vol. 17, no. 2, pp. 473–491, 2008.

[77]

J. Slack, K. Hildebrand, and T. Munzner, “PRISAD: A partitioned rendering infrastructure for scalable accordion drawing,” in Proc. IEEE Symp. Inf. Vis., 2005, pp. 41–48.

Cited By

Klötzl DKrake THeyen FBecher MKoch MWeiskopf DKurzhals K(2024)NMF-Based Analysis of Mobile Eye-Tracking DataProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3653518(1-9)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3653518
Angelini MBlasilli GLenti SSantucci G(2024)A Visual Analytics Conceptual Framework for Explorable and Steerable Partial Dependence AnalysisIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.326373930:8(4497-4513)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3263739
Geringer SGeiselhart FBäuerle ADec DOdenthal OReina GRopinski TWeiskopf D(2024)mint: Integrating scientific visualizations into virtual realityJournal of Visualization10.1007/s12650-024-01011-y27:6(1143-1169)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s12650-024-01011-y

Recommendations

Performance, Interactivity, and Collaboration: Scalability Improvements to the Data Visualization Ecosystem
The Perceptual Scalability of Visualization

Larger, higher resolution displays can be used to increase the scalability of information visualizations. But just how much can scalability increase using larger displays before hitting human perceptual or cognitive limits? Are the same visualization ...
Dynamic switching of data visualization method for increased plotting scalability
NordiCHI '14: Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational

In information visualization every method performs best in its own range of plotting density. When the available area for the visualization is unknown or when the data is supplied in packages of various sizes, the best suited method cannot be chosen in ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Visualization and Computer Graphics

IEEE Transactions on Visualization and Computer Graphics Volume 30, Issue 7

July 2024

1367 pages

Issue’s Table of Contents

© 2022 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.

Publisher

IEEE Educational Activities Department

United States

Publication History

Published: 29 December 2022

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Klötzl DKrake THeyen FBecher MKoch MWeiskopf DKurzhals K(2024)NMF-Based Analysis of Mobile Eye-Tracking DataProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3653518(1-9)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3653518
Angelini MBlasilli GLenti SSantucci G(2024)A Visual Analytics Conceptual Framework for Explorable and Steerable Partial Dependence AnalysisIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.326373930:8(4497-4513)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3263739
Geringer SGeiselhart FBäuerle ADec DOdenthal OReina GRopinski TWeiskopf D(2024)mint: Integrating scientific visualizations into virtual realityJournal of Visualization10.1007/s12650-024-01011-y27:6(1143-1169)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s12650-024-01011-y

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents