research-article

The Design and Evaluation of Interfaces for Navigating Gigapixel Images in Digital Pathology

Authors:

Rhys G. Thomas,

Rebecca Randell,

Darren TreanorAuthors Info & Claims

ACM Transactions on Computer-Human Interaction (TOCHI), Volume 23, Issue 1

Article No.: 5, Pages 1 - 29

https://doi.org/10.1145/2834117

Published: 13 January 2016 Publication History

Abstract

This article describes the design and evaluation of two generations of an interface for navigating datasets of gigapixel images that pathologists use to diagnose cancer. The interface design is innovative because users panned with an overview:detail view scale difference that was up to 57 times larger than established guidelines, and 1 million pixel “thumbnail” overviews that leveraged the real estate of high-resolution workstation displays. The research involved experts performing real work (pathologists diagnosing cancer), using datasets that were up to 3,150 times larger than those used in previous studies that involved navigating images. The evaluation provides evidence about the effectiveness of the interfaces and characterizes how experts navigate gigapixel images when performing real work. Similar interfaces could be adopted in applications that use other types of high-resolution images (e.g., remote sensing or high-throughput microscopy).

References

[1]

S. Al-Janabi, A. Huisman, and P. J. Van Diest. 2012. Digital pathology: Current status and future perspectives. Histopathology 61, 1--9.

[2]

C. Anslow, S. Marshall, J. Noble, E. Tempero, and R. Biddle. 2010. User evaluation of polymetric views using a large visualization wall. In Proceedings of the 5th International Symposium on Software Visualization. ACM, New York, 25--34.

Digital Library

[3]

C. Appert, O. Chapuis, and E. Pietriga. 2010. High-precision magnification lenses. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 273--282.

Digital Library

[4]

A. Aretz. 1991. The design of electronic map displays. Human Factors 33, 85--101.

Digital Library

[5]

R. Ball, C. North, and D. A. Bowman. 2007. Move to improve: Promoting physical navigation to increase user performance with large displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 191--200.

Digital Library

[6]

P. Baudisch, N. Good, V. Bellotti, and P. Schraedley. 2002. Keeping things in context: A comparative evaluation of focus plus context screens, overviews, and zooming. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 259--266.

Digital Library

[7]

P. Baudisch, B. Lee, and L. Hanna. 2004. Fishnet, a fisheye web browser with search term popouts: A comparative evaluation with overview and linear view. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI’04). ACM, New York, 133--140.

Digital Library

[8]

D. V. Beard and J. Q. Walker. 1990. Navigational techniques to improve the display of large two-dimensional spaces. Behaviour & Information Technology 9, 451--466.

[9]

M. Beaudouin-Lafon, S. Huot, H. Olafsdottir, and P. Dragicevic. 2014. GlideCursor: Pointing with an inertial cursor. In Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces (AVI’14). ACM, New York, 49--56.

Digital Library

[10]

B. Bederson. 2000. Fisheye menus. In Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology. ACM, New York, 217--225.

Digital Library

[11]

S. Burigat and L. Chittaro. 2011. Visualizing references to off-screen content on mobile devices: A comparison of Arrows, Wedge, and Overview+Detail. Interacting with Computers 23, 156--166.

Digital Library

[12]

S. K. Card, G. G. Robertson, and J. D. Mackinlay. 1991. The information visualizer, an information workspace. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 181--186.

Digital Library

[13]

S. Carpendale, J. Ligh, and E. Pattison. 2004. Achieving higher magnification in context. In Proceedings of the 17th annual ACM Symposium on User Interface Software and Technology (UIST’04). ACM, New York, 71--80.

Digital Library

[14]

G. Casiez and N. Roussel. 2011. No more bricolage&excl;: Methods and tools to characterize, replicate and compare pointing transfer functions. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST’11). ACM, New York, 603--614.

Digital Library

[15]

G. Casiez, D. Vogel, R. Balakrishnan, and A. Cockburn. 2008. The impact of control-display gain on user performance in pointing tasks. Human-Computer Interaction 23, 215--250.

[16]

A. Cockburn, A. Karlson, and B. B. Bederson. 2008. A review of overview+detail, zooming, and focus+context interfaces. ACM Computing Surveys 41, 1, 31 pages.

Digital Library

[17]

A. Cockburn and J. Savage. 2004. Comparing speed-dependent automatic zooming with traditional scroll, pan and zoom methods. In People and Computers XVII — Designing for Society. Springer, 87--102.

[18]

R. Datta, D. Joshi, J. Li, and J. Z. Wang. 2008. Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys 40, 2, 60 pages.

Digital Library

[19]

W. Gray and W. Fu. 2001. Ignoring perfect knowledge in-the-world for imperfect knowledge in-the-head: Implications of rational analysis for interface design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 112--119.

Digital Library

[20]

Y. Guiard and M. Beaudouin-Lafon. 2004. Target acquisition in multiscale electronic worlds. International Journal of Human-Computer Studies 61, 875--905.

Digital Library

[21]

Y. Guiard, M. Beaudouin-Lafon, J. Bastin, D. Pasveer, and S. Zhai. 2004. View size and pointing difficulty in multi-scale navigation. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI’04). ACM, New York, 117--124.

Digital Library

[22]

N. A. Hamilton, J. T. Wang, M. C. Kerr, and R. D. Teasdale. 2009. Statistical and visual differentiation of subcellular imaging. BMC Bioinformatics 10, 94.

[23]

M. J. Henry, S. Hampton, A. Endert, I. Roberts, and D. Payne. 2013. MultiFacet: A faceted interface for browsing large multimedia collections. In IEEE International Symposium on Multimedia (ISM’13). IEEE, Los Alamitos, CA, 347--350.

Digital Library

[24]

E. Hooper and B. Coury. 1994. Graphical displays for orientation information. Human Factors 36, 62--78.

[25]

K. Hornbæk, B. B. Bederson, and C. Plaisant. 2002. Navigation patterns and usability of zoomable user interfaces with and without an overview. ACM Transactions on Computer-Human Interaction 9, 362--389.

Digital Library

[26]

K. Hornbæk and E. Frøkjær. 2003. Reading patterns and usability in visualizations of electronic documents. ACM Transactions on Computer-Human Interaction 10, 119--149.

Digital Library

[27]

M. R. Jakobsen and K. Hornbæk. 2011. Sizing up visualizations: Effects of display size in focus+context, overview+detail, and zooming interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 1451--1460.

Digital Library

[28]

W. Javed, S. Ghani, and N. Elmqvist. 2012. Polyzoom: multiscale and multifocus exploration in 2d visual spaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 287--296.

Digital Library

[29]

M. Levine, I. Marchon, and G. Hanley. 1984. The placement and misplacement of you-are-here maps. Environment and Behavior 16, 139--157.

[30]

J. Moore, C. Allan, J.-M. Burel, B. Loranger, D. Macdonald, J. Monk, and J. R. Swedlow. 2008. Open tools for storage and management of quantitative image data. Methods in Cell Biology 85, 555--570.

[31]

M. Nancel, J. Wagner, E. Pietriga, O. Chapuis, and W. Mackay. 2011. Mid-air pan-and-zoom on wall-sized displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 177--186.

Digital Library

[32]

D. Nekrasovski, A. Bodnar, J. Mcgrenere, F. Guimbretière, and T. Munzner. 2006. An evaluation of pan & zoom and rubber sheet navigation with and without an overview. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 11--20.

Digital Library

[33]

K. Perlin and D. Fox. 1993. Pad: An alternative approach to the computer interface. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques. ACM, New York, 57--64.

Digital Library

[34]

E. Pietriga, C. Appert, and M. Beaudouin-Lafon. 2007. Pointing and beyond: An operationalization and preliminary evaluation of multi-scale searching. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 1215--1224.

Digital Library

[35]

P. Pirolli, S. K. Card, and M. M. Van Der Wege. 2003. The effects of information scent on visual search in the hyperbolic tree browser. ACM Transactions on Computer-Human Interaction 10, 20--53.

Digital Library

[36]

C. Plaisant, D. Carr, and B. Shneiderman. 1995. Image-browser taxonomy and guidelines for designers. IEEE Software 12, 21--32.

Digital Library

[37]

A. Pretorius, Y. Zhou, and R. Ruddle. 2015. Visual parameter optimisation for biomedical image processing. BMC bioinformatics 16 (Suppl. 11), S9.

[38]

A. J. Pretorius, M. P. Bray, A. E. Carpenter, and R. A. Ruddle. 2011. Visualization of parameter space for image analysis. IEEE Transactions on Visualization and Computer Graphics 17, 2402--2411.

Digital Library

[39]

R. Randell, G. Hutchins, J. Sandars, T. Ambepitiya, D. Treanor, R. Thomas, and R. A. Ruddle. 2012. Using a high-resolution wall-sized virtual microscope to teach undergraduate medical students. In CHI’12 Extended Abstracts on Human Factors in Computing Systems. ACM, New York, 2435--2440.

Digital Library

[40]

R. Randell, R. A. Ruddle, C. Mello-Thoms, R. G. Thomas, P. Quirke, and D. Treanor. 2013. Virtual reality microscope versus conventional microscope on time to diagnosis: An experimental study. Histopathology 62, 351--358.

[41]

R. Randell, R. A. Ruddle, P. Quirke, R. G. Thomas, and D. Treanor. 2012. Working at the microscope: Analysis of the activities involved in diagnostic pathology. Histopathology 60, 504--510.

[42]

R. Randell, R. A. Ruddle, R. G. Thomas, C. Mello-Thoms, and D. Treanor. 2014. Diagnosis of major cancer resection specimens with virtual slides: Impact of a novel digital pathology workstation. Human Pathology 45, 2101--2106.

[43]

C. Rooney and R. A. Ruddle. 2012. Improving window manipulation and content interaction on high resolution, wall-sized displays. International Journal of Human-Computer Interaction 28, 423--432.

[44]

R. A. Ruddle, W. Fateen, D. Treanor, P. Sondergeld, and P. Quirke. 2013. Leveraging wall-sized high-resolution displays for comparative genomics analyses of copy number variation. In Proceedings of the 3rd IEEE Symposium on Biological Data Visualization. IEEE, Los Alamitos, CA, 89--96.

[45]

R. A. Ruddle and D. M. Jones. 2001. Movement in cluttered virtual environments. Presence: Teleoperators and Virtual Environments 10, 511--524.

Digital Library

[46]

R. A. Ruddle, S. J. Payne, and D. M. Jones. 1999. The effects of maps on navigation and search strategies in very-large-scale virtual environments. Journal of Experimental Psychology: Applied 5, 54--75.

[47]

R. A. Ruddle, R. G. Thomas, R. S. Randell, P. Quirke, and D. Treanor. 2015. Performance and interaction behaviour during visual search on large, high-resolution displays. Information Visualization 14, 137--147.

[48]

B. Shneiderman. 1996. The eyes have it: A task by data type taxonomy for information visualizations. In IEEE Symposium on Visual Languages. IEEE, Los Alamitos, CA, 336--343.

Digital Library

[49]

B. Shneiderman. 1998. Designing the User Interface: Strategies for Effective Human-computer Interaction. Addison-Wesley, Reading, MA.

Digital Library

[50]

L. Shupp, C. Andrews, M. Dickey-Kurdziolek, B. Yost, and C. North. 2009. Shaping the display of the future: The effects of display size and curvature. Human-Computer Interaction 24, 230--272.

[51]

A. Skopik and C. Gutwin. 2005. Improving revisitation in fisheye views with visit wear. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 771--780.

Digital Library

[52]

C. Tominski, S. Gladisch, U. Kister, R. Dachselt, and H. Schumann. 2014. A survey on interactive lenses in visualization. EuroVis State-of-the-Art Reports. Eurographics Association, Aire-la-Ville, Switzerland, 43--62.

[53]

D. Treanor, N. Jordan Owers, J. Hodrien, P. Quirke, and R. A. Ruddle. 2009. Virtual reality Powerwall versus conventional microscope for viewing pathology slides: An experimental comparison. Histopathology 5, 294--300.

[54]

D. Treanor and P. Quirke. 2007. The virtual slide and conventional microscope -- a direct comparison of their diagnostic efficiency. In Proceedings of the Annual Meeting of the Pathological Society of Great Britain and Ireland. Pathological Society of Great Britain and Ireland, Glasgow.

[55]

J. J. Van Wijk and W. A. Nuij. 2003. Smooth and efficient zooming and panning. In IEEE Symposium on Information Visualization. IEEE, Los Alamitos, CA, 15--23.

Digital Library

[56]

N. Velez, D. Jukic, and J. Ho. 2008. Evaluation of 2 whole-slide imaging applications in dermatopathology. Human Pathology 39, 1341--1349.

[57]

T. Walter, D. W. Shattuck, R. Baldock, M. E. Bastin, A. E. Carpenter, S. Duce, J. Ellenberg, A. Fraser, N. Hamilton, and S. Pieper. 2010. Visualization of image data from cells to organisms. Nature Methods 7, S26--S41.

[58]

C. Ware and M. Lewis. 1995. The DragMag image magnifier. In Conference Companion on Human Factors in Computing Systems. ACM, New York, 407--408.

Digital Library

[59]

C. Wickens and T. Prevett. 1995. Exploring the dimensions of egocentricity in aircraft navigation displays. Journal of Experimental Psychology: Applied 1, 110--135.

[60]

B. Yost, Y. Haciahmetoglu, and C. North. 2007. Beyond visual acuity: The perceptual scalability of information visualizations for large displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 101--110.

Digital Library

[61]

A. Zanella, M. S. T. Carpendale, and M. Rounding. 2002. On the effects of viewing cues in comprehending distortions. In Proceedings of the Second Nordic Conference on Human-computer Interaction. ACM, New York, 119--128.

Digital Library

[62]

S. Zhai and P. Milgram. 1998. Quantifying coordination in multiple DOF movement and its application to evaluating 6 DOF input devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 320--327.

Digital Library

Cited By

Yang CHaeri MMagaki SZarrin-Khameh NGu HChen X(2024)A Human-AI Collaborative System to Support Mitosis Assessment in PathologyCompanion Proceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640544.3645233(58-61)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640544.3645233
Jennings CTreanor DBrettle D(2024)Pathologists light level preferences using the microscope—study to guide digital pathology display useJournal of Pathology Informatics10.1016/j.jpi.2024.10037915(100379)Online publication date: Dec-2024
https://doi.org/10.1016/j.jpi.2024.100379
Gu HYang CHaeri MWang JTang SYan WHe SWilliams CMagaki SChen X(2023)Augmenting Pathologists with NaviPath: Design and Evaluation of a Human-AI Collaborative Navigation SystemProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580694(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580694
Show More Cited By

Index Terms

The Design and Evaluation of Interfaces for Navigating Gigapixel Images in Digital Pathology
1. Human-centered computing

Recommendations

Annotating gigapixel images
UIST '08: Proceedings of the 21st annual ACM symposium on User interface software and technology

Panning and zooming interfaces for exploring very large images containing billions of pixels (gigapixel images) have recently appeared on the internet. This paper addresses issues that arise when creating and rendering auditory and textual annotations ...
An ensemble of deep CNNs for automatic grading of breast cancer in digital pathology images
Abstract
Histopathological diagnosis is the mainstay of present-day preventive medical care service to guide the therapy and treatment of breast cancer at an early stage. Manual examination of histologic data based on clinicians’ subjective knowledge is a ...
Centroid-Aware Feature Recalibration for Cancer Grading in Pathology Images
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023
Abstract
Cancer grading is an essential task in pathology. The recent developments of artificial neural networks in computational pathology have shown that these methods hold great potential for improving the accuracy and quality of cancer diagnosis. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer-Human Interaction

ACM Transactions on Computer-Human Interaction Volume 23, Issue 1

February 2016

147 pages

ISSN:1073-0516

EISSN:1557-7325

DOI:10.1145/2872314

Editor:
Ken Hinckley
Microsoft Research

Issue’s Table of Contents

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 January 2016

Accepted: 01 October 2015

Revised: 01 August 2015

Received: 01 July 2014

Published in TOCHI Volume 23, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Best Paper

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
976
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)2

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yang CHaeri MMagaki SZarrin-Khameh NGu HChen X(2024)A Human-AI Collaborative System to Support Mitosis Assessment in PathologyCompanion Proceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640544.3645233(58-61)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640544.3645233
Jennings CTreanor DBrettle D(2024)Pathologists light level preferences using the microscope—study to guide digital pathology display useJournal of Pathology Informatics10.1016/j.jpi.2024.10037915(100379)Online publication date: Dec-2024
https://doi.org/10.1016/j.jpi.2024.100379
Gu HYang CHaeri MWang JTang SYan WHe SWilliams CMagaki SChen X(2023)Augmenting Pathologists with NaviPath: Design and Evaluation of a Human-AI Collaborative Navigation SystemProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580694(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580694
Gruget MTouya GMuehlenhaus I(2023)Missing the city for buildings? A critical review of pan-scalar map generalization and design in contemporary zoomable mapsInternational Journal of Cartography10.1080/23729333.2022.21534679:2(255-285)Online publication date: 10-Jan-2023
https://doi.org/10.1080/23729333.2022.2153467
Li FChen CSourina OXu GHan S(2023)A Hierarchical Task Network-Based Approach to Generating Generic Tasks for Interface Usability EvaluationMan-Machine-Environment System Engineering10.1007/978-981-99-4882-6_60(435-442)Online publication date: 5-Sep-2023
https://doi.org/10.1007/978-981-99-4882-6_60
Gu HLiang YXu YWilliams CMagaki SKhanlou NVinters HChen ZNi SYang CYan WZhang XLi YHaeri MChen X(2022)Improving Workflow Integration with xPath: Design and Evaluation of a Human-AI Diagnosis System in PathologyACM Transactions on Computer-Human Interaction10.1145/357701130:2(1-37)Online publication date: 16-Dec-2022
https://dl.acm.org/doi/10.1145/3577011
Clarke EDoherty DRandell RGrek JThomas RRuddle RTreanor D(2022)Faster than light (microscopy): superiority of digital pathology over microscopy for assessment of immunohistochemistryJournal of Clinical Pathology10.1136/jclinpath-2021-20796176:5(333-338)Online publication date: 17-Jan-2022
https://doi.org/10.1136/jclinpath-2021-207961
Calisto FSantiago CNunes NNascimento J(2021)Introduction of human-centric AI assistant to aid radiologists for multimodal breast image classificationInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2021.102607150(102607)Online publication date: Jun-2021
https://doi.org/10.1016/j.ijhcs.2021.102607
Lebre RJesus RNunes PCosta C(2021)Pathobox: The Collaborative Tele-pathology Platform with Access ManagementBiomedical Engineering Systems and Technologies10.1007/978-3-030-72379-8_20(407-424)Online publication date: 30-Mar-2021
https://doi.org/10.1007/978-3-030-72379-8_20
Falk MYnnerman ATreanor DLundström C(2018)Interactive Visualization of 3D Histopathology in Native ResolutionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2018.286481625:1(1008-1017)Online publication date: 7-Dec-2018
https://dl.acm.org/doi/10.1109/TVCG.2018.2864816
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents