article

Evaluating the educational impact of visualization

Authors:

Stephen Cooper,

Boris Koldehofe,

Guido Rößling,

Jarmo Rantakokko,

Rockford J. Ross,

Rudolf Fleischer,

Marja Kuittinen,

Myles McNallyAuthors Info & Claims

ACM SIGCSE Bulletin, Volume 35, Issue 4

Pages 124 - 136

https://doi.org/10.1145/960492.960540

Published: 30 June 2003 Publication History

Abstract

The educational impact of visualization depends not only on how well students learn when they use it, but also on how widely it is used by instructors. Instructors believe that visualization helps students learn. The integration of visualization techniques in classroom instruction, however, has fallen far short of its potential. This paper considers this disconnect, identifying its cause in a failure to understand the needs of a key member in the hierarchy of stakeholders, namely the instructor. We describe these needs and offer guidelines for both the effective deployment of visualizations and the evaluation of instructor satisfaction. We then consider different forms of evaluation and the impact of student learning styles on learner outcomes.

References

[1]

ACM Special Interest Group on Computer Science Education. SIGCSE Education Links. Available online at http://www.sigcse.org/topics, 2003.

[2]

Akingbade, A., Finley, T., Jackson, D., Patel, P., and Rodger, S. H. JAWAA: Easy Web-Based Animation from CS O to Advanced CS Courses. In Proceedings of the 34thACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE 2003), Reno, Nevada (2003), ACM Press, New York, pp. 162--166.

Digital Library

[3]

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. Standards for Educational and Psychological Testing. American Educational Research Association, Washington, D.C., USA, 1999.

[4]

Bederson, B. Interfaces for Staying in the Flow. Keynote at IEEE Human-Centric Computing Languages and Environments. Available online at http://www.cs.umd.edu/~bederson/talks/HCCKeynote-Sept2002.ppt (seen July 14, 2003).

[5]

Bloom, B. S., and Krathwohl, D. R. Taxonomy of Educational Objectives; the Classification of Educational Goals, Handbook I: Cognitive Domain. Addison-Wesley, 1956.

[6]

Boroni, C. M., Goosey, F. W., Grinder, M. T., and Ross, R. J. Engaging Students with Active Learning Resources: Hypertextbooks for the Web. In Proceedings of the 32nd ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE 2001), Charlotte, North Carolina (2001), ACM Press, New York, pp. 65--69.

Digital Library

[7]

Brummund, P. The Complete Collection of Algorithm Animations, 1998. Available online at http://cs.hope.edu/~alganim/ccaa/ (see July 14, 2003).

[8]

CITIDEL - Computing and Information Technology Interactive Digital Educational Library. Available online at www.citidel.org (seen July 14, 2003).

[9]

Cooper, S., Dann, W., and Pausch, R. Introduction to OO: Teaching Objects-First in Introductory Computer Science. In Proceedings of the 34th ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE 2003), Reno, Nevada (2003), ACM Press, New York, pp. 191--195.

Digital Library

[10]

Crescenzi, P., Faltin, N., Fleischer, R., Hundhausen, C., Näher, S., Rößling, G., Stasko, J., and Sutinen, E. The Algorithm Animation Repository. Proceedings of the Second International Program Visualization Workshop, HornstrupCentret, Denmark (2002), 14--16. Available online at http://www.daimi.au.dk/PB/567/PB-567.pdf.

[11]

Dann, W., Cooper, S., and Pausch, R. Making the Connection: Programming With Animated Small World. In Proceedings of the 5th Annual ACM SIGCSE/SIGCUE Conference on Innovation and Technology in Computer Science Education (ITiCSE 2000), Helsinki, Finland (2000), ACM Press, New York, pp. 41--44.

Digital Library

[12]

Felder, R. M. Author's preface -- June 2002. http://www2.ncsu.edu/unity/lockers/users/f/felder/public/Papers/LS-1988%.pdf (seen July 14, 2003), 2002.

[13]

Felder, R. M., and Silverman, L. K. Learning Styles and Teaching Styles in Engineering Education. Engineering Education 78, 7 (1988), 674--681.

[14]

Fleischer, R. COMP 272: Theory of Computation --- A proposed study on the learning effectiveness of visualizations, 2003. Manuscript.

[15]

Francis, L. Attitude towards Computers Scale. Computers in Education 20, 3 (1993), 251--255.

Digital Library

[16]

Frechtling, J. The 2002 User Friendly Handbook for Project Evaluation. National Science Foundation, 2002.

[17]

Frechtling, J., and Sharp, L. User-Friendly Handbook for Project Evaluation. National Science Foundation, 1997.

[18]

Good, J. Programming Paradigms, Information Types and Graphical Representations: Empirical Investigations of Novice Program Comprehension. PhD thesis, University of Edinburgh, 1999.

[19]

Good, J., and Brna, P. Toward Authentic Measures of Program Comprehension. In Proceedings of the Fifteenth Annual Workshop of the Psychology of Programming Interest Group (PPIG 2003) (2003), pp. 29--49.

[20]

Grissom, S., McNally, M., and Naps, T. L. Algorithm Visualization in Computer Science Education: Comparing Levels of Student Engagement. In Proceedings of the First ACM Symposium on Software Visualization, San Diego, California (2003), ACM Press, New York, pp. 87--94.

Digital Library

[21]

Jenkins, J., and Visser, G. Making Learning Fun. Available online at http://www.imaginal.nl/learningFun.htm (seen July 14, 2003).

[22]

Keirsey, D. M. Keirsey Temperament and Character Web Site. Available online at WWW: http://www.keirsey.com (seen July 14, 2003), 2002.

[23]

Kolb, D. A., Ed. Experiential Learning: Experience as the Source of Learning and Development. Prentice-Hall Inc, New Jersey, USA, 1984.

[24]

Koldehofe, B., Papatriantafilou, M., and Tsigas, P. LYDIAN, An Extensible Educational Animation Environment for Distributed Algorithms. In Proceedings of the 4th Annual ACM SIGCSE/SIGCUE Conference on Innovation and Technology in Computer Science Education (ITiCSE 2000) (July 2000), ACM Press, New York, p. 189.

Digital Library

[25]

Koldehofe, B., Papatriantafilou, M., and Tsigas, P. Integrating a Simulation-Visualization Environment in a Basic Distributed Systems Course: A Case Study Using LYDIAN. In Proceedings of the 8th Annual ACM SIGCSE/SIGCUE Conference on Innovation and Technology in Computer Science Education (ITiCSE 2003) (June 2002), p. 226.

Digital Library

[26]

Korhonen, A., and Malmi, L. Algorithm Simulation with Automatic Assessment. In Proceedings of the 5th Annual ACM SIGCSE/SIGCUE Conference on Innovation and Technology in Computer Science Education (ITiCSE 2000) (Helsinki, Finland, 2000), ACM, pp. 160--163.

Digital Library

[27]

Korhonen, A., Malmi, L., Mård, P., Salonen, H., and Silvasti, P. Electronic course material on Data structures and Algorithms. In Proceedings of the Second Annual Finnish / Baltic Sea Conference on Computer Science Education (October 2002), pp. 16--20.

[28]

Korhonen, A., Malmi, L., Myllyselkä, P., and Scheinin, P. Does it Make a Difference if Students Exercise on the Web or in the Classroom? In Proceedings of the 7th Annual ACM SIGCSE/SIGCUE Conference on Innovation and Technology in Computer Science Education (ITiCSE 2002) (Århus, Denmark, 2002), ACM Press, New York, pp. 121--124.

Digital Library

[29]

Korhonen, A., Malmi, L., Nikander, J., and Silvasti, P. Algorithm Simulation --- A Novel Way to Specify Algorithm Animations. In Proceedings of the Second International Program Visualization Workshop, Hornstrup Centret, Denmark (2002), pp. 28--36. Available online at http://www.daimi.au.dk/PB/567/PB-567.pdf.

[30]

Korhonen, A., Malmi, L., Nikander, J., and Tenhunen, P. Interaction and Feedback in Automatically Assessed Algorithm Simulation Exercises. Accepted for publication in Journal of Information Technology Education (2003).

[31]

Kuittinen, M., and Sajaniemi, J. First Results of An Experiment on Using Roles of Variables in Teaching. In EASE & PPIG 2003, Papers of the Joint Conference at Keele University (2003), pp. 347--357.

[32]

Levy, R. B.-B., Ben-Ari, M., and Uronen, P. A. An Extended Experiment with Jeliot 2000. In Proceedings of the First International Program Visualization Workshop, Porvoo, Finland (July 2001), University of Joensuu Press, Finland, pp. 131--140.

[33]

Loyd, B. H., and Gressard, C. P. Computer Attitude Scale. Journal of Computing Research 15, 3 (1996), 241--259.

[34]

Malmi, L., Korhonen, A., and Saikkonen, R. Experiences in Automatic Assessment on Mass Courses and Issues for Designing Virtual Courses. In Proceedings of the 7th Annual ACM SIGCSE/SIGCUE Conference on Innovation and Technology in Computer Science Education (ITiCSE 2002) (Århus, Denmark, 2002), ACM Press, New York, pp. 55--59.

Digital Library

[35]

Naps, T. L., Rößling, G., Almstrum, V., Dann, W., Fleischer, R., Hundhausen, C., Korhonen, A., Malmi, L., McNally, M., Rodger, S., and Vel&aacutelzquez-Iturbide, J. ÁÁ. Exploring the Role of Visualization and Engagement in Computer Science Education. ACM SIGCSE Bulletin 35, 2 (June 2003), 131--152.

Digital Library

[36]

Nielsen, J. Designing Web Usability. The Practice of Simplicity. New Riders Publishing, 1999.

Digital Library

[37]

Norman, D. "Top Ten Mistakes" Revisited Three Years Later. Available online at http://www.useit.com/alertbox/990502.html (seen July 14, 2003), May 1999.

[38]

Pennington, N. Comprehension Strategies in Programming. In Empirical Studies of Programmers: Second Workshop (1987), G. M. Olson, S. Sheppard, and E. Soloway, Eds., Ablex Publishing Company, pp. 100--113.

Digital Library

[39]

Price, B., Baecker, R., and Small, I. An Introduction to Software Visualization. In Software Visualization, J. Stasko, J. Domingue, M. H. Brown, and B. A. Price, Eds. MIT Press, 1998, ch. 1, pp. 3--27.

[40]

Ross, R. J. Hypertextbooks for the Web. In Proceedings of the First International Program Visualization Workshop, Porvoo, Finland (July 2001), University of Joensuu Press, Finland, pp. 221--233.

[41]

Ross, R. J., and Grinder, M. T. Hypertextbooks: Animated, Active Learning, Comprehensive Teaching and Learning Resources for the Web. In Software Visualization (2002), S. Diehl, Ed., no. 2269 in Lecture Notes in Computer Science, Springer, pp. 269--284.

Digital Library

[42]

Rößling, G. Algorithm Animation Repository. Available online at http://www.animal.ahrgr.de/ (seen July 14, 2003), 2001.

[43]

Rößling, G., and Freisleben, B. ANIMAL: A System for Supporting Multiple Roles in Algorithm Animation. Journal of Visual Languages and Computing 13, 2 (2002), 341--354.

[44]

Sajaniemi, J., and Kuittinen, M. An Experiment on Using Roles of Variables in Teaching Introductory Programming. Submitted to Empirical Software Engineering (2003).

[45]

Schwarm, S., and VanDeGrift, T. Making Connections: Using Classroom Assessment to Elicit Students' Prior Knowledge and Construction of Concepts. In Proceedings of the 8th Annual ACM SIGCSE/SIGCUE Conference on Innovation and Technology in Computer Science Education (ITiCSE 2003) (2003), ACM Press, New York, pp. 65--69.

Digital Library

[46]

Visualization Working Group Home Page. Available online at http://www.algoanim.net, 2003.

[47]

Kuittinen, M., and Sajaniemi, J. Home Page of the Study on Variable Roles and Role-Based Animation. Available online at http://cs.joensuu.fi/~saja/var_roles/, 2003.

[48]

Leska, C. Homepage of the Visualization Study in Computer Literacy. Available online at http://faculty.rmc.edu/cleska/public_html/algvizstudy.htm, 2003.

[49]

Rantakokko, J. Homepage of the Visualization Study in Parallel Computing. Available online at http://user.it.uu.se/~jarmo/hgur.html, 2003.

Cited By

Mukasheva MKalkabayeva ZPussyrmanov N(2023)Visualization of sorting algorithms in the virtual reality environmentFrontiers in Education10.3389/feduc.2023.11952008Online publication date: 9-Aug-2023
https://doi.org/10.3389/feduc.2023.1195200
Mtaho AMselle LMasoud M(2023)Влияние использования конгруэнтного подхода к визуализации на изучение курса CS2Education & Pedagogy Journal10.23951/2782-2575-2023-2-60-76(60-76)Online publication date: 14-Jul-2023
https://doi.org/10.23951/2782-2575-2023-2-60-76
Robbins HGutekunst SShmoys DWilliamson DDoyle MStephenson BDorn BSoh LBattestilli L(2023)GILP: An Interactive Tool for Visualizing the Simplex AlgorithmProceedings of the 54th ACM Technical Symposium on Computer Science Education V. 110.1145/3545945.3569815(108-114)Online publication date: 2-Mar-2023
https://dl.acm.org/doi/10.1145/3545945.3569815
Show More Cited By

Index Terms

Evaluating the educational impact of visualization
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education
        Information science education

Recommendations

Evaluating the educational impact of visualization
ITiCSE-WGR '03: Working group reports from ITiCSE on Innovation and technology in computer science education

The educational impact of visualization depends not only on how well students learn when they use it, but also on how widely it is used by instructors. Instructors believe that visualization helps students learn. The integration of visualization ...
Merging interactive visualizations with hypertextbooks and course management
ITiCSE-WGR '06: Working group reports on ITiCSE on Innovation and technology in computer science education

As a report of a working group at ITiCSE 2006, this paper provides a vision of how visualizations and the software that generates them may be integrated into hypertextbooks and course management systems. This integration generates a unique synergy that ...
Flipped University Classrooms: Using Technology to Enable Sound Pedagogy

The three case studies in this paper show how flipped classroom approaches can facilitate the renewal of university teaching. The case studies form part of a scholarship of teaching and learning that provides opportunities for educators to learn from ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM SIGCSE Bulletin

ACM SIGCSE Bulletin Volume 35, Issue 4

December 2003

144 pages

ISSN:0097-8418

DOI:10.1145/960492

Issue’s Table of Contents

ITiCSE-WGR '03: Working group reports from ITiCSE on Innovation and technology in computer science education
June 2003
291 pages
ISBN:9781450374538
DOI:10.1145/960875
Conference Chairs:
Vassilios Dagdilelis
University of Macedonia, Greece
,
Maya Satratzemi
University of Macedonia, Greece
,
Editor:
David Finkel
Worcester Polytechnic Institute
,
Program Chairs:
Roger Boyle
University of Leeds, UK
,
Georgios Evangelidis
University of Macedonia, Greece

Copyright © 2003 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 June 2003

Published in SIGCSE Volume 35, Issue 4

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

110
Total Citations
View Citations
2,780
Total Downloads

Downloads (Last 12 months)159
Downloads (Last 6 weeks)14

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mukasheva MKalkabayeva ZPussyrmanov N(2023)Visualization of sorting algorithms in the virtual reality environmentFrontiers in Education10.3389/feduc.2023.11952008Online publication date: 9-Aug-2023
https://doi.org/10.3389/feduc.2023.1195200
Mtaho AMselle LMasoud M(2023)Влияние использования конгруэнтного подхода к визуализации на изучение курса CS2Education & Pedagogy Journal10.23951/2782-2575-2023-2-60-76(60-76)Online publication date: 14-Jul-2023
https://doi.org/10.23951/2782-2575-2023-2-60-76
Robbins HGutekunst SShmoys DWilliamson DDoyle MStephenson BDorn BSoh LBattestilli L(2023)GILP: An Interactive Tool for Visualizing the Simplex AlgorithmProceedings of the 54th ACM Technical Symposium on Computer Science Education V. 110.1145/3545945.3569815(108-114)Online publication date: 2-Mar-2023
https://dl.acm.org/doi/10.1145/3545945.3569815
Kakucs AKátai ZHarangus K(2023)A Context-Based Approach to Teaching Dynamic ProgrammingProceedings of the International Conference on Paradigms of Computing, Communication and Data Sciences10.1007/978-981-19-8742-7_53(671-683)Online publication date: 24-Feb-2023
https://doi.org/10.1007/978-981-19-8742-7_53
Safari Bazargani JZafari MSadeghi-Niaraki AChoi S(2022)A Survey of GIS and AR Integration: ApplicationsSustainability10.3390/su14161013414:16(10134)Online publication date: 16-Aug-2022
https://doi.org/10.3390/su141610134
Constantinescu ZVladoiu M(2021)viztop – Intuitive Visualization of Remote Real-Time Monitoring of Linux Processes2021 20th RoEduNet Conference: Networking in Education and Research (RoEduNet)10.1109/RoEduNet54112.2021.9638285(1-6)Online publication date: 4-Nov-2021
https://doi.org/10.1109/RoEduNet54112.2021.9638285
Prvan MOžEGOVIć J(2020)Methods in Teaching Computer NetworksACM Transactions on Computing Education10.1145/339496320:3(1-35)Online publication date: 16-Jun-2020
https://dl.acm.org/doi/10.1145/3394963
Kahng MChau D(2020)How Does Visualization Help People Learn Deep Learning? Evaluating GAN Lab with Observational Study and Log Analysis2020 IEEE Visualization Conference (VIS)10.1109/VIS47514.2020.00060(266-270)Online publication date: Oct-2020
https://doi.org/10.1109/VIS47514.2020.00060
Lyra KReis RCruz WIsotani S(2019)Um framework de classificação de complexidade para infográficosRevista Brasileira de Informática na Educação10.5753/rbie.2019.27.01.19627:1(196-223)Online publication date: 1-Jan-2019
https://doi.org/10.5753/rbie.2019.27.01.196
Lytle NFloryan MBarnes THawthorne EPérez-Quiñones MHeckman SZhang J(2019)Effects of a Pathfinding Program Visualization on Algorithm DevelopmentProceedings of the 50th ACM Technical Symposium on Computer Science Education10.1145/3287324.3287391(225-231)Online publication date: 22-Feb-2019
https://dl.acm.org/doi/10.1145/3287324.3287391
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents