research-article

Flipping the Assessment of Cognitive Load: Why and How

Authors:

Barry WilksAuthors Info & Claims

ICER '16: Proceedings of the 2016 ACM Conference on International Computing Education Research

Pages 43 - 52

https://doi.org/10.1145/2960310.2960321

Published: 25 August 2016 Publication History

Abstract

Cognitive load theory is typically used to evaluate and improve learning materials, with the goal of optimising students' opportunity to acquire new knowledge and understanding. The cognitive load on a student is typically assessed either objectively, by taking physiological measurements while the student is learning, or subjectively, by asking the student to complete an appropriate questionnaire after the learning experience. However, there are circumstances in which a decision on learning materials must be made before those materials are developed and deployed, whereupon it is not helpful to measure the students during learning or to survey them after learning. Such circumstances necessitate a completely different approach, in which the assessment of the likely imposition of cognitive load is made by the instructors and informs the development of the learning materials.

This paper explores such a situation: the choice of a programming language and integrated development environment for an introductory programming course. The paper explains the impracticality of addressing this choice by way of the usual measures of cognitive load. It then presents the approach that was used, flipping the assessment of expected cognitive load from the students to the instructors. The paper explains how this was done, presents the findings, and concludes by suggesting possibilities for future work in the area.

References

[1]

Paul Ayres and John Sweller. 1990. Locus of Difficulty in Multistage Mathematics Problems. Am. J. Psychol. 103, 2 (1990), 167.

[2]

A. Baddeley. 1992. Working Memory. Science 255 (1992), 556--559.

[3]

Jens Bennedsen and Michael E. Caspersen. 2007. Failure rates in introductory programming. SIGCSE Bulletin 39, 2 (2007), 32--36.

Digital Library

[4]

Benedict du Boulay. 1986. Some difficulties of learning to program. J. Educ. Comput. Res. 2, 1 (1986), 57--73.

[5]

Duane Buck and David J. Stucki. 2000. Design early considered harmful: graduated exposure to complexity and structure based on levels of cognitive development. (2000). http://doi.acm.org.ezproxy.scu.edu.au/10.1145/330908.331817

Digital Library

[6]

Richard Catrambone. 1998. The subgoal learning model: Creating better examples so that students can solve novel problems. J. Exp. Psychol. Gen. 127, 4 (1998), 355--376.

[7]

Paul Chandler and John Sweller. 1991. Cognitive Load Theory and the Format of Instruction. Cogn. Instr. 8 (1991), 293--332.

[8]

Paul Chandler and John Sweller. 1996. Cognitive Load While Learning to Use a Computer Program. Appl. Cogn. Psychol. 10, 2 (April 1996), 151--170.

[9]

Graham Cooper. 1998. Research into Cognitive Load Theory and Instructional Design at UNSW. (1998). Retrieved October 12, 2000 from http://dwb4.unl.edu/Diss/Cooper/UNSW.htm

[10]

Graham Cooper and John Sweller. 1987. Effects of schema acquisition and rule automation on mathematical problem-solving transfer. J. Educ. Psychol. 79 (1987), 347--362.

[11]

Graham Cooper, Sharon Tindall-Ford, Paul Chandler, and John Sweller. 2001. Learning by imagining. J. Exp. Psychol. Appl. 7, 1 (2001), 68--82.

[12]

N. Cowan. 2001. The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behav. Brain Sci. 24, 1 (February 2001), 87--114; discussion 114--185.

[13]

Mark H. Goadrich and Michael P. Rogers. 2011. Smart smartphone development: iOS versus android. In ACM Press, 607.

Digital Library

[14]

Slava Kalyuga. 2007. Expertise Reversal Effect and Its Implications for Learner-Tailored Instruction. Educ. Psychol. Rev. 19, 4 (October 2007), 509--539.

[15]

Slava Kalyuga, Paul Chandler, and John Sweller. 1999. Managing split-attention and redundancy in multimedia instruction. Appl. Cogn. Psychol. 13, 4 (August 1999), 351--371.

[16]

Slava Kalyuga, Paul Chandler, Juhani Tuovinen, and John Sweller. 2001. When problem solving is superior to studying worked examples. J. Educ. Psychol. 93, 3 (2001), 579--588.

[17]

Caitlin Kelleher and Randy Pausch. 2007. Using storytelling to motivate programming. Commun. ACM 50, 7 (July 2007), 58.

Digital Library

[18]

M. Kölling, B. Quig, A. Patterson, and J. Rosenberg. 2003. The BlueJ system and its pedagogy. J. Comput. Sci. Educ. Spec. Issue Learn. Teach. Object Technol. 13, 4 (2003).

[19]

Raymond Lister et al. 2004. A Multi-National Study of Reading and Tracing Skills in Novice Programmers. ACM SIGCSE Bull. 36, 4 (2004), 119--150.

Digital Library

[20]

Lauren E. Margulieux, Mark Guzdial, and Richard Catrambone. 2012. Subgoal-labeled instructional material improves performance and transfer in learning to develop mobile applications. In ICER '12 Proceedings of the ninth annual international conference on International computing education research. Auckland, New Zealand: ACM Press, 71--78.

Digital Library

[21]

F. Marton and R. Säljö. 1976. On Qualitative Differences in Learning: I - Outcome and Process. Br. J. Educ. Psychol. 46, 1 (February 1976), 4--11.

[22]

Raina Mason and Graham Cooper. 2014. Introductory Programming Courses in Australia and New Zealand in 2013 - trends and reasons. In Jacqueline Whalley & Daryl D'Souza, eds. Proceedings of the Sixteenth Australasian Computing Education Conference (ACE2014). CRPIT. Auckland, New Zealand: ACS, 139--147.

Digital Library

[23]

Raina Mason, Graham Cooper, and Michael de Raadt. 2012. Trends in Introductory Programming Courses in Australian Universities Languages, Environments and Pedagogy. In Michael de Raadt & Angela Carbone, eds. Proceedings of the Fourteenth Australasian Computing Education Conference (ACE2012). Melbourne, Australia: Australian Computer Society, Inc., 33--42.

Digital Library

[24]

Raina Mason, Graham Cooper, Simon, and Barry Wilks. 2015. Using Cognitive Load Theory to select an environment for teaching mobile apps development. In Daryl D'Souza & Katrina Falkner, eds. Proceedings of the 17th Australasian Computing Education Conference (ACE2015). Sydney, Australia: ACS, 47--56.

[25]

Richard E. Mayer. 2001. Multimedia Learning, New York, NY: Cambridge University Press.

[26]

Richard E. Mayer and Paul Chandler. 2001. When learning is just a click away: Does simple user interaction foster deeper understanding of multimedia messages? J. Educ. Psychol. 93 (2001), 390--397.

[27]

Michael McCracken et al. 2001. A multi-national, multi-institutional study of assessment of programming skills of first-year CS students. ACM SIGCSE Bull. 33, 4 (2001), 125--180. group report

Digital Library

[28]

George Miller. 1956. The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information. Psychol. Rev. 63 (1956), 81--97.

[29]

Briana B. Morrison, Brian Dorn, and Mark Guzdial. 2014. Measuring cognitive load in introductory CS: adaptation of an instrument. In ACM Press, 131--138.

Digital Library

[30]

Fred Paas, Alexander Renkl, and John Sweller. 2004. Cognitive Load Theory: Instructional implications of the interaction between information structures and cognitive architecture. Instr. Sci. 32 (2004), 1--8.

[31]

C. Schulte and J. Bennedsen. 2006. Teachers & learners: which to study? What do teachers teach in introductory programming? (2006). http://doi.acm.org.ezproxy.scu.edu.au/10.1145/1151588.1151593

Digital Library

[32]

Ben Shneiderman and Catherine Plaisant. 2010. Designing the user interface: strategies for effective human-computer interaction 5th ed., Boston: Addison-Wesley.

Digital Library

[33]

Simon, and David Conforth. 2014. Teaching Mobile Apps for Windows Devices Using TouchDevelop. In Proceedings of the 16th Australasian Computing Education Conference. Auckland, New Zealand: Australian Computer Society, Inc., 75--82.

Digital Library

[34]

John Sweller. 1988. Cognitive Load During Problem Solving: Effects on Learning. Cogn. Sci. 12 (1988), 257--285.

[35]

John Sweller. 1999. Instructional design in technical areas, Camberwell, VIC: The Australian Council for Educational Research Ltd.

[36]

J. Sweller, P. Ayres, and S. Kalyuga. Cognitive Load Theory, volume 1. Springer, 2011.

[37]

John Sweller and Paul Chandler. 1994. Why some material is difficult to learn. Cogn. Instr. 12 (1994), 185--233.

[38]

John Sweller and Graham Cooper. 1985. The Use of Worked Examples as a Substitute for Problem Solving in Learning Algebra. Cogn. Instr. 2, 1 (1985), 59--89.

[39]

John Sweller, J.J.G. Van Merriënboer, and Fred Paas. 1998. Cognitive Architecture and Instructional Design. Educ. Psychol. Rev. 10, 3 (1998), 251--296.

[40]

Rohani A. Tarmizi and John Sweller. 1988. Guidance during mathematical problem solving. J. Educ. Psychol. 80, 4 (1988), 424--436.

[41]

The LEGO Group. 2009. MINDSTORMS. (2009). Retrieved August 28, 2009 from http://mindstorms.lego.com/en-us/Default.aspx

[42]

Christopher Watson and Frederick W.B. Li. 2014. Failure rates in introductory programming revisited. In ITiCSE '14 Proceedings of the 2014 conference on Innovation & Technology in Computer Science Education. Uppsala, Sweden: ACM Press, 39--44.

Digital Library

Cited By

Duran RZavgorodniaia ASorva J(2022)Cognitive Load Theory in Computing Education Research: A ReviewACM Transactions on Computing Education10.1145/348384322:4(1-27)Online publication date: 15-Sep-2022
https://dl.acm.org/doi/10.1145/3483843
Nelson GKo A(2019)John Henry AWARDACM Inroads10.1145/330615310:1(74-83)Online publication date: 8-Feb-2019
https://dl.acm.org/doi/10.1145/3306153
Nelson GKo AMalmi LKorhonen AMcCartney RPetersen AMalmi LMcCartney R(2018)On Use of Theory in Computing Education ResearchProceedings of the 2018 ACM Conference on International Computing Education Research10.1145/3230977.3230992(31-39)Online publication date: 8-Aug-2018
https://dl.acm.org/doi/10.1145/3230977.3230992
Show More Cited By

Index Terms

Flipping the Assessment of Cognitive Load: Why and How
1. Social and professional topics
  1. Professional topics
    1. Computing education

Recommendations

Cognitive Load Theory and Computer Science Education
SIGCSE '16: Proceedings of the 47th ACM Technical Symposium on Computing Science Education

Cognitive load theory uses our knowledge of human cognitive architecture to devise instructional procedures, most of which are directly relevant to computer science education. There are several basic aspects of human cognition that are critical to ...
A Pedagogical Framework for Teaching Computer Programming: A Social Constructivist and Cognitive Load Theory Approach
ICER 2021: Proceedings of the 17th ACM Conference on International Computing Education Research

Computer programming is widely regarded as a complex learning domain with high element interactivity leading to some students experiencing low levels of success due to high cognitive load. Collaborative cognitive load theory may provide a way to ...
Measuring cognitive load in introductory CS: adaptation of an instrument
ICER '14: Proceedings of the tenth annual conference on International computing education research

A student's capacity to learn a concept is directly related to how much cognitive load is used to comprehend the material. The central problem identified by Cognitive Load Theory is that learning is impaired when the total amount of processing ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICER '16: Proceedings of the 2016 ACM Conference on International Computing Education Research

August 2016

310 pages

ISBN:9781450344494

DOI:10.1145/2960310

General Chairs:
Judy Sheard
Monash University, Australia
,
Josh Tenenberg
University of Washington Tacoma, USA
,
Donald Chinn
University of Washington Tacoma, USA
,
Brian Dorn
University of Nebraska at Omaha, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 August 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICER '16

Sponsor:

SIGCSE

ICER '16: International Computing Education Research Conference

September 8 - 12, 2016

VIC, Melbourne, Australia

Acceptance Rates

ICER '16 Paper Acceptance Rate 26 of 102 submissions, 25%;

Overall Acceptance Rate 189 of 803 submissions, 24%

Upcoming Conference

ICER 2025

Sponsor:
sigcse

ACM Conference on International Computing Education Research

August 3 - 6, 2025

Charlottesville , VA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
445
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)5

Reflects downloads up to 30 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Duran RZavgorodniaia ASorva J(2022)Cognitive Load Theory in Computing Education Research: A ReviewACM Transactions on Computing Education10.1145/348384322:4(1-27)Online publication date: 15-Sep-2022
https://dl.acm.org/doi/10.1145/3483843
Nelson GKo A(2019)John Henry AWARDACM Inroads10.1145/330615310:1(74-83)Online publication date: 8-Feb-2019
https://dl.acm.org/doi/10.1145/3306153
Nelson GKo AMalmi LKorhonen AMcCartney RPetersen AMalmi LMcCartney R(2018)On Use of Theory in Computing Education ResearchProceedings of the 2018 ACM Conference on International Computing Education Research10.1145/3230977.3230992(31-39)Online publication date: 8-Aug-2018
https://dl.acm.org/doi/10.1145/3230977.3230992
Toma LVahrenhold JMalmi LKorhonen AMcCartney RPetersen AMalmi LMcCartney R(2018)Self-Efficacy, Cognitive Load, and Emotional Reactions in Collaborative Algorithms Labs - A Case StudyProceedings of the 2018 ACM Conference on International Computing Education Research10.1145/3230977.3230980(1-10)Online publication date: 8-Aug-2018
https://dl.acm.org/doi/10.1145/3230977.3230980

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents