research-article

Developing Assessments to Determine Mastery of Programming Fundamentals

Authors:

Andrew Luxton-Reilly,

Brett A. Becker,

Roger McDermott,

Claudio Mirolo,

Andreas Mühling,

Andrew Petersen,

Jacqueline WhalleyAuthors Info & Claims

ITiCSE-WGR '17: Proceedings of the 2017 ITiCSE Conference on Working Group Reports

Pages 47 - 69

https://doi.org/10.1145/3174781.3174784

Published: 30 January 2018 Publication History

Abstract

Current learning outcomes for introductory programming courses are relatively general, specifying tasks such as designing, implementing, testing and debugging programs that use some fundamental programming constructs. These outcomes impact what we teach, our expectations, and our assessments. Although prior work has demonstrated the utility of single concept assessments, most assessments used in formal examinations combine numerous heterogeneous concepts, resulting in complex and difficult tasks.

As a consequence, teachers may not be able to diagnose the actual difficulties faced by students and students are not provided with accurate feedback about their achievements. Such limitations on the nature and quality of feedback to teachers and students alike may contribute to the perceived difficulty and high dropout rates commonly observed in introductory programming courses.

In this paper we review the concepts that Computer Science education researchers have identified as important for novice programming. We survey expected learning outcomes for introductory programming courses, and analyse assessments designed for such courses to determine the individual components of syntax and semantics required to complete them. Having recognized the implicit and explicit expectations of novice programming courses, we look at the relationships between components, and progression between concepts. Finally, we demonstrate how some complex assessments can be decomposed into atomic elements that can be assessed independently.

References

[1]

Alireza Ahadi and Raymond Lister Geek Genes, Prior Knowledge, Stumbling Points and Learning Edge Momentum: Parts of the One Elephant?. Proceedings of the Ninth International Computing Education Research Conference (ICER '13) 123--128.

Digital Library

[2]

Deborah J. Armstrong 2006 The Quarks of Object-oriented Development.journal Commun. ACMvolume49,number2 (\dateFebruaryyear2006),pages123--128. ISSN0001-0782

Digital Library

[3]

Association for Computing Machinery (ACM) and IEEE Computer Society, Joint Task Force on Computing Curricula2001}% taskforce.2001.acm.computing\bibfieldauthor\ Association for Computing Machinery (ACM) and IEEE Computer Society, Joint Task Force on Computing Curricula 2001\natexlab.booktitle Computing Curricula 2001: Computer Science. ACM, New York, NY, USA. ISBNx978--1--4503--2309--3

[4]

Association for Computing Machinery (ACM) and IEEE Computer Society, Joint Task Force on Computing Curricula2013}% acmCC.2013\bibfieldauthor\ Association for Computing Machinery (ACM) and IEEE Computer Society, Joint Task Force on Computing Curricula .booktitle Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science. ACM, New York, NY, USA. ISBNx978--1--4503--2309--3

Digital Library

[5]

Mordechai Ben-Ari 1998 Constructivism in Computer Science Education. Proceedings of the 29th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '98) 257--261. ISBNx0--89791--994--7

Digital Library

[6]

Jens Bennedsen and Michael E. Caspersen 2007 Failure Rates in Introductory Programming.journal SIGCSE Bull.volume39,number2 (\dateJuneyear2007),pages32--36. ISSN0097--8418

Digital Library

[7]

Marc Berges, Andreas Mühling, and Peter Hubwieser 2012 The Gap between Knowledge and Ability. Proceedings of the 12th Koli Calling International Conference on Computing Education Research (Koli Calling 2012) 126--134. ISBNx978--1--4503--1795--5

Digital Library

[8]

James H. Block and Robert B. Burns 1976 Mastery Learning.journal Review of Research in Educationvolume4,number1 (1976),pages3--49.

[9]

Benjamin S. Bloom 1968 Learning for Mastery.journal Evaluation Commentvolume1,number2 (1968).

[10]

Benjamin S. Bloom 1984 The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring.journal Educational Researchervolume13,number6 (1984),pages4--16. ISSN0013189X, 1935102X http://www.jstor.org/stable/1175554

[11]

Richard Bornat, Saeed Dehnadi, and Simon 2008 Mental Models, Consistency and Programming Aptitude. Proceedings of the Tenth Australasian Computing Education Conference (ACE '08) 53--61. ISBNx978--1--920682--59-0 http://dl.acm.org/citation.cfm?id=1379249.1379253

Digital Library

[12]

Pauli Byckling and Jorma Sajaniemi 2006 A Role-based Analysis Model for the Evaluation of Novices' Programming Knowledge Development. Proceedings of the Second International Computing Education Research Workshop (ICER '06) 85--96. ISBNx1--59593--494--4

Digital Library

[13]

Ricardo Caceffo, Steve Wolfman, Kellogg S. Booth, and Rodolfo Azevedo 2016 Developing a Computer Science Concept Inventory for Introductory Programming. Proceedings of the 47th SIGCSE Technical Symposium on Computing Science Education (SIGCSE '16) 364--369.

Digital Library

[14]

Dino Capovilla, Marc Berges, Andreas Mühling, and Peter Hubwieser 2015 Handling Heterogeneity in Programming Courses for Freshmen. Proceedings of the International Conference on Learning and Teaching in Computing and Engineering (LaTiCE 2015) 197--203.

Digital Library

[15]

Yuliya Cherenkova, Daniel Zingaro, and Andrew Petersen 2014 Identifying Challenging CS1 Concepts in a Large Problem Dataset. Proceedings of the 45th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '14) 695--700. ISBNx978--1--4503--2605--6

Digital Library

[16]

Liberal Arts Computer Science Consortium 2007 A 2007 Model Curriculum for a Liberal Arts Degree in Computer Science.journal J. Educ. Resour. Comput.volume7,number2, Articlearticleno2 (Juneyear2007). ISSN1531--4278

Digital Library

[17]

Albert T. Corbett and John R. Anderson 1992 Student Modeling and Mastery Learning in a Computer-based Programming Tutor. Proceedings of the Intelligent Tutoring Systems: Second International Conference (ITS '92) 413--420. ISBNx978--3--540--47254--4

Digital Library

[18]

Malcolm Corney, Raymond Lister, and Donna Teague 2011 Early Relational Reasoning and the Novice Programmer: Swapping as the Hello World of Relational Reasoning. Proceedings of the 13th Australasian Computing Education Conference (ACE '11) 95--104. http://crpit.com/confpapers/CRPITV114Corney.pdf

Digital Library

[19]

Nell Dale 2005 Content and Emphasis in CS1.journal SIGCSE Bull.volume37,number4 (\dateDecemberyear2005),pages69--73. ISSN0097--8418

Digital Library

[20]

Stephen Davies, Jennifer A. Polack-Wahl, and Karen Anewalt 2011 A Snapshot of Current Practices in Teaching the Introductory Programming Sequence. Proceedings of the 42nd ACM Technical Symposium on Computer Science Education (SIGCSE '11) 625--630. ISBNx978--1--4503-0500--6

Digital Library

[21]

Ton de Jong 2010 Cognitive load theory, educational research, and instructional design: some food for thought.journal Instructional Sciencevolume38,number2 (\date01 Maryear2010),pages105--134. ISSN1573--1952

[22]

Michael de Raadt, Richard Watson, and Mark Toleman 2005 Textbooks: Under inspection.howpublishedphTechnical Report -- Department of Maths and Computing, University of Southern Queensland. Toowoomba, Queensland, Australia. https://eprints.usq.edu.au/167/1/TechReport_Draft_10.pdf note-- retrieved: September 2017.

[23]

Benedict du Boulay 1986 Some Difficulties of Learning to Program.journal J. of Educational Comput. Researchvolume2,number1 (1986),pages57--73.

[24]

Anna Eckerdal, Robert McCartney, Jan Erik Moström, Mark Ratcliffe, and Carol Zander 2006 Can Graduating Students Design Software Systems?. Proceedings of the 37th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '06) 403--407. ISBNx1--59593--259--3

Digital Library

[25]

Allison Elliott Tew and Mark Guzdial 2010 Developing a Validated Assessment of Fundamental CS1 Concepts. Proceedings of the 41st SIGCSE Technical Symposium on Computer Science Education (SIGCSE '10) 97--101. ISBNx978--1--4503-0006--3

Digital Library

[26]

Allison Elliott Tew and Mark Guzdial 2011 The FCS1: A Language Independent Assessment of CS1 Knowledge. Proceedings of the 42nd SIGCSE Technical Symposium on Computer Science Education (SIGCSE '11) 111--116. ISBNx978--1--4503-0500--6

Digital Library

[27]

Sophie Engle and Sami Rollins Expert Code Review and Mastery Learning in a Software Development Course.journal J. Comput. Sci. Coll.volume28,number4 (2013),pages139--147.

Digital Library

[28]

Heidi L. Eyre 2007 Keller's Personalized System of Instruction: Was it a Fleeting Fancy or is there a Revival on the Horizon'journal The Behavior Analyst Todayvolume8,number3 (2007),pages317--324.

[29]

Mark Guzdial and Briana Morrison 2016 Growing Computer Science Education into a STEM Education Discipline.journal Commun. ACMvolume59,number11 (\dateOctoberyear2016),pages31--33. ISSN0001-0782

Digital Library

[30]

John Hattie and Helen Timperley 2007 The Power of Feedback.journal Review of Educational Researchvolume77,number1 (\dateMarchyear2007),pages81--112. ISSN0034--6543

[31]

Matthew Hertz 2010 What Do “CS1” and “CS2” Mean?: Investigating Differences in the Early Courses. Proceedings of the 41st SIGCSE Technical Symposium on Computer Science Education (SIGCSE '10) 199--203. ISBNx978--1--4503-0006--3

Digital Library

[32]

Cruz Izu, Amali Weerasinghe, and Cheryl Pope 2016 A Study of Code Design Skills in Novice Programmers Using the SOLO Taxonomy. Proceedings of the 12th International Computing Education Research Conference (ICER '16) 251--259. ISBNx978--1--4503--4449--4

Digital Library

[33]

Philip Johnson-Laird 1983\natexlab.booktitle Mental Models. Harvard University Press, Cambridge, USA. ISBNx0--674--56881--8

[34]

Fred. S. Keller 1968 Goodbye, Teacher ...journal Journal of Applied Behavior Analysisnumber1 (1968),pages79--89.

[35]

Chen-Lin C. Kulik, James A. Kulik, and Robert L. Bangert-Drowns 1990 Effectiveness of Mastery Learning Programs: A Meta-Analysis.journal Review of Educational Researchvolume60,number2 (1990),pages265--299.

[36]

Amruth N. Kumar 1996 Fork Diagrams for Teaching Selection in CS I. Proceedings of the 27th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '96) 348--352. ISBNx0--89791--757-X

Digital Library

[37]

Noel LeJeune 2010 Contract Grading with Mastery Learning in CS 1.journal J. Comput. Sci. Coll.volume26,number2 (\dateDecemberyear2010),pages149--156. ISSN1937--4771 http://dl.acm.org/citation.cfm?id=1858583.1858604

Digital Library

[38]

Colleen M. Lewis 2014 Exploring Variation in Students' Correct Traces of Linear Recursion. Proceedings of the Tenth International Computing Education Research Conference (ICER '14) 67--74. ISBNx978--1--4503--2755--8

Digital Library

[39]

Raymond Lister, Elizabeth S. Adams, Sue Fitzgerald, William Fone, John Hamer, Morten Lindholm, Robert McCartney, Jan Erik Moström, Kate Sanders, Otto Seppälä, Beth Simon, and Lynda Thomas 2004 A Multi-national Study of Reading and Tracing Skills in Novice Programmers. 2004 ITiCSE Working Group Reports (ITiCSE-WGR '04) 119--150.

Digital Library

[40]

Raymond Lister, Beth Simon, Errol Thompson, Jacqueline L. Whalley, and Christine Prasad 2006 Not Seeing the Forest for the Trees: Novice Programmers and the SOLO Taxonomy. Proceedings of the 11th Conference on Innovation and Technology in Computer Science Education (ITICSE '06) 118--122. ISBNx1--59593-055--8

Digital Library

[41]

Mike Lopez, Jacqueline Whalley, Phil Robbins, and Raymond Lister 2008 Relationships between Reading, Tracing and Writing Skills in Introductory Programming. Proceedings of the Fourth International Computing Education Research Workshop (ICER '08) 101--112. ISBNx978--1--60558--216-0

Digital Library

[42]

Andrew Luxton-Reilly 2016 Learning to Program is Easy. Proceedings of the 21st Conference on Innovation and Technology in Computer Science Education (ITiCSE '16) 284--289. ISBNx978--1--4503--4231--5

Digital Library

[43]

Andrew Luxton-Reilly and Andrew Petersen 2017 The Compound Nature of Novice Programming Assessments. Proceedings of the 19th Australasian Computing Education Conference (ACE '17) 26--35. ISBNx978--1--4503--4823--2

Digital Library

[44]

Linxiao Ma, John Ferguson, Marc Roper, and Murray Wood 2007 Investigating the Viability of Mental Models Held by Novice Programmers. Proceedings of the 38th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '07) 499--503. ISBNx1--59593--361--1

Digital Library

[45]

Philipp Mayring 2014 title Qualitative Content Analysis: Theoretical Foundation, Basic Procedures and Software Solution. Klagenfurt.

[46]

Brendan McCane, Claudia Ott, Nick Meek, and Anthony Robins 2017 Mastery Learning in Introductory Programming. Proceedings of the 19th Australasian Computing Education Conference (ACE '17) 1--10. ISBNx978--1--4503--4823--2

Digital Library

[47]

Robert McCartney, Jonas Boustedt, Anna Eckerdal, Kate Sanders, and Carol Zander Can First-year Students Program Yet?: A Study Revisited. Proceedings of the Ninth International Computing Education Research Conference (ICER '13) 91--98. ISBNx978--1--4503--2243-0

Digital Library

[48]

Michael McCracken, Vicki Almstrum, Danny Diaz, Mark Guzdial, Dianne Hagan, Yifat Ben-David Kolikant, Cary Laxer, Lynda Thomas, Ian Utting, and Tadeusz Wilusz 2001 A Multi-national, Multi-institutional Study of Assessment of Programming Skills of First-year CS Students.journal SIGCSE Bull.volume33,number4 (\dateDecemberyear2001),pages125--180. ISSN0097--8418

Digital Library

[49]

Iain Milne and Glenn Rowe 2002 Difficulties in Learning and Teaching Programming--Views of Students and Tutors.journal Education and Information Technologiesvolume7,number1 (March 2002), pages 55--66. ISSN1360--2357

Digital Library

[50]

Luiz Augusto de Macêdo Morais, Jorge C. A. Figueiredo, and Dalton D. S. Guerrero 2014 Students Satisfaction with Mastery Learning in an Introductory Programming Course. III Congresso Brasileiro de Informática na Educac cåo (CBIE 2014) 1048--1052.

[51]

Briana B. Morrison, Lauren E. Margulieux, and Mark Guzdial 2015 Subgoals, Context, and Worked Examples in Learning Computing Problem Solving. Proceedings of the 11th Annual International Computing Education Research Conference (ICER '15) 21--29. ISBNx978--1--4503--3630--7

Digital Library

[52]

Andreas Mühling, Alexander Ruf, and Peter Hubwieser 2015 Design and First Results of a Psychometric Test for Measuring Basic Programming Abilities. Proceedings of the Workshop in Primary and Secondary Computing Education (WiPSCE '15) 2--10. ISBNx978--1--4503--3753--3

Digital Library

[53]

Pavol Návrat 1994 Hierarchies of Programming Concepts: Abstraction, Generality, and Beyond.journal SIGCSE Bull.volume26,number3 (Septemberyear1994), pages17--21. ISSN0097--8418

Digital Library

[54]

Fred G. W. C. Paas and Jeroen J. G. van Merriënboer 1994 Instructional Control of Cognitive Load in the Training of Complex Cognitive Tasks.journal Educational Psychology Review volume6,number4 (1994),pages351--371.

[55]

Miranda C. Parker, Mark Guzdial, and Shelly Engleman 2016 Replication, Validation, and Use of a Language Independent CS1 Knowledge Assessment. Proceedings of the 12th International Computing Education Research Conference (ICER '16) 93--101.

Digital Library

[56]

Terence Parr and Sam Harwell 2012--2016 Java 1.7 grammar for ANTLR v4.howpublishedhttps://github.com/antlr/grammars-v4/blob/master/java/Java.g4. (2012--2016).

[57]

Elizabeth Patitsas, Jesse Berlin, Michelle Craig, and Steve Easterbrook 2016 Evidence That Computer Science Grades Are Not Bimodal. Proceedings of the 12th International Computing Education Research Conference (ICER '16) 113--121.

Digital Library

[58]

Michela Pedroni and Bertrand Meyer 2010 Object-Oriented Modeling of Object-Oriented Concepts. Proceedings of the Fourth International Conference on Informatics in Secondary Schools - Evolution and Perspectives (ISSEP 2010). Springer,pages155--169.

Digital Library

[59]

Andrew Petersen, Michelle Craig, Jennifer Campbell, and Anya Tafliovich 2016 Revisiting Why Students Drop CS1. Proceedings of the 16th Koli Calling International Conference on Computing Education Research (Koli Calling 2016) 71--80. ISBNx978--1--4503--4770--9

Digital Library

[60]

Andrew Petersen, Michelle Craig, and Daniel Zingaro 2011 Reviewing CS1 Exam Question Content. Proceedings of the 42nd SIGCSE Technical Symposium on Computer Science Education (SIGCSE '11) 631--636. ISBNx978--1--4503-0500--6

Digital Library

[61]

Jean Piaget 1976 Piaget's Theory. Piaget and His School: A Reader in Developmental Psychology, editor Bärbel Inhelder, Harold H. Chipman, and Charles Zwingmann (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, pages11--23. ISBNx978--3--642--46323--5

[62]

roulx2000}% proulx.2000.sigcse.patterns\bibfieldauthor\ Viera K. Proulx 2000 Programming Patterns and Design Patterns in the Introductory Computer Science Course. Proceedings of the 31st SIGCSE Technical Symposium on Computer Science Education (SIGCSE '00) 80--84. ISBNx1--58113--213--1

Digital Library

[63]

obins2010}% robins.2010.cse.learning\bibfieldauthor\ Anthony Robins 2010 Learning Edge Momentum: A New Account of Outcomes in CS1.journal Computer Science Educationvolume20,number1 (\dateMarchyear2010),pages37--71. ISSN0899--3408, 1744--5175

[64]

obins et al\mbox.2003}% robins.2003.cse.learning\bibfieldauthor\ Anthony Robins, Janet Rountree, and Nathan Rountree 2003 Learning and Teaching Programming: A Review and Discussion.journal Computer Science Educationvolume13,number2 (\dateJuneyear2003),pages137--142. ISSN0899--3408

[65]

ajaniemi et al\mbox.2006}% sajaniemi.2006.compsciedu.roles\bibfieldauthor\ Jorma Sajaniemi, Mordechai Ben-Ari, Pauli Byckling, Petri Gerdt, and Yevgeniya Kulikova 2006 Roles of Variables in Three Programming Paradigms.journal Computer Science Educationvolume16,number4 (\dateDecemberyear2006),pages261--279. ISSN0737-0024

[66]

anders et al\mbox.2013}% sanders.2013.canterbury\bibfieldauthor\ Kate Sanders, Marzieh Ahmadzadeh, Tony Clear, Stephen H. Edwards, Mikey Goldweber, Chris Johnson, Raymond Lister, Robert McCartney, Elizabeth Patitsas, and Jaime Spacco The Canterbury QuestionBank: Building a Repository of Multiple-choice CS1 and CS2 Questions. 2013 ITiCSE Working Group Reports (ITiCSE-WGR '13) 33--52. ISBNx978--1--4503--2665-0

Digital Library

[67]

anders and Thomas2007}% sanders.2007.sigcse.checklists\bibfieldauthor\ Kate Sanders and Lynda Thomas 2007 Checklists for Grading Object-oriented CS1 Programs: Concepts and Misconceptions. Proceedings of the 12th Conference on Innovation and Technology in Computer Science Education (ITiCSE '07) 166--170. ISBNx978--1--59593--610--3

Digital Library

[68]

choltz and Sanders2010}% scholtz.2010.iticse.mental\bibfieldauthor\ Tamarisk Lurlyn Scholtz and Ian Sanders 2010 Mental Models of Recursion: Investigating Students' Understanding of Recursion. Proceedings of the 15th Conference on Innovation and Technology in Computer Science Education (ITiCSE '10) 103--107. ISBNx978--1--60558--820--9

Digital Library

[69]

chulte and Bennedsen2006}% schulte.2006.icer.teachers\bibfieldauthor\ Carsten Schulte and Jens Bennedsen 2006 What Do Teachers Teach in Introductory Programming?. Proceedings of the Second International Computing Education Research Workshop (ICER '06) 17--28. ISBNx1--59593--494--4

Digital Library

[70]

"a, Ihantola, Isohanni, Sorva, and VihavainenSepp\"al\"a et al\mbox.2015}% seppala.2015.koli.know\bibfieldauthor\ Otto Sepp\"al\"a, Petri Ihantola, Essi Isohanni, Juha Sorva, and Arto Vihavainen 2015 Do We Know How Difficult the Rainfall Problem Is?. Proceedings of the 15th Koli Calling Conference on Computing Education Research (Koli Calling 2015) 87--96. ISBNx978--1--4503--4020--5

Digital Library

[71]

heard et al\mbox.2008}% sheard.2008.iticse.solo\bibfieldauthor\ Judy Sheard, Angela Carbone, Raymond Lister, Beth Simon, Errol Thompson, and Jacqueline L. Whalley 2008 Going SOLO to Assess Novice Programmers. Proceedings of the 13th Conference on Innovation and Technology in Computer Science Education (ITiCSE '08) 209--213. ISBNx978--1--60558-078--4

Digital Library

[72]

Carbone, Chinn, Clear, Corney, D'Souza, Fenwick, Harland, Laakso, and TeagueSheard et al\mbox.2013}% sheard.2013.ace.difficult\bibfieldauthor\ Judy Sheard, Simon, Angela Carbone, Donald Chinn, Tony Clear, Malcolm Corney, Daryl D'Souza, Joel Fenwick, James Harland, Mikko-Jussi Laakso, and Donna Teague How Difficult Are Exams?: A Framework for Assessing the Complexity of Introductory Programming Exams. Proceedings of the 15th Australasian Computing Education Conference (ACE '13) 145--154. ISBNx978--1--921770--21-0 http://dl.acm.org/citation.cfm?id=2667199.2667215

Digital Library

[73]

huhidan et al\mbox.2009}% shuhidan.2009.ace.taxonomic\bibfieldauthor\ Shuhaida Shuhidan, Margaret Hamilton, and Daryl D'Souza A Taxonomic Study of Novice Programming Summative Assessment. Proceedings of the 11th Australasian Computing Education Conference (ACE '09) 147--156. ISBNx978--1--920682--76--7 http://dl.acm.org/citation.cfm?id=1862712.1862734

Digital Library

[74]

Robert M. Siegfried, Jason P. Siegfried, and Gina Alexandro 2016 A Longitudinal Analysis of the Reid List of First Programming Languages.journal Information Systems Education Journalvolume14,number6 pages47--54. ISSN1545--679X http://isedj.org/2016--14/

[75]

Simon Soloway's Rainfall Problem Has Become Harder. Proceedings of the 2013 Conference on Learning and Teaching in Computing and Engineering 130--135.

Digital Library

[76]

Simon, Donald Chinn, Michael de Raadt, Anne Philpott, Judy Sheard, Mikko-Jussi Laakso, Daryl D'Souza, James Skene, Angela Carbone, Tony Clear, Raymond Lister, and Geoff Warburton 2012 Introductory Programming: Examining the Exams. Proceedings of the 14th Australasian Computing Education Conference (ACE '12) 61--70. ISBNx978--1--921770-04--3 http://dl.acm.org/citation.cfm?id=2483716.2483724

Digital Library

[77]

Simon, Judy Sheard, Daryl D'Souza, Peter Klemperer, Leo Porter, Juha Sorva, Martijn Stegeman, and Daniel Zingaro 2016\natexlaba.showarticletitleBenchmarking Introductory Programming Exams: How and Why. Proceedings of the 21st Conference on Innovation and Technology in Computer Science Education 154--159. ISBNx978--1--4503--4231--5

Digital Library

[78]

Simon, Judy Sheard, Daryl D'Souza, Peter Klemperer, Leo Porter, Juha Sorva, Martijn Stegeman, and Daniel Zingaro 2016\natexlabb.showarticletitleBenchmarking Introductory Programming Exams: Some Preliminary Results. Proceedings of the 12th International Computing Education Research Conference (ICER '16) 103--111. ISBNx978--1--4503--4449--4

Digital Library

[79]

imon et al\mbox.2006}% simon.2006.icer.commonsense\bibfieldauthor\ Beth Simon, Tzu-Yi Chen, Gary Lewandowski, Robert McCartney, and Kate Sanders 2006 Commonsense Computing: What Students Know before We Teach (Episode 1: Sorting). Proceedings of the Second International Computing Education Research Workshop (ICER '06) 29--40. ISBNx1--59593--494--4

Digital Library

[80]

kudder and Luxton-Reilly2014}% skudder.2014.ace.worked\bibfieldauthor\ Ben Skudder and Andrew Luxton-Reilly 2014 Worked Examples in Computer Science. Proceedings of the 16th Australasian Computing Education Conference (ACE '14) 59--64. ISBNx978--1--921770--31--9 http://dl.acm.org/citation.cfm?id=2667490.2667497

Digital Library

[81]

oloway et al\mbox.1982}% soloway.1982.chi.tapping\bibfieldauthor\ Elliot Soloway, Kate Ehrlich, and Jeffrey Bonar 1982 Tapping into Tacit Programming Knowledge. Proceedings of the 1982 Conference on Human Factors in Computing Systems (CHI '82). ACM, New York, NY, USA,pages52--57.

Digital Library

[82]

Juha Sorva 2012 Visual Program Simulation in Introductory Programming Education.thesistype Ph.D. Dissertation.school Aalto University.

[83]

Juha Sorva, Ville Karavirta, and Lauri Malmi A Review of Generic Program Visualization Systems for Introductory Programming Education.journal Trans. Comput. Educ.volume13,number4, Articlearticleno15 (November 2013). ISSN1946--6226

Digital Library

[84]

Andreas Stefik and Susanna Siebert An Empirical Investigation into Programming Language Syntax.journal ACM Transactions on Computing Educationvolume13,number4 (2013),pages19:1. ISSN1946--6226

Digital Library

[85]

John Sweller Cognitive Load during Problem Solving: Effects on Learning.journal Cognitive Science volume12,number2 (1988),pages257--285. ISSN1551--6709

[86]

John Sweller 1994 Cognitive Load Theory, Learning Difficulty, and Instructional Design.journal Learning and Instruction volume4 (1994), pages295--312.

[87]

John Sweller and Paul Chandler 1994 Why Some Material Is Difficult to Learn.journal Cognition and Instructionvolume12,number3 (1994),pages185--233.

[88]

Cynthia Taylor, Daniel Zingaro, Leo Porter, Kevin Webb, Cynthia Bailey Lee, and Mike Clancy 2014 Computer Science Concept Inventories: Past and Future.journal Computer Science Educationvolume24,number4 (\dateOctoberyear2014),pages253--276.

[89]

Donna Teague and Raymond Lister 2014 Longitudinal Think Aloud Study of a Novice Programmer. Proceedings of the 16th Australasian Computing Education Conference (ACE '14) 41--50. ISBNx978--1--921770--31--9 http://dl.acm.org/citation.cfm?id=2667490.2667495

Digital Library

[90]

Mark Urban-Lurain and Donald J. Weinshank 1999 “I Do and I Understand”: Mastery Model Learning for a Large Non-major Course. Proceedings of the 30th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '99) 150--154. ISBNx1--58113-085--6

Digital Library

[91]

Jeroen J. G. van Merriënboer, Otto Jelsma, and Fred G. W. C. Paas 1992 Training for Reflective Expertise: A Four-Component Instructional Design Model for Complex Cognitive Skills.journal Educational Technology Research and Developmentvolume40,number2 (01 Jun 1992), pages23--43. ISSN1556--6501

[92]

Anne Venables, Grace Tan, and Raymond Lister A Closer Look at Tracing, Explaining and Code Writing Skills in the Novice Programmer. Proceedings of the Fifth International Computing Education Research Workshop (ICER '09) 117--128. ISBNx978--1--60558--615--1

Digital Library

[93]

Christopher Watson and Frederick W.B. Li 2014 Failure Rates in Introductory Programming Revisited. Proceedings of the 19th Conference on Innovation and Technology in Computer Science Education (ITiCSE '14) 39--44. ISBNx978--1--4503--2833--3

Digital Library

[94]

Jacqueline Whalley and Nadia Kasto 2014 How Difficult Are Novice Code Writing Tasks?: A Software Metrics Approach. Proceedings of the 16th Australasian Computing Education Conference (ACE '14) 105--112. ISBNx978--1--921770--31--9 http://dl.acm.org/citation.cfm?id=2667490.2667503

Digital Library

Cited By

Bubnic BMernik MKosar T(2024)Exploring the Predictive Potential of Complex Problem-Solving in Computing Education: A Case Study in the Introductory Programming CourseMathematics10.3390/math1211165512:11(1655)Online publication date: 24-May-2024
https://doi.org/10.3390/math12111655
Bubnic BKovačević ŽKosar T(2024)Can metacognition predict your success in solving problems? An exploratory case study in programmingProceedings of the 24th Koli Calling International Conference on Computing Education Research10.1145/3699538.3699593(1-12)Online publication date: 12-Nov-2024
https://dl.acm.org/doi/10.1145/3699538.3699593
Lovellette EBouvier DMatta J(2024)Contextualization, Authenticity, and the Problem Description EffectACM Transactions on Computing Education10.1145/364386424:2(1-32)Online publication date: 5-Feb-2024
https://dl.acm.org/doi/10.1145/3643864
Show More Cited By

Index Terms

Developing Assessments to Determine Mastery of Programming Fundamentals
1. Social and professional topics
  1. Professional topics
    1. Computing education

Recommendations

Introductory programming: a systematic literature review
ITiCSE 2018 Companion: Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been ...
Developing Assessments to Determine Mastery of Programming Fundamentals
ITiCSE '17: Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education

Current CS1 learning outcomes are relatively general, specifying tasks such as designing, implementing, testing and debugging programs that use some fundamental programming constructs. These outcomes impact what we teach, our expectations, and our ...
A review of introductory programming research 2003–2017
ITiCSE 2018: Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education

A broad review of research on the teaching and learning of programming was conducted by Robins et al. in 2003. Since this work there have been several reviews of research concerned with the teaching and learning of programming, in particular introductory ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ITiCSE-WGR '17: Proceedings of the 2017 ITiCSE Conference on Working Group Reports

January 2018

162 pages

ISBN:9781450356275

DOI:10.1145/3174781

General Chairs:
Judithe Sheard
Monash University, Australia
,
Ari Korhonen
Aalto Univerisity, Finland

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 January 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ITiCSE '17

Sponsor:

SIGCSE

ITiCSE '17: Innovation and Technology in Computer Science Education

July 3 - 5, 2017

Bologna, Italy

Acceptance Rates

ITiCSE-WGR '17 Paper Acceptance Rate 8 of 16 submissions, 50%;

Overall Acceptance Rate 552 of 1,613 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

42
Total Citations
View Citations
1,068
Total Downloads

Downloads (Last 12 months)123
Downloads (Last 6 weeks)12

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bubnic BMernik MKosar T(2024)Exploring the Predictive Potential of Complex Problem-Solving in Computing Education: A Case Study in the Introductory Programming CourseMathematics10.3390/math1211165512:11(1655)Online publication date: 24-May-2024
https://doi.org/10.3390/math12111655
Bubnic BKovačević ŽKosar T(2024)Can metacognition predict your success in solving problems? An exploratory case study in programmingProceedings of the 24th Koli Calling International Conference on Computing Education Research10.1145/3699538.3699593(1-12)Online publication date: 12-Nov-2024
https://dl.acm.org/doi/10.1145/3699538.3699593
Lovellette EBouvier DMatta J(2024)Contextualization, Authenticity, and the Problem Description EffectACM Transactions on Computing Education10.1145/364386424:2(1-32)Online publication date: 5-Feb-2024
https://dl.acm.org/doi/10.1145/3643864
Ali MGhosh SRao PDhegaskar RJawort SMedler AShi MDasgupta S(2023)Taking Stock of Concept Inventories in Computing Education: A Systematic Literature ReviewProceedings of the 2023 ACM Conference on International Computing Education Research - Volume 110.1145/3568813.3600120(397-415)Online publication date: 7-Aug-2023
https://dl.acm.org/doi/10.1145/3568813.3600120
Li YZhang RQu DSamary M(2023)Leveraging LLMs and MLPs in Designing a Computer Science Placement Test System2023 International Conference on Computational Science and Computational Intelligence (CSCI)10.1109/CSCI62032.2023.00276(1670-1676)Online publication date: 13-Dec-2023
https://doi.org/10.1109/CSCI62032.2023.00276
Weeda RSmetsers SBarendsen E(2023)Unraveling novices’ code composition difficultiesComputer Science Education10.1080/08993408.2023.216906734:3(414-441)Online publication date: 24-Jan-2023
https://doi.org/10.1080/08993408.2023.2169067
Velázquez-Iturbide J(2023)Designing Exercises for Block-Based Languages: The Case of ScratchJrProceedings TEEM 2022: Tenth International Conference on Technological Ecosystems for Enhancing Multiculturality10.1007/978-981-99-0942-1_5(50-59)Online publication date: 4-May-2023
https://doi.org/10.1007/978-981-99-0942-1_5
Bastian MMühling A(2022)Comparing estimates of difficulty of programming constructsProceedings of the 22nd Koli Calling International Conference on Computing Education Research10.1145/3564721.3565950(1-12)Online publication date: 17-Nov-2022
https://dl.acm.org/doi/10.1145/3564721.3565950
Cheng QTao AChen HSamary M(2022)Design an Assessment for an Introductory Computer Science Course: A Systematic Literature Review2022 IEEE Frontiers in Education Conference (FIE)10.1109/FIE56618.2022.9962584(1-8)Online publication date: 8-Oct-2022
https://doi.org/10.1109/FIE56618.2022.9962584
Ramabu TSanders ISchoeman M(2022)Nested-Decider: An Animation Program for Aiding Teaching and Learning of Decisions/Nested DecisionsICT Education10.1007/978-3-031-21076-1_8(129-148)Online publication date: 8-Dec-2022
https://doi.org/10.1007/978-3-031-21076-1_8
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents