research-article

Discriminating Programming Strategies in Scratch: Making the Difference between Novice and Experienced Programmers

Authors:

Max Kesselbacher,

Andreas BollinAuthors Info & Claims

WiPSCE '19: Proceedings of the 14th Workshop in Primary and Secondary Computing Education

Article No.: 20, Pages 1 - 10

https://doi.org/10.1145/3361721.3361727

Published: 23 October 2019 Publication History

Abstract

Nowadays, block-based programming environments are often used to offer a gentle introduction to learning a programming language. However, an assessment of students' programming skills based on the results of a programming task is not sufficient to determine all areas students are struggling with. We therefore introduce a learning analytics approach of measuring and evaluating the programming sequences of students that program with Scratch 3. With our measurement framework, it is possible to record, store and analyze programming sequences done on a publicly-available, instrumented Scratch 3 environment. Changes in the programming sequence are categorized regarding the used block types and types of program change. We conducted an exploratory programming trial with lower and upper secondary school students to investigate small-scale programming strategies in the recorded programming sequences. Our goals are to identify students in need of support and to identify recurring patterns used by students successful in the trial. Clustering with k-means makes it possible to identify struggling students based on both interacted block types and types of program changes. Recurring patterns in the programming sequences of successful students show that small-scale programming strategies are very diverse.

References

[1]

Efthimia Aivaloglou and Felienne Hermans. 2016. How Kids Code and How We Know. Proceedings ICER '16 (2016), 53--61.

Digital Library

[2]

L. W. Anderson, D. R. Krathwohl, P. W. Airasian, K. A. Cruikshank, R. E. Mayer, Pintrich P. R., J. D. Raths, and M. C. Wittrock. 2001. A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives (1 ed.). New York: Longman.

[3]

Michal Armoni, Orni Meerbaum-Salant, and Mordechai Ben-Ari. 2015. From Scratch to "real" programming. ACM TOCE 14, 4 (2015), 1--15.

Digital Library

[4]

R. Baker and K. Yacef. 2009. The State of Educational Data Mining in 2009: A review and future visions. JEDM 1, 1 (2009), 3--17.

[5]

Yoav Benjamini and Yosef Hochberg. 1995. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society. Series B (Methodological) 57, 1 (1995), 289--300.

[6]

J. Biggs and K. Collis. 1982. Evaluating the quality of learning: The SOLO taxonomy (Structure of the Observed Learning Outcome) (1 ed.). NY Academic Press.

[7]

Paulo Blikstein, Marcelo Worsley, Chris Piech, Mehran Sahami, Steven Cooper, and Daphne Koller. 2014. Programming Pluralism: Using Learning Analytics to detect patterns in the learning of computer programming. Journal of the Learning Sciences 23, 4 (2014), 561--599.

[8]

Bryce Boe, Charlotte Hill, Michelle Len, Greg Dreschler, Phillip Conrad, and Diana Franklin. 2013. Hairball: Lint-inspired Static Analysis of Scratch Projects. Proceeding SIGCSE '13 (2013), 215.

Digital Library

[9]

Andreas Bollin, Heike Demarle-Meusel, Max Kesselbacher, Corinna Mößlacher, Marianne Rohrer, and Julia Sylle. 2018. The bebras contest in Austria - Do personality, self-concept and general interests play an influential role?. In The 11th Conference on Informatics in Schools - ISSEP '18, Vol. 11169 LNCS. 283--294.

[10]

Code.Org.2019. code.org. Retrieved June 2019 from https://code.org/

[11]

Jacob Cohen. 1977. Differences between Correlation Coefficients. In Statistical Power Analysis for the Behav. Sciences, Jacob Cohen (Ed.). Acad. Press, 109--143.

[12]

Daniel A. Filvà, Marc A. Forment, Francisco J. García-Peñalvo, David F. Escudero, and María J. Casañ. 2019. Clickstream for learning analytics to assess students' behavior with Scratch. Future Generation Computer Systems 93 (2019), 673--686.

Digital Library

[13]

Google Developers. 2019. Blockly. Retrieved June 2019 from https://developers.google.com/blockly/

[14]

Andreas Grillenberger and Ralf Romeike. 2017. Real-Time Data Analyses in Secondary Schools Using a Block-Based Programming Language. In Informatics in Schools: Focus on Learning Programming, Valentina Dagienė and Arto Hellas (Eds.). Springer International Publishing, Cham, 207--218.

[15]

Shuchi Grover and Satabdi Basu. 2017. Measuring student learning in introductory block-based programming: Examining misconceptions of loops, variables, and boolean logic. ACM SIGCSE Technical Symposium on Computer Science Education 2017 (2017), 267--272.

Digital Library

[16]

Jiawei Han, Micheline Kamber, and Jian Pei. 2011. Data Mining: Concepts and Techniques (3 ed.). Morgan Kaufmann.

[17]

Felienne Hermans and Efthimia Aivaloglou. 2016. Do code smells hamper novice programming? A controlled experiment on Scratch programs. IEEE ICPC July (2016), 1--10.

[18]

Felienne Hermans, Kathryn T. Stolee, and David Hoepelman. 2016. Smells in block-based programming languages. Proceedings of IEEE Symposium VL/HCC November (2016), 68--72.

[19]

C. D. Hundhausen, D. M. Olivares, and A. S. Carter. 2017. IDE-Based Learning Analytics for Computing Education: A Process Model, Critical Review, and Research Agenda. ACM Trans. Comput. Educ 17, 26 (2017), 1--26.

Digital Library

[20]

Petri Ihantola, Arto Vihavainen, Alireza Ahadi, Matthew Butler, Jürgen Börstler, Stephen H. Edwards, Essi Isohanni, Ari Korhonen, Andrew Petersen, Kelly Rivers, Miguel Rubio, Judy Sheard, Bronius Skupas, Jaime Spacco, Claudia Szabo, and Daniel Toll. 2015. Educational Data Mining and Learning Analytics in Programming: Literature Review and Case Studies. ITiCSE WGR'16 (2015), 41--63.

Digital Library

[21]

Max Kesselbacher and Andreas Bollin. 2019. Quantifying Patterns and Programming Strategies in Block-based Programming Environments. In Companion Proceedings ICSE '19. IEEE Press, Piscataway, NJ, USA, 254--255.

[22]

K. Kontogiannis. 1997. Evaluation experiments on the detection of programming patterns using software metrics. In Proceedings of the Fourth Working Conference on Reverse Engineering. IEEE Computer Society, Washington, DC, USA, 44--54.

[23]

Raymond Lister. 2011. Concrete and other neo-piagetian forms of reasoning in the novice programmer. Conferences in Research and Practice in Information Technology Series 114, January 2011 (2011), 9--18.

[24]

Raymond Lister. 2016. Toward a developmental epistemology of computer programming. WiPSCE '16 (2016), 5--16.

Digital Library

[25]

Orni Meerbaum-Salant, Michal Armoni, and Mordechai Ben-Ari. 2011. Habits of programming in scratch. ITiCSE '11 (2011), 168.

Digital Library

[26]

Orni Meerbaum-Salant, Michal Armoni, and Mordechai Ben-Ari. 2013. Learning computer science concepts with Scratch. Computer Science Education 23, 3 (2013), 239--264.

[27]

MIT Media Lab. 2019. Scratch. Retrieved June 2019 from https://scratch.mit.edu/

[28]

MIT Media Lab. 2019. Scratch-Blocks. Retrieved June 2019 from https://github.com/LLK/scratch-blocks

[29]

MIT Media Lab. 2019. Scratch-VM. Retrieved June 2019 from https://github.com/LLK/scratch-vm

[30]

J. Moreno and G. Robles. 2014. Automatic detection of bad programming habits in scratch: A preliminary study. In 2014 IEEE FIE Conference Proceedings. 1--4.

[31]

S. Papavlasopoulou, M. N. Giannakos, and L. Jaccheri. 2018. Discovering children's competences in coding through the analysis of Scratch projects. In 2018 IEEE Global Engineering Education Conference (EDUCON). 1127--1133.

[32]

Simon Papert. 1980. Mindstorms: Children, Computers, and Powerful Ideas. Basic Books, Inc., New York, USA.

Digital Library

[33]

Viera K. Proulx. 2000. Programming patterns and design patterns in the introductory computer science course. SIGCSE Bull. 32, 1 (March 2000), 80--84.

Digital Library

[34]

Kelly Rivers, Erik Harpstead, and Ken Koedinger. 2016. Learning Curve Analysis for Programming. In Proceedings ICER '16. 143--151.

Digital Library

[35]

Alaaeddin Swidan, Alexander Serebrenik, and Felienne Hermans. 2017. How do Scratch programmers name variables and procedures? IEEE SCAM 2017 October (2017), 51--60.

[36]

Donna Teague and Raymond Lister. 2014. Longitudinal think aloud study of a novice programmer. CRPIT 148 (2014), 41--50.

[37]

Giovanni M. Troiano, Sam Snodgrass, Erinc Argimak, Gregorio Robles, Gillian Smith, Michael Cassidy, Eli Tucker-Raymond, Gillian Puttick, and Casper Harteveld. 2019. Is My Game OK Dr. Scratch?: Exploring Programming and Computational Thinking Development via Metrics in Student-Designed Serious Games for STEM. In Proceedings IDC '19. ACM, New York, NY, USA, 208--219.

Digital Library

[38]

Christopher Watson and Frederick W.B. Li. 2014. Failure rates in introductory programming revisited. In ITiCSE '14. ACM, New York, USA, 39--44.

Cited By

Pozzan GPadova CMontuori CArfé BVardanega TMonga MLonati VBarendsen ESheard JPaterson J(2024)Experimental Analysis of First-Grade Students' Block-Based Programming Problem Solving ProcessesProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653586(143-149)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653586
Valle Torre MOertel CSpecht M(2024)The Sequence Matters in Learning - A Systematic Literature ReviewProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636880(263-272)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3636555.3636880
Greifenstein LBrune MFuchs THeuer UFraser G(2023)Impact of Hint Content on Performance and Learning: A Study with Primary School Children in a Scratch CourseProceedings of the 18th WiPSCE Conference on Primary and Secondary Computing Education Research10.1145/3605468.3605498(1-10)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3605468.3605498
Show More Cited By

Index Terms

Discriminating Programming Strategies in Scratch: Making the Difference between Novice and Experienced Programmers
1. Social and professional topics
  1. Professional topics
    1. Computing education
2. Software and its engineering
  1. Software creation and management

Recommendations

Quantifying patterns and programming strategies in block-based programming environments
ICSE '19: Proceedings of the 41st International Conference on Software Engineering: Companion Proceedings

Pupils are often first exposed to programming in block-based programming environments like Scratch. Identifying and measuring the previous experience of students learning to program is a key to improve the teaching of programming. In this contribution, ...
Scratch and Google Blockly: How Girls' Programming Skills and Attitudes are Influenced
Koli Calling '19: Proceedings of the 19th Koli Calling International Conference on Computing Education Research

Block-based programming has become popular to teach programming to young students in introductory programming environments. Nevertheless, in most western countries, girls show a lack of interest in computer science, including programming. This paper ...
Aiding the Transition from Novice to Traditional Programming Environments
IDC '17: Proceedings of the 2017 Conference on Interaction Design and Children

Novice programming environments are increasingly popular as introductory tools for teaching programming. Many of these tools differ significantly from traditional programming environments and are successful in motivating novices and making it simple to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

WiPSCE '19: Proceedings of the 14th Workshop in Primary and Secondary Computing Education

October 2019

127 pages

ISBN:9781450377041

DOI:10.1145/3361721

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

GI: Gesellschaft für Informatik e.V.
University of Glasgow: University of Glasgow

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WiPSCE'19

WiPSCE'19: 14th Workshop in Primary and Secondary Computing Education

October 23 - 25, 2019

Scotland, Glasgow, Uk

Acceptance Rates

WiPSCE '19 Paper Acceptance Rate 23 of 43 submissions, 53%;

Overall Acceptance Rate 104 of 279 submissions, 37%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
301
Total Downloads

Downloads (Last 12 months)38
Downloads (Last 6 weeks)4

Reflects downloads up to 17 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pozzan GPadova CMontuori CArfé BVardanega TMonga MLonati VBarendsen ESheard JPaterson J(2024)Experimental Analysis of First-Grade Students' Block-Based Programming Problem Solving ProcessesProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653586(143-149)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653586
Valle Torre MOertel CSpecht M(2024)The Sequence Matters in Learning - A Systematic Literature ReviewProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636880(263-272)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3636555.3636880
Greifenstein LBrune MFuchs THeuer UFraser G(2023)Impact of Hint Content on Performance and Learning: A Study with Primary School Children in a Scratch CourseProceedings of the 18th WiPSCE Conference on Primary and Secondary Computing Education Research10.1145/3605468.3605498(1-10)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3605468.3605498
Caspari LGreifenstein LHeuer UFraser GLaakso MMonga MSimon Sheard J(2023)ScratchLog: Live Learning Analytics for ScratchProceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 110.1145/3587102.3588836(403-409)Online publication date: 29-Jun-2023
https://dl.acm.org/doi/10.1145/3587102.3588836
Tsung SWei HLi HWang YXia MQu HKizilcec RDavis KOchoa X(2022)BlockLens: Visual Analytics of Student Coding Behaviors in Block-Based Programming EnvironmentsProceedings of the Ninth ACM Conference on Learning @ Scale10.1145/3491140.3528298(299-303)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1145/3491140.3528298
Zhang WZeng XWang JMing DLi P(2022)An analysis of learners’ programming skills through data miningEducation and Information Technologies10.1007/s10639-022-11079-427:8(11615-11633)Online publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1007/s10639-022-11079-4
Apiola MLipponen SSeitamaa AKorhonen THakkarainen K(2022)Learning Analytics for Knowledge Creation and Inventing in K-12: A Systematic ReviewIntelligent Computing10.1007/978-3-031-10467-1_15(238-257)Online publication date: 7-Jul-2022
https://doi.org/10.1007/978-3-031-10467-1_15
Mößlacher CKesselbacher MBollin A(2022)A Closer Look at and Confirmation of the General and Study Interests of Future Computer Science StudentsDigital Transformation of Education and Learning - Past, Present and Future10.1007/978-3-030-97986-7_8(91-103)Online publication date: 13-Mar-2022
https://doi.org/10.1007/978-3-030-97986-7_8
Lopez-Pernas SSaqr M(2021)Bringing Synchrony and Clarity to Complex Multi-Channel Data: A Learning Analytics Study in Programming EducationIEEE Access10.1109/ACCESS.2021.31348449(166531-166541)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3134844
Kesselbacher MWiltschnig KBollin A(2020)Block-based learning analytics repository and dashboardProceedings of the 15th Workshop on Primary and Secondary Computing Education10.1145/3421590.3421662(1-2)Online publication date: 28-Oct-2020
https://dl.acm.org/doi/10.1145/3421590.3421662

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents