research-article

CATNIP: An Automated Hint Generation Tool for Scratch

Authors:

Florian Obermüller,

Gordon FraserAuthors Info & Claims

ITiCSE '22: Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1

Pages 124 - 130

https://doi.org/10.1145/3502718.3524820

Published: 07 July 2022 Publication History

Abstract

Taking the first steps when learning how to program can be hard. Block-based programming languages like Scratch lower this hurdle, but learners may nevertheless get stuck when trying to solve a specific task and need help. This can also challenge teachers when facing many raised hands at the same time in the classroom. Consequently, it is desirable for learners and teachers alike to have access to systems that automatically generate hints on which steps to take next in a programming assignment. In this paper we introduce Catnip, a tool that generates next step hints for the Scratch programming language based on a structural comparison between model solutions and the current student attempt. Catnip uses extensive postprocessing to improve the generated hints, and displays them directly inside the Scratch framework, suggesting where to add or reorder blocks while working on a programming task.

References

[1]

Paolo Antonucci, Christian Estler, Durica Nikoliç, Marco Piccioni, and Bertrand Meyer. 2015. An Incremental Hint System For Automated Programming Assignments. In Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education (Vilnius, Lithuania) (ITiCSE '15). Association for Computing Machinery, New York, NY, USA, 320--325. https://doi.org/10.1145/2729094.2742607

Digital Library

[2]

Nikolaus Augsten, Michael Böhlen, and Johann Gamper. 2005. Approximate Matching of Hierarchical Data Using pq-Grams. In Proceedings of the 31st International Conference on Very Large Data Bases (Trondheim, Norway) (VLDB '05). VLDB Endowment, 301--312.

Digital Library

[3]

Bryce Boe, Charlotte Hill, Michelle Len, Greg Dreschler, Phillip Conrad, and Diana Franklin. 2013. Hairball: Lint-inspired static analysis of scratch projects. SIGCSE 2013 - Proceedings of the 44th ACM Technical Symposium on Computer Science Education, 215--220. https://doi.org/10.1145/2445196.2445265

Digital Library

[4]

Allan Collins and Manu Kapur. 2014. Cognitive Apprenticeship 2 ed.). Cambridge University Press, 109--127. https://doi.org/10.1017/CBO9781139519526.008

[5]

Gordon Fraser, Ute Heuer, Nina Körber, Florian Obermüller, and Ewald Wasmeier. 2021. LitterBox: A Linter for Scratch Programs. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET). 183--188. https://doi.org/10.1109/ICSE-SEET52601.2021.00028

Digital Library

[6]

Dan Garcia, Brian Harvey, and Tiffany Barnes. 2015. The Beauty and Joy of Computing. ACM Inroads, Vol. 6, 4 (Nov. 2015), 71--79. https://doi.org/10.1145/2835184

Digital Library

[7]

Sumit Gulwani, Ivan Radivcek, and Florian Zuleger. 2018. Automated Clustering and Program Repair for Introductory Programming Assignments. SIGPLAN Not., Vol. 53, 4 (jun 2018), 465--480. https://doi.org/10.1145/3296979.3192387

Digital Library

[8]

Stuart Hansen and Erica Eddy. 2007. Engagement and Frustration in Programming Projects. In Proceedings of the 38th SIGCSE Technical Symposium on Computer Science Education (Covington, Kentucky, USA) (SIGCSE '07). Association for Computing Machinery, New York, NY, USA, 271--275. https://doi.org/10.1145/1227310.1227407

Digital Library

[9]

John Maloney, Mitchel Resnick, Natalie Rusk, Brian Silverman, and Evelyn Eastmond. 2010. The Scratch Programming Language and Environment. ACM Transactions on Computing Education (TOCE), Vol. 10 (11 2010), 16. https://doi.org/10.1145/1868358.1868363

Digital Library

[10]

Monica M. McGill and Adrienne Decker. 2020. Tools, Languages, and Environments Used in Primary and Secondary Computing Education. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education (Trondheim, Norway) (ITiCSE '20). Association for Computing Machinery, New York, NY, USA, 103--109. https://doi.org/10.1145/3341525.3387365

Digital Library

[11]

James Munkres. 1957. Algorithms for the Assignment and Transportation Problems. J. Soc. Indust. Appl. Math., Vol. 5, 1 (March 1957), 32--38. https://doi.org/10.1137/0105003

[12]

Florian Obermüller, Lena Bloch, Luisa Greifenstein, Ute Heuer, and Gordon Fraser. 2021 a. Code Perfumes: Reporting Good Code to Encourage Learners .Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3481312.3481346

Digital Library

[13]

Florian Obermüller, Ute Heuer, and Gordon Fraser. 2021 b. Guiding Next-Step Hint Generation Using Automated Tests .Association for Computing Machinery, New York, NY, USA, 220--226. https://doi.org/10.1145/3430665.3456344

Digital Library

[14]

Thomas Price, Rui Zhi, and Tiffany Barnes. 2017b. Evaluation of a Data-driven Feedback Algorithm for Open-ended Programming.

[15]

Thomas W. Price, Yihuan Dong, and Tiffany Barnes. 2016. Generating Data-driven Hints for Open-ended Programming. In EDM .

[16]

Thomas W. Price, Yihuan Dong, and Dragan Lipovac. 2017a. ISnap: Towards Intelligent Tutoring in Novice Programming Environments. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (Seattle, Washington, USA) (SIGCSE '17). Association for Computing Machinery, New York, NY, USA, 483--488. https://doi.org/10.1145/3017680.3017762

Digital Library

[17]

Thomas W. Price, Rui Zhi, and Tiffany Barnes. 2017c. Hint Generation Under Uncertainty: The Effect of Hint Quality on Help-Seeking Behavior. In Artificial Intelligence in Education, Elisabeth André, Ryan Baker, Xiangen Hu, Ma. Mercedes T. Rodrigo, and Benedict du Boulay (Eds.). Springer International Publishing, Cham, 311--322.

[18]

Brian J. Reiser and Iris Tabak. 2014. Scaffolding 2 ed.). Cambridge University Press, 44--62. https://doi.org/10.1017/CBO9781139519526.005

[19]

K Ann Renninger. 2010. Working with and cultivating the development of interest, self-efficacy, and self-regulation. Innovations in Educational Psychology (2010), 107.

[20]

Andreas Stahlbauer, Marvin Kreis, and Gordon Fraser. 2019. Testing scratch programs automatically. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 165--175.

Digital Library

[21]

Peeratham Techapalokul and Eli Tilevich. 2017. Quality Hound - An online code smell analyzer for scratch programs. In 2017 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 337--338. https://doi.org/10.1109/VLHCC.2017.8103498

[22]

Jooyong Yi, Umair Z. Ahmed, Amey Karkare, Shin Hwei Tan, and Abhik Roychoudhury. 2017. A Feasibility Study of Using Automated Program Repair for Introductory Programming Assignments. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (Paderborn, Germany) (ESEC/FSE 2017). Association for Computing Machinery, New York, NY, USA, 740--751. https://doi.org/10.1145/3106237.3106262

Digital Library

[23]

K. Zimmerman and C. R. Rupakheti. 2015. An Automated Framework for Recommending Program Elements to Novices (N). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) . 283--288.

Cited By

Feldmeier PFraser GHeuer UObermüller FSteckenbiller S(2024)A Block-Based Testing Framework for ScratchProceedings of the 24th Koli Calling International Conference on Computing Education Research10.1145/3699538.3699547(1-12)Online publication date: 12-Nov-2024
https://dl.acm.org/doi/10.1145/3699538.3699547
Akbar UAkbar SAli MUllah RKhan RLopes I(2024)Extending Scratch Framework to Improve the Analytical Skills of Undergraduate StudentsProceedings of the 2024 10th International Conference on Computer Technology Applications10.1145/3674558.3674561(15-19)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3674558.3674561
Deiner AFraser GRoychoudhury APaiva AAbreu RStorey M(2024)NuzzleBug: Debugging Block-Based Programs in ScratchProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3623331(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3623331

Index Terms

CATNIP: An Automated Hint Generation Tool for Scratch
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Software engineering education
      2. K-12 education
2. Software and its engineering
  1. Software notations and tools
    1. Context specific languages
      1. Visual languages

Recommendations

Effects of Automated Feedback in Scratch Programming Tutorials
ITiCSE 2023: Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1

Block-based programming languages like Scratch are commonly used to introduce young learners to programming. While coding, learners may encounter problems, which may require teachers to intervene. However, teachers may be overwhelmed with help requests ...
Guiding Next-Step Hint Generation Using Automated Tests
ITiCSE '21: Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1

Learning basic programming with Scratch can be hard for novices and tutors alike: Students may not know how to advance when solving a task, teachers may face classrooms with many raised hands at a time, and the problem is exacerbated when novices are on ...
ScratchLog: Live Learning Analytics for Scratch
ITiCSE 2023: Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1

Scratch is a hugely popular block-based programming environment that is often used in educational settings, and has therefore recently also become a focus for research on programming education. Scratch provides dedicated teacher accounts that make it ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ITiCSE '22: Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1

July 2022

686 pages

ISBN:9781450392013

DOI:10.1145/3502718

General Chairs:
Brett A. Becker
University College Dublin, Ireland
,
Keith Quille
Technological University Dublin, Ireland
,
Mikko-Jussi Laakso
University of Turku, Finland
,
Program Chairs:
Erik Barendsen
Radboud University & Open University, The Netherlands
,
Simon
Unaffiliated, Australia

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Deutsche Forschungsgemeinschaft

Conference

ITiCSE 2022

Sponsor:

SIGCSE

ITiCSE 2022: Innovation and Technology in Computer Science Education

July 8 - 13, 2022

Dublin, Ireland

Acceptance Rates

Overall Acceptance Rate 552 of 1,613 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
181
Total Downloads

Downloads (Last 12 months)46
Downloads (Last 6 weeks)8

Reflects downloads up to 08 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Feldmeier PFraser GHeuer UObermüller FSteckenbiller S(2024)A Block-Based Testing Framework for ScratchProceedings of the 24th Koli Calling International Conference on Computing Education Research10.1145/3699538.3699547(1-12)Online publication date: 12-Nov-2024
https://dl.acm.org/doi/10.1145/3699538.3699547
Akbar UAkbar SAli MUllah RKhan RLopes I(2024)Extending Scratch Framework to Improve the Analytical Skills of Undergraduate StudentsProceedings of the 2024 10th International Conference on Computer Technology Applications10.1145/3674558.3674561(15-19)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3674558.3674561
Deiner AFraser GRoychoudhury APaiva AAbreu RStorey M(2024)NuzzleBug: Debugging Block-Based Programs in ScratchProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3623331(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3623331

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents