Multi-Institutional Multi-National Studies of Parsons Problems

Students are often asked to learn programming by writing code from scratch. However, many novices struggle to write code and get frustrated when their code does not work. Parsons problems can reduce the difficulty of a coding problem by providing mixed-up blocks the learner rearranges into the correct order. These mixed-up blocks can include distractor blocks that are not needed in a correct solution. Distractor blocks can include common errors, which may help students learn to recognize and fix such errors. Evidence suggests students find Parsons problems engaging, useful for learning to program, and typically easier and faster to solve than writing code from scratch, but with equivalent learning gains. Most research on Parsons problems prior to this work has been conducted at a single institution. This work addresses the need for replication across multiple contexts.

A 2022 ITiCSE Parsons Problems Working Group conducted an extensive literature review of Parsons problems, designed several experimental studies for Parsons problems in Python, and created 'study-in-a-box' materials to help instructors run the experimental studies, but the 2022 working group had only sufficient time to pilot two of these studies.

Our 2023 ITiCSE Parsons Problems Working Group reviewed these studies, revised some of the studies, expanded both the programming and natural languages used in some of the studies, created new studies, conducted think-aloud observations on some of the studies, and ran both revised as well as new experimental studies. The think-aloud observations and experimental studies provide evidence for using Parsons problems to help students learn common algorithms such as swap, and the usefulness of distractors in helping students learn to recognize, fix, and avoid common errors. In addition, our 2023 ITiCSE Parsons Problems Working Group reviewed Parsons problem papers published after the 2022 literature review and provided a literature review of multi-national (MIMN) studies conducted in computer science education to better understand the motivations and challenges in performing such MIMN studies.

In summary, this article contributes an analysis of recent Parsons problem research papers, an itemization of considerations for MIMN studies, the results from our MIMN studies of Parsons problems, and a discussion of recent and future directions for MIMN studies of Parsons problems and more generally.