research-article

Investigating Programming Students Problem Comprehension Ability and its Association With Learning Performance

Authors:

Leonardo Silva,

António José Mendes,

Gabriel FortesAuthors Info & Claims

IEEE Transactions on Education, Volume 66, Issue 2

Pages 156 - 162

https://doi.org/10.1109/TE.2022.3204906

Published: 01 April 2023 Publication History

Abstract

Contribution: Students’ problem-understanding abilities and their relationship with programming learning were investigated using a methodology little explored in the existing literature. Background: Problem comprehension is an ability used during software development. Current research points to conflicting results on students’ ability to interpret problems, which calls for further research. In addition, the influence of this skill in programming learning also deserves additional studies. Research Questions: The following research questions were developed: RQ1) Can introductory programming students correctly interpret the statement of programming questions? RQ2) Is the student’s problem comprehension ability associated with creating correct programs? Methodology: Forty-eight students enrolled in an online introductory programming course participated in the investigation. Students’ problem-understanding externalizations were analyzed, and statistical tests were performed to assess the association of this ability with programming learning performance. Findings: Most students externalized a satisfactory degree of problem-understanding competency. The cases in which students’ misinterpreted the problem were associated with creating faulty software. However, a closer analysis suggests that there are other factors that must also be considered.

References

[1]

A. V. Robins, “Novice programmers and introductory programming,” in Cambridge Handbook of Computing Education Research (Cambridge Handbooks Psychology). Cambridge, U.K.: Cambridge Univ. Press, 2019, pp. 327–376.

[2]

A. Gomes and A. J. Mendes, “Problem solving in programming,” in Proc. PPIG, 2007, p. 18.

[3]

D. Loksa and A. J. Ko, “The role of self-regulation in programming problem solving process and success,” in Proc. ACM Conf. Int. Comput. Educ. Res., 2016, pp. 83–91.

[4]

C. Cabo, “Fostering problem understanding as a precursor to problem-solving in computer programming,” in Proc. IEEE Front. Educ. Conf. (FIE), 2019, pp. 1–9.

[5]

D. Loksa, B. Xie, H. Kwik, and A. J. Ko, “Investigating novices’ in situ reflections on their programming process,” in Proc. 51st ACM Tech. Symp. Comput. Sci. Educ., 2020, pp. 149–155.

[6]

P. Denny, J. Prather, B. A. Becker, Z. Albrecht, D. Loksa, and R. Pettit, “A closer look at metacognitive scaffolding: Solving test cases before programming,” in Proc. 19th Koli Calling Int. Conf. Comput. Educ. Res., 2019, pp. 1–10.

[7]

J. Finnie-Ansley, P. Denny, and A. Luxton-Reilly, “A semblance of similarity: Student categorisation of simple algorithmic problem statements,” in Proc. 17th ACM Conf. Int. Comput. Educ. Res., 2021, pp. 198–212.

[8]

D. Loksa, A. J. Ko, W. Jernigan, A. Oleson, C. J. Mendez, and M. M. Burnett, “Programming, problem solving, and self-awareness: Effects of explicit guidance,” in Proc. CHI Conf. Human Factors Comput. Syst., 2016, pp. 1449–1461.

[9]

A. A. Lawan, A. S. Abdi, A. A. Abuhassan, and M. S. Khalid, “What is difficult in learning programming language based on problem-solving skills?” in Proc. Int. Conf. Adv. Sci. Eng. (ICOASE), 2019, pp. 18–22.

[10]

G. Schellings and B. Van Hout-Wolters, “Measuring strategy use with self-report instruments: Theoretical and empirical considerations,” Metacogn. Learn., vol. 6, no. 2, pp. 83–90, 2011.

[11]

M. Nascimento, E. Araujo, D. Serey, and J. Figueiredo, “The role of source code vocabulary in programming teaching and learning,” in Proc. IEEE Front. Educ. Conf. (FIE), Oct. 2020, pp. 1–8.

[12]

E. Thompson, A. Luxton-Reilly, J. L. Whalley, M. Hu, and P. Robbins, “Bloom’s taxonomy for CS assessment,” in Proc. 10th Conf. Aust. Comput. Educ., vol. 78, 2008, pp. 155–161.

[13]

M. C. Parker, M. Guzdial, and S. Engleman, “Replication, validation, and use of a language independent CS1 knowledge assessment,” in Proc. ACM Conf. Int. Comput. Educ. Res., 2016, pp. 93–101.

[14]

G. Polya, How to Solve It: A New Aspect of Mathematical Method. Princeton, NJ, USA: Princeton Univ. Press, 2004.

[15]

M. L. McHugh, “Interrater reliability: The kappa statistic,” Biochemia Medica, vol. 22, no. 3, pp. 276–282, 2012.

[16]

A. Field, Discovering Statistics Using IBM SPSS Statistics. London, U.K.: SAGE, 2013.

Digital Library

[17]

S. Sun, W. Pan, and L. L. Wang, “A comprehensive review of effect size reporting and interpreting practices in academic journals in education and psychology,” J. Educ. Psychol., vol. 102, no. 4, p. 989, 2010.

[18]

J. Cohen, Statistical Power Analysis for the Behavioral Sciences. New York, NY, USA: Psychol. Press, 2013.

[19]

T. J. McGill and S. E. Volet, “A conceptual framework for analyzing students’ knowledge of programming,” J. Res. Comput. Educ., vol. 29, no. 3, pp. 276–297, 1997.

[20]

K. Arakawa, Q. Hao, T. Greer, L. Ding, C. D. Hundhausen, and A. Peterson, “In situ identification of student self-regulated learning struggles in programming assignments,” in Proc. 52nd ACM Tech. Symp. Comput. Sci. Educ., 2021, pp. 467–473.

[21]

A. K. Veerasamy, D. D’Souza, R. Lindén, and M.-J. Laakso, “Relationship between perceived problem-solving skills and academic performance of novice learners in introductory programming courses,” J. Comput. Assist. Learn., vol. 35, no. 2, pp. 246–255, 2019.

[22]

M. McCrackenet al., “A multi-national, multi-institutional study of assessment of programming skills of first-year CS students,” in Proc. Working Group Rep. ITiCSE Innov. Technol. Comput. Sci. Educ., 2001, pp. 125–180.

[23]

E. Soloway, “Learning to program = learning to construct mechanisms and explanations,” Commun. ACM, vol. 29, no. 9, pp. 850–858, 1986.

Digital Library

[24]

B. Breed, E. Mentz, and G. Van der Westhuizen, “A metacognitive approach to pair programming: Influence on metacognitive awareness,” Electron. J. Res. Educ. Psychol., vol. 12, no. 1, pp. 33–60, 2014.

[25]

K. Bielaczyc, P. L. Pirolli, and A. L. Brown, “Training in self-explanation and self-regulation strategies: Investigating the effects of knowledge acquisition activities on problem solving,” Cogn. Instruct., vol. 13, no. 2, pp. 221–252, 1995.

[26]

A. Lishinski, A. Yadav, R. Enbody, and J. Good, “The influence of problem solving abilities on students’ performance on different assessment tasks in CS1,” in Proc. 47th ACM Tech. Symp. Comput. Sci. Educ., 2016, pp. 329–334.

[27]

A. Raabe, A. S. Zanini, A. L. M. Santana, and M. F. V. Vieira, “Influência dos enunciados na resolução de problemas de programação introdutória,” Revista Brasileira de Informática na Educação, vol. 24, no. 1, p. 66, 2016.

[28]

D. Bouvieret al., “Novice programmers and the problem description effect,” in Proc. ITiCSE Working Group Reports, 2016, pp. 103–118.

[29]

T. Beaubouef, R. Lucas, and J. Howatt, “The UNLOCK system: Enhancing problem solving skills in CS-1 students,” ACM SIGCSE Bull., vol. 33, no. 2, pp. 43–46, 2001.

Digital Library

[30]

Y. Qian and J. Lehman, “Students’ misconceptions and other difficulties in introductory programming: A literature review,” ACM Trans. Comput. Educ., vol. 18, no. 1, pp. 1–24, 2017.

[31]

S. E. Volet and C. P. Lund, “Metacognitive instruction in introductory computer programming: A better explanatory construct for performance than traditional factors,” J. Educ. Comput. Res., vol. 10, no. 4, pp. 297–328, Jun. 1994. [Online]. Available: https://doi.org/10.2190/9a08-y2q0-6aer-6klq

[32]

J. Leinonen, P. Denny, and J. Whalley, “Exploring the effects of Contextualized problem descriptions on problem solving,” in Proc. Aust. Comput. Educ. Conf., 2021, pp. 30–39.

[33]

A. D. Hilton, G. M. Lipp, and S. H. Rodger, “Translation from problem to code in seven steps,” in Proc. ACM Conf. Global Comput. Educ., 2019, pp. 78–84.

[34]

L. Silva, A. J. Mendes, A. Gomes, and G. F. Cavalcanti de Macêdo, “Regulation of learning interventions in programming education: A systematic literature review and guideline proposition,” in Proc. 52nd ACM Tech. Symp. Comput. Sci. Educ., 2021, pp. 647–653.

Cited By

Lovellette EBouvier DMatta J(2024)Contextualization, Authenticity, and the Problem Description EffectACM Transactions on Computing Education10.1145/364386424:2(1-32)Online publication date: 5-Feb-2024
https://dl.acm.org/doi/10.1145/3643864
Joseph ASarma SShelly Kumar V(2023)Bug-eecha 2.0: An Educational Game for CS1 Students and InstructorsProceedings of the 16th Annual ACM India Compute Conference10.1145/3627217.3627236(61-65)Online publication date: 9-Dec-2023
https://dl.acm.org/doi/10.1145/3627217.3627236
Pawagi MKumar V(2023)GuardRails: Automated Suggestions for Clarifying Ambiguous Purpose StatementsProceedings of the 16th Annual ACM India Compute Conference10.1145/3627217.3627234(55-60)Online publication date: 9-Dec-2023
https://dl.acm.org/doi/10.1145/3627217.3627234

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Education

IEEE Transactions on Education Volume 66, Issue 2

April 2023

93 pages

ISSN:0018-9359

Issue’s Table of Contents

1557-9638 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 April 2023

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lovellette EBouvier DMatta J(2024)Contextualization, Authenticity, and the Problem Description EffectACM Transactions on Computing Education10.1145/364386424:2(1-32)Online publication date: 5-Feb-2024
https://dl.acm.org/doi/10.1145/3643864
Joseph ASarma SShelly Kumar V(2023)Bug-eecha 2.0: An Educational Game for CS1 Students and InstructorsProceedings of the 16th Annual ACM India Compute Conference10.1145/3627217.3627236(61-65)Online publication date: 9-Dec-2023
https://dl.acm.org/doi/10.1145/3627217.3627236
Pawagi MKumar V(2023)GuardRails: Automated Suggestions for Clarifying Ambiguous Purpose StatementsProceedings of the 16th Annual ACM India Compute Conference10.1145/3627217.3627234(55-60)Online publication date: 9-Dec-2023
https://dl.acm.org/doi/10.1145/3627217.3627234

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents