research-article

Where exactly are the difficulties in reasoning logically about code? experimentation with an online system

Authors:

Jason O. Hallstrom,

Joseph E. Hollingsworth,

Murali SitaramanAuthors Info & Claims

ITiCSE 2018: Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education

Pages 39 - 44

https://doi.org/10.1145/3197091.3197133

Published: 02 July 2018 Publication History

Abstract

CS students can typically reason about what a piece of code does on specific inputs. While this is a useful starting point, graduates must also be able to logically analyze, comprehend, and predict the behavior of their code in more general terms, no matter what the inputs are. Results of data collection and analysis from an online educational system show it can help to pinpoint the difficulties in doing this for individual students and groups, and to partition the groups in terms of their difficulties so that instructional interventions may be better targeted. Unlike traditional debugging, this online system helps reveal difficulties in reasoning in more general terms because it is equipped with a verification engine.

References

[1]

V. Aleven, B. M. McLaren, and J. Sewall. 2009. Scaling Up Programming by Demonstration for Intelligent Tutoring Systems Development: An Open-Access Web Site for Middle School Mathematics Learning. IEEE Trans. on Learning Technologies 2, 64-78.

Digital Library

[2]

B. S. Alqadi and J. I. Maletic. 2017. An Empirical Study of Debugging Patterns Among Novices Programmers. In Procs. SIGCSE ‘17, ACM, NY, USA, 15-20.

Digital Library

[3]

J. R. Anderson, A. T. Corbett, K. R. Koedinger, and R. Pelletier. 1995. Cognitive Tutors: Lessons Learned, The Journal of the Learning Sciences 4, 167-207.

[4]

R. Azevedo and R. M. Bernard. 1995. A Meta-Analysis of the Effects of Feedback in Computer-Based Instruction. Journal of Educational Computing Research 13, 2, 111 – 127.

[5]

N. Brown and A. Altadmri. 2014. Investigating novice programming mistakes: educator beliefs vs. student data. In Procs. ICER '14. ACM, NY, USA, 43-50.

Digital Library

[6]

P. Bhattacharya, L. Tao, B. W. Kai Qian, and E. K. Palmer. 2011. A Cloud-based Cyberlearning Environment for Introductory Computing Programming Education. In Procs. 11th ICALT, 12-13.

Digital Library

[7]

B. S. Bloom, et al. (Eds.) 1956. Taxonomy of Educational Objectives. The Classification of Educational Goals, Handbook I: Cognitive Domain. David McKay Company, Inc., NY, USA.

[8]

M. P. Cook. 2006. Visual representations in science education: The influence of prior knowledge and cognitive load theory on instructional design principles. Science Education 90, 6, 1073-1091.

[9]

C. T. Cook, H. Harton, H. Smith, and M. Sitaraman. 2012. Specification Engineering and Modular Verification Using a Web-Integrated Verifying Compiler. Proc. 34th ICSE, IEEE/ACM, NY, USA, 1379-1382.

Digital Library

[10]

C. T. Cook, S. Drachova, Y-S. Sun, M. Sitaraman, J. Carver, and J. E. Hollingsworth. 2013. Specification and Reasoning in SE Projects Using a Web-IDE. In Procs. 26th CSEE&T, IEEE, NY, USA.

[11]

D. De Bock, J. Deprej, W. van Duren, M. Roelens, and L. Verschaffel. 2011. Abstract or Concrete Examples in Learning Mathematics? A Replication an Elaboration of Kaminsky, Stoutsky, and Heckler’s Study. Journal for Research in Mathematics Education 42, 2, 109-126.

[12]

S. V. Drachova, J. O. Hallstrom, J. E. Hollingsworth, J. Krone, R. Pak, and M. Sitaraman. 2015. Teaching Mathematical Reasoning Principles for Software Correctness and Its Assessment. ACM Trans. Comput. Education 15, 3, 1-22.

Digital Library

[13]

K. A. Ericsson and H. A. Simon. 1998. How to study thinking in everyday life: Contrasting think-aloud protocols with descriptions and explanations of thinking. Mind, Culture, and Activity 5, 3, 178-186.

[14]

S. Fitzgerald, B. Simon, and L. Thomas. 2005. Strategies that students use to trace code: an analysis based in grounded theory. ICER '05 ACM, NY, 69-80.

Digital Library

[15]

K. Goldman, et al. 2008. Identifying important and difficult concepts in introductory computing courses using a Delphi process,” In Procs. 39th SIGCSE ’08, ACM, 256-260.

Digital Library

[16]

J. O. Hallstrom, C. Hochrine, J. Sorber, and M. Sitaraman. 2014. An ACM 2013 exemplar course integrating fundamentals, languages, and software engineering. In Procs. 45th SIGCSE '14. ACM, NY, USA, 211-216.

Digital Library

[17]

P. B. Henderson. 2003. Mathematical Reasoning in Software Engineering Education. Communications of the ACM 46, 9, 45-50.

Digital Library

[18]

C. D. Hundhausen, J. L. Brown, S. Farley, and D. Skarpas. 2006. A methodology for analyzing the temporal evolution of novice programs based on semantic components. In Procs. ICER '06. ACM, NY, USA, 59-71.

Digital Library

[19]

T. Kohn. 2017. Variable Evaluation: An Exploration of Novice Programmers' Understanding and Common Misconceptions. In Procs. ACM SIGCSE '17, NY, USA, 345-350.

Digital Library

[20]

Y. B-D. Kolikant. 2005. Students' alternative standards for correctness. In Procs. First ICER '05. ACM, NY, USA, 37-43.

Digital Library

[21]

R. Kumar, C. P. Rose, Y. Wang, M. Joshi, and A. Robinson. 2007. Tutorial Dialog as Adaptive Collaborative Learning Support. In Procs. Conference on Artificial Intelligence in Education: Building Technology Rich Contexts That Work, IOS Press, 383-390.

Digital Library

[22]

C. M. Lewis. 2014. Exploring variation in students' correct traces of linear recursion. In Procs. ICER '14. ACM, NY, USA, 67-74.

Digital Library

[23]

C. Li, Z. Dong, R. Untch, Chasteen, and N. Reale. 2011. PeerSpace—An Online Collaborative Learning Environment for Computer Science Students. In Procs. 11th ICALT, 409-411.

Digital Library

[24]

K. Martinez and M. Eisenhart. 2004. L.E.A.P. – Literature Review of Best Practices in College Physics and Best Practices for Women in College Physics, January 2004.

[25]

R. Moreno. 2004. Decreasing Cognitive Load for Novice Students: Effects of Explanatory versus Corrective Feedback in Discovery-Based Multimedia. Instructional Science 32, 99-113.

[26]

T. Price, Y. Dong, and D. Lipovac. 2017. iSnap: Towards Intelligent Tutoring in Novice Programming Environments. SIGCSE '17, ACM, NY, 483-488.

Digital Library

[27]

A. L. Santos. 2015. Collaborative course project for practicing componentbased software engineering, In Procs. 15th Koli Calling Conference on Computing Education Research, ACM, NY, 142-146.

Digital Library

[28]

M. Sitaraman, et al. 2011. Building a Push-Button RESOLVE Verifier: Progress and Challenges. Springer Formal Aspects of Computing 23, 607-626.

[29]

J. C. Spohrer, and E. Soloway. 1986. Novice mistakes: are the folk wisdoms correct? Communications of the ACM 29, 7, 624-632.

Digital Library

[30]

A. L. Strauss and J. Corbin. 1998. Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage Publications, Thousand Oaks, CA.

[31]

J. Sweller, P. Ayres, and S. Kalyuga, 2011. Cognitive load theory. Springer, NY,.

[32]

A. E. Tew. 2010. Assessing fundamental introductory computing concept knowledge in a language independent manner, Ph.D. Dissertation, Georgia Institute of Technology, 146 pages.

Digital Library

[33]

Web-CAT is an advanced automated grading system that can grade students on how well they test their own code. http://web-cat.org/home

[34]

J. B. Wiggins, et al. 2015. JavaTutor: An Intelligent Tutoring System that Adapts to Cognitive and Affective States during Computer Programming. In Procs. SIGCSE ’15, ACM, NY, USA, 599-599.

Digital Library

[35]

P. Wouters, F. Paas, and J. J. G. van Merriënboer. 2008. How to Optimize Learning from Animated Models: A Review of Guidelines Based on Cognitive Load. Review of Educational Research.

Cited By

Hurtig NHollingsworth JBlankenship SKraemer ESitaraman MHallstrom JBecker BQuille KLaakso MBarendsen ESimon (2022)Network Visualization and Assessment of Student Reasoning About ConditionalsProceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 110.1145/3502718.3524793(255-261)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3502718.3524793
Almazova NHallstrom JFowler MHollingsworth JKraemer ESitaraman MWashington G(2022)Automated Analysis of Student Verbalizations in Online Learning EnvironmentsEmerging Technologies for Education10.1007/978-3-030-92836-0_25(290-302)Online publication date: 28-Jan-2022
https://doi.org/10.1007/978-3-030-92836-0_25
Fowler MKraemer ESun YSitaraman MHallstrom JHollingsworth J(2020)Tool-Aided Assessment of Difficulties in Learning Formal Design-by-Contract AssertionsProceedings of the 4th European Conference on Software Engineering Education10.1145/3396802.3396807(52-60)Online publication date: 18-Jun-2020
https://dl.acm.org/doi/10.1145/3396802.3396807
Show More Cited By

Index Terms

Where exactly are the difficulties in reasoning logically about code? experimentation with an online system

Recommendations

Impact of Steps, Instruction, and Motivation on Learning Symbolic Reasoning Using an Online Tool
SIGCSE '19: Proceedings of the 50th ACM Technical Symposium on Computer Science Education

Several research studies have shown the benefits of code tracing to promote student understanding of program behavior. While code tracing on specific input values is a useful starting point, students ultimately need to be able to reason rigorously and ...
Tool-Aided Assessment of Difficulties in Learning Formal Design-by-Contract Assertions
ECSEE '20: Proceedings of the 4th European Conference on Software Engineering Education

Object-based development using design-by-contract (DbC) is broadly taught and practiced. Students must be able to read and write symbolic DbC assertions that are sufficiently precise and be able to use these assertions to trace program code. This paper ...
A study of nonmonotonic reasoning

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ITiCSE 2018: Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education

July 2018

394 pages

ISBN:9781450357074

DOI:10.1145/3197091

General Chairs:
Irene Polycarpou
UCLan Cyprus, Cyprus
,
Janet C. Read
UCLan, UK
,
Program Chairs:
Panayiotis Andreou
Cyprus University of Technology, Cyprus
,
Michal Armoni
Weizmann Institute of Science, Israel

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 July 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ITiCSE '18

Sponsor:

SIGCSE

ITiCSE '18: 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education

July 2 - 4, 2018

Larnaca, Cyprus

Acceptance Rates

Overall Acceptance Rate 552 of 1,613 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
252
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hurtig NHollingsworth JBlankenship SKraemer ESitaraman MHallstrom JBecker BQuille KLaakso MBarendsen ESimon (2022)Network Visualization and Assessment of Student Reasoning About ConditionalsProceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 110.1145/3502718.3524793(255-261)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3502718.3524793
Almazova NHallstrom JFowler MHollingsworth JKraemer ESitaraman MWashington G(2022)Automated Analysis of Student Verbalizations in Online Learning EnvironmentsEmerging Technologies for Education10.1007/978-3-030-92836-0_25(290-302)Online publication date: 28-Jan-2022
https://doi.org/10.1007/978-3-030-92836-0_25
Fowler MKraemer ESun YSitaraman MHallstrom JHollingsworth J(2020)Tool-Aided Assessment of Difficulties in Learning Formal Design-by-Contract AssertionsProceedings of the 4th European Conference on Software Engineering Education10.1145/3396802.3396807(52-60)Online publication date: 18-Jun-2020
https://dl.acm.org/doi/10.1145/3396802.3396807
Hollingsworth JKraemer ESitaraman MHawthorne EPérez-Quiñones MHeckman SZhang J(2019)Engaging in Logical Code Reasoning with an Activity-Based Online ToolProceedings of the 50th ACM Technical Symposium on Computer Science Education10.1145/3287324.3293754(1289-1289)Online publication date: 22-Feb-2019
https://dl.acm.org/doi/10.1145/3287324.3293754
Fowler MCook MPlis KSchwab TSun YSitaraman MHallstrom JHollingsworth JHawthorne EPérez-Quiñones MHeckman SZhang J(2019)Impact of Steps, Instruction, and Motivation on Learning Symbolic Reasoning Using an Online ToolProceedings of the 50th ACM Technical Symposium on Computer Science Education10.1145/3287324.3287401(1039-1045)Online publication date: 22-Feb-2019
https://dl.acm.org/doi/10.1145/3287324.3287401
Kennedy CKraemer EJoy MIhantola P(2018)What Are They Thinking?Proceedings of the 18th Koli Calling International Conference on Computing Education Research10.1145/3279720.3279728(1-10)Online publication date: 22-Nov-2018
https://dl.acm.org/doi/10.1145/3279720.3279728

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents