research-article

Uncovering reviewing and reflecting behaviors from paper-based formal assessment

Authors:

Hannah MurphyAuthors Info & Claims

LAK '17: Proceedings of the Seventh International Learning Analytics & Knowledge Conference

Pages 319 - 328

https://doi.org/10.1145/3027385.3027415

Published: 13 March 2017 Publication History

Abstract

In this paper, we study students' learning effectiveness through their use of a homegrown educational technology, Web Programming Grading Assistant (WPGA), which facilitates grading and feedback delivery of paper-based assessments. We designed a classroom study and collected data from a lower-division blended-instruction computer science class. We tracked and modeled students' reviewing and reflecting behaviors from WPGA. The results show that students demonstrated an effort and desire to review assessments regardless of if they were graded for academic performance or for attendance. Hardworking students achieved higher exam scores on average and were found to review their exams and the correct questions frequently. Additionally, student cohorts exhibited similar initial reviewing patterns, but different in-depth reviewing and reflecting strategies. Ultimately, this work contributes to the aggregation of multidimensional learning analytics across the physical and cybersphere.

References

[1]

Trees, A.R. and M.H. Jackson, (2007) The learning environment in clicker classrooms: student processes of learning and involvement in large university level courses using student response systems. Learning, Media and Technology. 32(1): p. 21--40.

[2]

Martinez-Maldonado, R., Dimitriadis, Y., Martinez-Monés, A., Kay, J., & Yacef, K., (2013) Capturing and analyzing verbal and physical collaborative learning interactions at an enriched interactive tabletop. International Journal of Computer-Supported Collaborative Learning, 8(4): p. 455--485.

[3]

R. Martinez-Maldonado, D. Suthers, N. R. Aljohani, D. Hernandez-Leo, K. Kitto, A. Pardo, S. Charleer, and H. Ogata. (2016) Cross-LAK: learning analytics across physical and digital spaces. in Proceedings of the Sixth International Conference on Learning Analytics & Knowledge.

Digital Library

[4]

Hsiao, I. H. (2016). Mobile Grading Paper-Based Programming Exams: Automatic Semantic Partial Credit Assignment Approach. In European Conference on Technology Enhanced Learning (pp. 110--123). Springer International Publishing.

[5]

Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C., & Norman, M. K., (2010) How learning works: Seven research-based principles for smart teaching. John Wiley & Sons.

[6]

Hattie, J. and H. Timperley, (2007) The power of feedback. Review of educational research. 77(1): p. 81--112.

[7]

Cutumisu, M. and D.L. Schwartz. (2016) Choosing versus Receiving Feedback: The Impact of Feedback Valence on Learning in an Assessment Game. in The 9th International Conference on Educational Data Mining. (pp. 341--346).

[8]

Kulkarni, C. E., Bernstein, M. S., & Klemmer, S. R. (2015). PeerStudio: rapid peer feedback emphasizes revision and improves performance. In Proceedings of the Second (2015) ACM Conference on Learning@ Scale (pp. 75--84). ACM.

Digital Library

[9]

Dihoff, R. E., Brosvic, G. M., Epstein, M. L., & Cook, M. J., (2004) Provision of feedback during preparation for academic testing: Learning is enhanced by immediate but not delayed feedback. The Psychological Record. 54(2): p. 207.

[10]

Tempelaar, D.T., B. Rienties, and B. Giesbers, (2015) In search for the most informative data for feedback generation: Learning Analytics in a data-rich context. Computers in Human Behavior. 47: p. 157--167.

Digital Library

[11]

Butler, D.L. and P.H. Winne, (1995) Feedback and self-regulated learning: A theoretical synthesis. Review of educational research. 65(3): p. 245--281.

[12]

Bjork, R.A., J. Dunlosky, and N. Kornell, (2013) Self-regulated learning: Beliefs, techniques, and illusions. Annual review of psychology. 64: p. 417--444.

[13]

Zimmerman, B.J. and D.H. Schunk, (2012) Self-regulated learning and academic achievement: Theory, research, and practice. Springer Science & Business Media.

[14]

Ertmer, P.A. and T.J. Newby, (1996) The expert learner: Strategic, self-regulated, and reflective. Instructional science. 24(1): p. 1--24.

[15]

de Bruin, A.B. & T. van Gog, (2012) Improving self-monitoring and self-regulation: From cognitive psychology to the classroom. Learning and Instruction. 22(4): p. 245--252.

[16]

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P., (2014) Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences. 111(23): p. 8410--8415.

[17]

Edwards, S. H., & Perez-Quinones, M. A. (2008). Web-CAT: automatically grading programming assignments. In ACM SIGCSE Bulletin (Vol. 40, No. 3, pp. 328--328). ACM.

Digital Library

[18]

Jackson, D., & Usher, M. (1997). Grading student programs using ASSYST. In ACM SIGCSE Bulletin (Vol. 29, No. 1, pp. 335--339). ACM.

Digital Library

[19]

Bloomfield, A., & Groves, J. F. (2008). A tablet-based paper exam grading system. In ACM SIGCSE Bulletin (Vol. 40, No. 3, pp. 83--87). ACM.

Digital Library

[20]

Titus, A.P., L.W. Martin, and R.J. Beichner, (1998) Web-based testing in physics education: Methods and opportunities. Computers in Physics. 12(2): p. 117--123.

Digital Library

[21]

Hsiao, I.-H., S. Sosnovsky, and P. Brusilovsky, (2010) Guiding students to the right questions: adaptive navigation support in an E-Learning system for Java programming. Journal of Computer Assisted Learning. 26(4): p. 270--283.

[22]

Gehringer, E. F. (2001). Electronic peer review and peer grading in computer-science courses. ACM SIGCSE Bulletin, 33(1), 139--143.

Digital Library

[23]

Denny, P., Luxton-Reilly, A., & Hamer, J. (2008, June). Student use of the PeerWise system. In ACM SIGCSE Bulletin (Vol. 40, No. 3, pp. 73--77). ACM.

Digital Library

[24]

Hsiao, I.-H., S.K.P. Govindarajan, and Y.-L. Lin. (2016) Semantic Visual Analytics for Today's Programming Classrooms. in The 6th international Learning Analytics & Knowledge Conference. Edinburgh, UK: ACM.

Digital Library

[25]

Chi, M.T., (2000) Self-explaining expository texts: The dual processes of generating inferences and repairing mental models. Advances in instructional psychology. 5: p. 161--238.

[26]

Roscoe, R.D. and M.T. Chi, (2007) Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors' explanations and questions. Review of Educational Research. 77(4): p. 534--574.

[27]

Han, S., Z. Yue, and D. He. (2013) Automatic detection of search tactic in individual information seeking: A hidden Markov model approach. in iConference. arXiv preprint arXiv:1304.1924.

[28]

Lu, Y. and I.-H. Hsiao. (2016) Seeking Programming-related Information from Large Scaled Discussion Forums, Help or Harm? The 9th International Conference on Educational Data Mining, EDM. NCSU. p.442--447.

[29]

Piech, C., Sahami, M., Koller, D., Cooper, S., & Blikstein, P., (2012) Modeling how students learn to program, in Proceedings of the 43rd ACM technical symposium on Computer Science Education, ACM: Raleigh, North Carolina, USA. p. 153--160.

Digital Library

[30]

Baum, L. E., Petrie, T., Soules, G., & Weiss, N., (1970) A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. The annals of mathematical statistics. 41(1): p. 164--171.

[31]

Ketchen, D.J. and C.L. Shook, (1996) The application of cluster analysis in strategic management research: an analysis and critique. Strategic management journal. 17(6): p. 441--458.

[32]

Dillenbourg, P., (2013) Design for classroom orchestration. Computers & Education. 69: p. 485--492.

Digital Library

Cited By

Silva LMendes AGomes AFortes G(2023)What Learning Strategies are Used by Programming Students? A Qualitative Study Grounded on the Self-regulation of Learning TheoryACM Transactions on Computing Education10.1145/363572024:1(1-26)Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1145/3635720
Silva LMendes AGomes ACavalcanti de Macêdo GSherriff MMerkle LCutter PMonge ASheard J(2021)Regulation of Learning Interventions in Programming EducationProceedings of the 52nd ACM Technical Symposium on Computer Science Education10.1145/3408877.3432363(647-653)Online publication date: 3-Mar-2021
https://dl.acm.org/doi/10.1145/3408877.3432363
Leite WJing ZKuang HKim DHuggins-Manley A(2021)Multilevel Mixture Modeling with Propensity Score Weights for Quasi-Experimental Evaluation of Virtual Learning EnvironmentsStructural Equation Modeling: A Multidisciplinary Journal10.1080/10705511.2021.1919895(1-19)Online publication date: 9-Jun-2021
https://doi.org/10.1080/10705511.2021.1919895
Show More Cited By

Index Terms

Uncovering reviewing and reflecting behaviors from paper-based formal assessment
1. Applied computing
  1. Education

Recommendations

Enhance Students' Computing Skills via Web-Mediated Self-Regulated Learning with Feedback in Blended Environment

The vocational colleges in Taiwan regard professional certifications as a badge of skills achievement. To enhance student learning in this specific context, the authors conducted a quasi-experiment to explore effects of web-mediated self-regulated ...
Understanding the Effect of In-Video Prompting on Learners and Instructors
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Online instructional videos are ubiquitous, but it is difficult for instructors to gauge learners' experience and their level of comprehension or confusion regarding the lecture video. Moreover, learners watching the videos may become disengaged or fail ...
Interactive Learning: Increasing Student Participation through Shorter Exercise Cycles
ACE '17: Proceedings of the Nineteenth Australasian Computing Education Conference

In large classes, there is typically a clear separation between content delivery in lectures on the one hand and content deepening in practical exercises on the other hand. This temporal and spatial separation has several disadvantages. In particular, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

LAK '17: Proceedings of the Seventh International Learning Analytics & Knowledge Conference

March 2017

631 pages

ISBN:9781450348706

DOI:10.1145/3027385

General Chairs:
Alyssa Wise
New York University, USA
,
Phillip H. Winne
Simon Fraser University, Canada
,
Grace Lynch
Society for Learning Analytics Research
,
Program Chairs:
Xavier Ochoa
Escuela Superior Politecnica del Litoral, Ecuador
,
Inge Molenaar
Radboud University, The Netherlands
,
Shane Dawson
University of South Australia, Australia
,
Publications Chair:
Marek Hatala
Simon Fraser University, Canada

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 March 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

LAK '17

LAK '17: 7th International Learning Analytics and Knowledge Conference

March 13 - 17, 2017

British Columbia, Vancouver, Canada

Acceptance Rates

LAK '17 Paper Acceptance Rate 36 of 114 submissions, 32%;

Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
326
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)2

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Silva LMendes AGomes AFortes G(2023)What Learning Strategies are Used by Programming Students? A Qualitative Study Grounded on the Self-regulation of Learning TheoryACM Transactions on Computing Education10.1145/363572024:1(1-26)Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1145/3635720
Silva LMendes AGomes ACavalcanti de Macêdo GSherriff MMerkle LCutter PMonge ASheard J(2021)Regulation of Learning Interventions in Programming EducationProceedings of the 52nd ACM Technical Symposium on Computer Science Education10.1145/3408877.3432363(647-653)Online publication date: 3-Mar-2021
https://dl.acm.org/doi/10.1145/3408877.3432363
Leite WJing ZKuang HKim DHuggins-Manley A(2021)Multilevel Mixture Modeling with Propensity Score Weights for Quasi-Experimental Evaluation of Virtual Learning EnvironmentsStructural Equation Modeling: A Multidisciplinary Journal10.1080/10705511.2021.1919895(1-19)Online publication date: 9-Jun-2021
https://doi.org/10.1080/10705511.2021.1919895
Chung CHsiao IZhang JSherriff MHeckman SCutter PMonge A(2020)Investigating Patterns of Study Persistence on Self-Assessment Platform of Programming Problem-SolvingProceedings of the 51st ACM Technical Symposium on Computer Science Education10.1145/3328778.3366827(162-168)Online publication date: 26-Feb-2020
https://dl.acm.org/doi/10.1145/3328778.3366827
Paredes YHuang PHsiao I(2019)Utilising behavioural analytics in a blended programming learning environmentNew Review of Hypermedia and Multimedia10.1080/13614568.2019.169596125:3(89-111)Online publication date: 2-Dec-2019
https://doi.org/10.1080/13614568.2019.1695961
Azcona DHsiao ISmeaton A(2019)Detecting students-at-risk in computer programming classes with learning analytics from students’ digital footprintsUser Modeling and User-Adapted Interaction10.1007/s11257-019-09234-7Online publication date: 25-Apr-2019
https://doi.org/10.1007/s11257-019-09234-7
Paredes YAzcona DHsiao ISmeaton A(2018)Learning by Reviewing Paper-Based Programming AssessmentsLifelong Technology-Enhanced Learning10.1007/978-3-319-98572-5_39(510-523)Online publication date: 14-Aug-2018
https://doi.org/10.1007/978-3-319-98572-5_39
Azcona DHsiao ISmeaton A(2018)Modelling Math Learning on an Open Access Intelligent TutorArtificial Intelligence in Education10.1007/978-3-319-93846-2_7(36-40)Online publication date: 20-Jun-2018
https://doi.org/10.1007/978-3-319-93846-2_7

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten