research-article

Assessing Programming Skills and Knowledge During the COVID-19 Pandemic: An Experience Report

Authors:

Norman Tiong Seng Lee,

Kenny Tsu Wei ChooAuthors Info & Claims

ITiCSE '21: Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1

Pages 352 - 358

https://doi.org/10.1145/3430665.3456323

Published: 26 June 2021 Publication History

Abstract

The current COVID-19 pandemic has resulted in disruption to the delivery of higher education. The government-mandated workplace closures that lasted for two months from April 2020 resulted in the closing of all university campuses in our city. This happened while our first-year introductory Python programming course was still in progress. We were thus unable to administer our final exam on campus. In this paper, we describe how our final exam, usually conducted on campus, was replaced with a performance-based assessment. This assessment tasked students to design and program their own game individually. After submitting their code, each student was then required to attend an oral exam that was administered online. We reflect on our experience, drawing from both instructors' and students' perspectives of the programming task and the assessment format. We conclude with a description of how the lessons learnt were applied to a subsequent run of the course.

References

[1]

Joel C. Adams. 1998. Chance-It: An Object-Oriented Capstone Project for CS-1. In Proceedings of the Twenty-Ninth SIGCSE Technical Symposium on Computer Science Education (Atlanta, Georgia, USA) (SIGCSE '98). Association for Computing Machinery, New York, NY, USA, 10--14. https://doi.org/10.1145/273133.273140

Digital Library

[2]

Alexandr Akimov and Mirela Malin. 2020. When old becomes new: a case study of oral examination as an online assessment tool. Assessment & Evaluation in Higher Education, Vol. 0, 0 (2020), 1--17. https://doi.org/10.1080/02602938.2020.1730301

[3]

Infocomm Media Development Authority. 2020. Infocomm Usage-Households and Individuals. Singapore. https://www.imda.gov.sg/infocomm-media-landscape/research-and-statistics/infocomm-usage-households-and-individuals Retrieved March 10, 2021 from

[4]

Jessica D. Bayliss. 2009. Using Games in Introductory Courses: Tips from the Trenches. In Proceedings of the 40th ACM Technical Symposium on Computer Science Education (Chattanooga, TN, USA) (SIGCSE '09). Association for Computing Machinery, New York, NY, USA, 337--341.

Digital Library

[5]

M. Bearman, P. Dawson, M. O'Donnell, J. Tai, and T. Jorre de St Jorre. 2020. Ensuring academic integrity and assessment security with redesigned online delivery. https://dteach.deakin.edu.au/2020/03/23/academic-integrity-online/

[6]

Katrin Becker. 2001. Teaching with Games: The Minesweeper and Asteroids Experience. J. Comput. Sci. Coll., Vol. 17, 2 (Dec. 2001), 23--33.

[7]

Tracey Bretag, Rowena Harper, Michael Burton, Cath Ellis, Philip Newton, Karen van Haeringen, Sonia Saddiqui, and Pearl Rozenberg. 2019. Contract cheating and assessment design: exploring the relationship. Assessment & Evaluation in Higher Education, Vol. 44, 5 (2019), 676--691. https://doi.org/10.1080/02602938.2018.1527892

[8]

Robert Clarke and Thomas Lancaster. 2006. Eliminating the successor to plagiarism? Identifying the usage of contract cheating sites. In proceedings of 2nd international plagiarism conference. Citeseer, 19--21.

[9]

Robert Clarke and Thomas Lancaster. 2007. Establishing a systematic six-stage process for detecting contract cheating. In 2007 2nd International conference on pervasive computing and applications. IEEE, 342--347.

[10]

D. C. Cliburn. 2006. The Effectiveness of Games as Assignments in an Introductory Programming Course. In Proceedings. Frontiers in Education. 36th Annual Conference. 6--10.

[11]

Joseph Crawford, Kerryn Butler-Henderson, Jürgen Rudolph, Bashar Malkawi, Matt Glowatz, Rob Burton, Paola A. Magni, and Sophia Lam. 2020. COVID-19: 20 countries' higher education intra-period digital pedagogy responses. Journal of Applied Learning & Teaching, Vol. 3, 1 (2020), 1--20. https://doi.org//10.37074/jalt.2020.3.1.7

[12]

Tom Crick, Cathryn Knight, Richard Watermeyer, and Janet Goodall. 2020. The impact of COVID-19 and ?Emergency Remote Teaching" on the UK computer science education community. In United Kingdom & Ireland Computing Education Research conference. 31--37.

Digital Library

[13]

Coleen Flaherty. 2020. https://www.insidehighered.com/news/2020/05/11/online-proctoring-surging-during-covid-19

[14]

Marcus L George. 2020. Effective teaching and examination strategies for undergraduate learning during COVID-19 school restrictions. Journal of Educational Technology Systems, Vol. 49, 1 (2020), 23--48.

[15]

Hasmik Gharibyan. 2005. Assessing Students' Knowledge: Oral Exams vs. Written Tests. SIGCSE Bull., Vol. 37, 3 (June 2005), 143--147.

[16]

Anandha Gopalan and Robert Chatley. 2020. Converting to online exams. http://www.imperial.ac.uk/staff/educational-development/teaching-toolkit/remote-online-learning/case-studies/converting-to-online-exams/ Retrieved June 25, 2020 from

[17]

Singapore Government. 2020. COVID-19 circuit breaker: Closure of workplace premises. https://www.gov.sg/article/covid-19-circuit-breaker-closure-of-workplace-premises Retrieved March 10, 2021 from

[18]

Arto Hellas, Juho Leinonen, and Petri Ihantola. 2017. Plagiarism in take-home exams: help-seeking, collaboration, and systematic cheating. In Proceedings of the 2017 ACM conference on innovation and technology in computer science education. 238--243.

Digital Library

[19]

Mark Huxham, Fiona Campbell, and Jenny Westwood. 2012. Oral versus written assessments: a test of student performance and attitudes. Assessment & Evaluation in Higher Education, Vol. 37, 1 (2012), 125--136.

[20]

Paola Iannone and Adrian Simpson. 2015. Students' views of oral performance assessment in mathematics: straddling the 'assessment of' and 'assessment for' learning divide. Assessment & Evaluation in Higher Education, Vol. 40, 7 (2015), 971--987. https://doi.org/10.1080/02602938.2014.961124

[21]

Respondus Inc. [n.d.] a. Respondus Monitor. https://web.respondus.com/he/monitor/ Retrieved June 25, 2020 from

[22]

Zoom Video Communications Inc. [n.d.] b. Zoom. https://www.zoom.us Retrieved Jan 10, 2021 from

[23]

Inspera. [n.d.]. Inspera Smarter Proctoring. https://www.inspera.com/smarter-proctoring Retrieved Jan 10, 2021 from

[24]

Kivy. [n.d.]. Kivy. https://kivy.org Retrieved June 25, 2020 from

[25]

Oka Kurniawan, Norman Tiong Seng Lee, and Christopher M. Poskitt. 2020. Securing Bring-Your-Own-Device (BYOD) Programming Exams. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (Portland, OR, USA) (SIGCSE '20). Association for Computing Machinery, New York, NY, USA, 880--886. https://doi.org/10.1145/3328778.3366907

[26]

Norman Lee, Kenny Choo, and Oka Kurniawan. 2020. Assessing Programming Knowledge and Ability with an Online Oral Exam. In EduSCAPES: An SUTD Pedagogy Magazine. https://www.sutd.edu.sg/learningsciences/Resources/SUTD-Pedagogy-Newsletter Retrieved June 10, 2020 from

[27]

Torben Lorenzen and Ward Heilman. 2002. CS1 and CS2: Write Computer Games in Java! SIGCSE Bull., Vol. 34, 4 (Dec. 2002), 99--100.

Digital Library

[28]

John M. Malouff, Ashley J. Emmerton, and Nicola S. Schutte. 2013. The Risk of a Halo Bias as a Reason to Keep Students Anonymous During Grading. Teaching of Psychology, Vol. 40, 3 (2013), 233--237. https://doi.org/10.1177/0098628313487425

[29]

Anita McCauley. 2020. Assessing Student Learning: Alternatives to Closed-Book Exams. https://cat.wfu.edu/2020/04/alternativestoexams/ Retrieved June 10, 2020 from

[30]

Microsoft. [n.d.]. Microsoft Teams. https://www.microsoft.com/en-sg/microsoft-365/microsoft-teams/group-chat-software Retrieved June 25, 2020 from

[31]

Kurt Nørmark, Lone Leth Thomsen, and Kristian Torp. 2008. Mini project programming exams. In Reflections on the Teaching of Programming. Springer, 228--242.

[32]

University of Bergen. 2020. Assessment and exams in times of Korona. https://mitt.uib.no/courses/25566/pages/assessment-and-exams-in-times-of-korona Retrieved June 10, 2020 from

[33]

Peter Ohmann. 2019. An Assessment of Oral Exams in Introductory CS. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (Minneapolis, MN, USA) (SIGCSE '19). Association for Computing Machinery, New York, NY, USA, 613--619. https://doi.org/10.1145/3287324.3287489

Digital Library

[34]

Lee Roecker. 2007. Using Oral Examination as a Technique To Assess Student Understanding and Teaching Effectiveness. Journal of Chemical Education, Vol. 84, 10 (2007), 1663. https://doi.org/10.1021/ed084p1663

[35]

Richard M Ryan. 1982. Control and information in the intrapersonal sphere: An extension of cognitive evaluation theory. Journal of personality and social psychology, Vol. 43, 3 (1982), 450.

[36]

Catherine Shea Sanger. 2020. Teaching intelligence: how to take your classes online. https://www.timeshighereducation.com/career/teaching-intelligence-how-take-your-classes-online Retrieved June 25, 2020 from

[37]

Daniel L. Schuster. 2010. CS1, Arcade Games and the Free Java Book. In Proceedings of the 41st ACM Technical Symposium on Computer Science Education (Milwaukee, Wisconsin, USA) (SIGCSE '10). Association for Computing Machinery, New York, NY, USA, 549--553. https://doi.org/10.1145/1734263.1734445

Digital Library

[38]

Margaret Scott and John Unsworth. 2018. Matching final assessment to employability: developing a digital viva as an end of programme assessment. Higher Education Pedagogies, Vol. 3, 1 (2018), 373--384. https://doi.org/10.1080/23752696.2018.1510294

[39]

Judy Sheard, Simon, Matthew Butler, Katrina Falkner, Michael Morgan, and Amali Weerasinghe. 2017. Strategies for Maintaining Academic Integrity in First-Year Computing Courses. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education (Bologna, Italy) (ITiCSE '17). Association for Computing Machinery, New York, NY, USA, 244--249. https://doi.org/10.1145/3059009.3059064

Digital Library

[40]

Guttorm Sindre, Steinar Line, and Ottar V. Valvrag. 2003. Positive Experiences with an Open Project Assignment in an Introductory Programming Course. In Proceedings of the 25th International Conference on Software Engineering (Portland, Oregon) (ICSE '03). IEEE Computer Society, USA, 608--613.

[41]

Popi Sotiriadou, Danielle Logan, Amanda Daly, and Ross Guest. 2019. The role of authentic assessment to preserve academic integrity and promote skill development and employability. Studies in Higher Education, Vol. 0, 0 (2019), 1--17. https://doi.org/10.1080/03075079.2019.1582015

[42]

CPP Stephen Saravara III JD. 2007. Business Continuity Planning in Higher Education Due to Pandemic Outbreaks. Journal of Security Education, Vol. 2, 3 (2007), 41--51. https://doi.org/10.1300/J460v02n03_04

[43]

Yang Wong. 2020. Coronavirus: NUS students get zero marks for cheating in take-home exam. The Straits Times (15 Jun 2020). https://www.straitstimes.com/singapore/education/coronavirus-nus-students-get-zero-marks-for-cheating-on-take-home-exam

[44]

Mark Zarb and Jen BirtlesKelman. 2020. Through the Lens: Enhancing Assessment with Video-Based Presentation. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education (Trondheim, Norway) (ITiCSE '20). Association for Computing Machinery, New York, NY, USA, 187--192. https://doi.org/10.1145/3341525.3387376

Digital Library

Cited By

Lubarda MPhan ASchurgers CDelson NGhazinejad MBaghdadchi SMinnes MKim MPilegard CRelaford-Doyle JSandoval CQi H(2024)Virtual pair programming and online oral exams: effects on social interaction, performance, and academic integrity in a remote computer programming courseComputer Science Education10.1080/08993408.2024.2344401(1-41)Online publication date: 5-May-2024
https://doi.org/10.1080/08993408.2024.2344401
Hickman HMcKeown PBell T(2023)Beyond Question Shuffling: Randomization Techniques in Programming Assessment2023 IEEE Frontiers in Education Conference (FIE)10.1109/FIE58773.2023.10342976(1-9)Online publication date: 18-Oct-2023
https://doi.org/10.1109/FIE58773.2023.10342976
Regnier JShafer ESobiesk EStave NHaynes M(2023)From crisis to opportunity: practices and technologies for a more effective post-COVID classroomEducation and Information Technologies10.1007/s10639-023-11929-929:5(5981-6003)Online publication date: 22-Jul-2023
https://dl.acm.org/doi/10.1007/s10639-023-11929-9
Show More Cited By

Index Terms

Assessing Programming Skills and Knowledge During the COVID-19 Pandemic: An Experience Report
1. Applied computing
  1. Education
2. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Student assessment

Recommendations

Assessing computational learning in K-12
ITiCSE '14: Proceedings of the 2014 conference on Innovation & technology in computer science education

As computing curricula continue to make their way into K-12 schools, the issue of assessing student learning of computational concepts remains a thorny one. This paper describes the multiple forms of assessments used in a 6-week middle school curriculum ...
Assessing the Development of Computer Science Pedagogical Content Knowledge in the TEALS Program (Abstract Only)
SIGCSE '16: Proceedings of the 47th ACM Technical Symposium on Computing Science Education

One of the critical barriers to increasing pre-collegiate computer science course offerings in the U.S. is a lack of qualified computer science teachers. Programs such as TEALS, a teacher preparation program pairing high school teachers with computing ...
A survey of Australian teachers' self-efficacy and assessment approaches for the K-12 digital technologies curriculum
WiPSCE '18: Proceedings of the 13th Workshop in Primary and Secondary Computing Education

As K-12 computer science (CS) education has been introduced to a number of countries around the world, the CS education community has been busy working to understand the learning and teaching of computing at these year levels, as well as how to build ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ITiCSE '21: Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1

June 2021

611 pages

ISBN:9781450382144

DOI:10.1145/3430665

General Chairs:
Carsten Schulte
Paderborn University, Germany
,
Brett A. Becker
University College Dublin, Ireland
,
Program Chairs:
Monica Divitini
NTNU, Norway
,
Erik Barendsen
Radboud University & Open University, The Netherlands

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ITiCSE 2021

Sponsor:

SIGCSE

ITiCSE 2021: 26th ACM Conference on Innovation and Technology in Computer Science Education

June 26 - July 1, 2021

Virtual Event, Germany

Acceptance Rates

Overall Acceptance Rate 552 of 1,613 submissions, 34%

Upcoming Conference

ITiCSE '25

Sponsor:
sigcse

Innovation and Technology in Computer Science Education

June 27 - July 2, 2025

Nijmegen , Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
263
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)3

Reflects downloads up to 15 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lubarda MPhan ASchurgers CDelson NGhazinejad MBaghdadchi SMinnes MKim MPilegard CRelaford-Doyle JSandoval CQi H(2024)Virtual pair programming and online oral exams: effects on social interaction, performance, and academic integrity in a remote computer programming courseComputer Science Education10.1080/08993408.2024.2344401(1-41)Online publication date: 5-May-2024
https://doi.org/10.1080/08993408.2024.2344401
Hickman HMcKeown PBell T(2023)Beyond Question Shuffling: Randomization Techniques in Programming Assessment2023 IEEE Frontiers in Education Conference (FIE)10.1109/FIE58773.2023.10342976(1-9)Online publication date: 18-Oct-2023
https://doi.org/10.1109/FIE58773.2023.10342976
Regnier JShafer ESobiesk EStave NHaynes M(2023)From crisis to opportunity: practices and technologies for a more effective post-COVID classroomEducation and Information Technologies10.1007/s10639-023-11929-929:5(5981-6003)Online publication date: 22-Jul-2023
https://dl.acm.org/doi/10.1007/s10639-023-11929-9
Kamthan P(2022)The Experience of Tests during the COVID-19 Pandemic-Induced Emergency Remote TeachingInternational Journal of Software Engineering and Knowledge Engineering10.1142/S021819402240002232:04(481-501)Online publication date: 8-Apr-2022
https://doi.org/10.1142/S0218194022400022
Seng Lee NKumiawan OWei Choo KDe Mari MPerrault S(2022)Laboratory Activities for an Introductory Programming Course2022 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE)10.1109/TALE54877.2022.00152(818-821)Online publication date: Dec-2022
https://doi.org/10.1109/TALE54877.2022.00152
Kohler JHidalgo LJara J(2022)Using machine learning techniques to predict academic success in an introductory programming course2022 41st International Conference of the Chilean Computer Science Society (SCCC)10.1109/SCCC57464.2022.10000360(1-8)Online publication date: 21-Nov-2022
https://doi.org/10.1109/SCCC57464.2022.10000360
Pesonen HHellas A(2022)On Things that Matter in Learning Programming: Towards a Scale for New Programming Students2022 IEEE Frontiers in Education Conference (FIE)10.1109/FIE56618.2022.9962572(1-9)Online publication date: 8-Oct-2022
https://doi.org/10.1109/FIE56618.2022.9962572
Srivatanakul T(2022)Emerging from the pandemic: instructor reflections and students’ perceptions on an introductory programming course in blended learningEducation and Information Technologies10.1007/s10639-022-11328-628:5(5673-5695)Online publication date: 4-Nov-2022
https://dl.acm.org/doi/10.1007/s10639-022-11328-6

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten