research-article

Public Access

Auto-Grading Jupyter Notebooks

Authors:

Stephen H. EdwardsAuthors Info & Claims

SIGCSE '20: Proceedings of the 51st ACM Technical Symposium on Computer Science Education

Pages 1139 - 1144

https://doi.org/10.1145/3328778.3366947

Published: 26 February 2020 Publication History

PDF eReader

Abstract

Jupyter Notebooks are becoming more widely used, both for data science applications and as a convenient environment for learning Python. Currently, grading of assignments done in Jupyter Notebooks is typically done manually. Manual grading results in students receiving feedback only long after the assignment is complete. We implemented support for auto-grading programs written in Jupyter Notebooks within the Web-CAT auto-grading system. Scores received are directly reported to the Canvas gradebook. A Jupyter notebook extension allows students to upload their notebook files to Web-CAT directly. Survey results from class use show that 80% of students believe that getting immediate feedback from Web-CAT improved their performance. Instructors report that this implementation has significantly reduced their workload.

References

[1]

Maha Aziz, Heng Chi, Anant Tibrewal, Max Grossman, and Vivek Sarkar. 2015. Auto-grading for Parallel Programs. In Proceedings of the Workshop on Education for High-Performance Computing. 8 pages.

Digital Library

Google Scholar

[2]

Stephen H Edwards and Manuel A Perez-Quinones. 2008. Web-CAT: Automatically Grading Programming Assignments . ACM SIGCSE Bulletin, Vol. 40, 3 (2008), 328--328.

Digital Library

Google Scholar

[3]

Jessica B Hamrick. 2016. Creating and Grading IPython/Jupyter Notebook Assignments with NbGrader. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education . 242--242.

Digital Library

Google Scholar

[4]

Colin A Higgins, Geoffrey Gray, Pavlos Symeonidis, and Athanasios Tsintsifas. 2005. Automated Assessment and Experiences of Teaching Programming . Journal on Educational Resources in Computing (JERIC), Vol. 5, 3, Article 5 (Sept. 2005).

Google Scholar

[5]

Tony Hirst. 2018. The Growing Popularity of Jupyter Notebooks . https://blog.ouseful.info/2018/09/10/the-growing-popularity-of-jupyter-notebooks/

Google Scholar

[6]

Mike Joy, Nathan Griffiths, and Russell Boyatt. 2005. The BOSS Online Submission and Assessment System . Journal on Educational Resources in Computing (JERIC), Vol. 5, 3, Article 2 (Sept. 2005).

Google Scholar

[7]

Jeffrey M. Perkel. 2018. Why Jupyter is Data Scientists' Computational Notebook of Choice . https://www.nature.com/articles/d41586-018-07196--1

Google Scholar

[8]

Charles Severance, Ted Hanss, and Josepth Hardin. 2010. IMS Learning Tools Interoperability: Enabling a Mash-up Approach to Teaching and Learning Tools . Technology, Instruction, Cognition and Learning, Vol. 7, 3--4 (2010), 245--262.

Google Scholar

[9]

Mark Sherman, Sarita Bassil, Derrell Lipman, Nat Tuck, and Fred Martin. 2013. Impact of Auto-grading on an Introductory Computing Course . Journal of Computing Sciences in Colleges, Vol. 28, 6 (2013), 69--75.

Digital Library

Google Scholar

Cited By

View all

Messer MBrown NKölling MShi M(2024)Automated Grading and Feedback Tools for Programming Education: A Systematic ReviewACM Transactions on Computing Education10.1145/363651524:1(1-43)Online publication date: 19-Feb-2024
https://dl.acm.org/doi/10.1145/3636515
Singh AFariha ABrooks CSoares GHenley ATiwari AM CChoi HGulwani SStephenson BStone JBattestilli LRebelsky SShoop L(2024)Investigating Student Mistakes in Introductory Data Science ProgrammingProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630884(1258-1264)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630884
Ferreira RSabino CCanesche MNeto ONacif J(2024)AIoT tool integration for enriching teaching resources and monitoring student engagementInternet of Things10.1016/j.iot.2023.10104526(101045)Online publication date: Jul-2024
https://doi.org/10.1016/j.iot.2023.101045
Show More Cited By

Index Terms

Auto-Grading Jupyter Notebooks
1. Applied computing
  1. Education
    1. E-learning
    2. Learning management systems
2. Software and its engineering
  1. Software organization and properties
    1. Extra-functional properties
      1. Interoperability

Recommendations

Better code, better sharing: on the need of analyzing jupyter notebooks
ICSE-NIER '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: New Ideas and Emerging Results

By bringing together code, text, and examples, Jupyter notebooks have become one of the most popular means to produce scientific results in a productive and reproducible way. As many of the notebook authors are experts in their scientific fields, but ...
Benefits and Pitfalls of Jupyter Notebooks in the Classroom
SIGITE '20: Proceedings of the 21st Annual Conference on Information Technology Education

Jupyter notebooks are widely used in industry and in academic research, but have only begun to make inroads into the classroom. The design of the Jupyter notebook is in many ways well suited for teaching subjects in information technology and computer ...
Restoring reproducibility of Jupyter notebooks
ICSE '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings

Jupyter notebooks---documents that contain live code, equations, visualizations, and narrative text---now are among the most popular means to compute, present, discuss and disseminate scientific findings. In principle, Jupyter notebooks should easily ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SIGCSE '20: Proceedings of the 51st ACM Technical Symposium on Computer Science Education

February 2020

1502 pages

ISBN:9781450367936

DOI:10.1145/3328778

General Chairs:
Jian Zhang
Texas Woman's University, USA
,
Mark Sherriff
University of Virginia, USA
,
Program Chairs:
Sarah Heckman
North Carolina State University, USA
,
Pamela Cutter
Kalamazoo College, USA
,
Alvaro Monge
California State University, Long Beach, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 February 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

SIGCSE '20

Sponsor:

SIGCSE

SIGCSE '20: The 51st ACM Technical Symposium on Computer Science Education

March 11 - 14, 2020

OR, Portland, USA

Acceptance Rates

Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

Upcoming Conference

SIGCSE Virtual 2024

Sponsor:
sigcse

1st ACM Virtual Global Computing Education Conference

December 5 - 8, 2024

Virtual Event , NC , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
1,222
Total Downloads

Downloads (Last 12 months)291
Downloads (Last 6 weeks)26

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Messer MBrown NKölling MShi M(2024)Automated Grading and Feedback Tools for Programming Education: A Systematic ReviewACM Transactions on Computing Education10.1145/363651524:1(1-43)Online publication date: 19-Feb-2024
https://dl.acm.org/doi/10.1145/3636515
Singh AFariha ABrooks CSoares GHenley ATiwari AM CChoi HGulwani SStephenson BStone JBattestilli LRebelsky SShoop L(2024)Investigating Student Mistakes in Introductory Data Science ProgrammingProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630884(1258-1264)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630884
Ferreira RSabino CCanesche MNeto ONacif J(2024)AIoT tool integration for enriching teaching resources and monitoring student engagementInternet of Things10.1016/j.iot.2023.10104526(101045)Online publication date: Jul-2024
https://doi.org/10.1016/j.iot.2023.101045
Elhayany MMeinel CSpikol DViberg OMartínez-Monés AGuo P(2023)Towards Automated Code Assessment with OpenJupyter in MOOCsProceedings of the Tenth ACM Conference on Learning @ Scale10.1145/3573051.3596180(321-325)Online publication date: 20-Jul-2023
https://dl.acm.org/doi/10.1145/3573051.3596180
Nafa FSreeramareddy LMallapuram SMoulema P(2023)Improving Educational Outcomes: Developing and Assessing Grading System (ProGrader) for Programming CoursesHuman Interface and the Management of Information10.1007/978-3-031-35129-7_24(322-342)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.1007/978-3-031-35129-7_24
González-Carrillo CRestrepo-Calle FRamírez-Echeverry JGonzález F(2021)Automatic Grading Tool for Jupyter Notebooks in Artificial Intelligence CoursesSustainability10.3390/su13211205013:21(12050)Online publication date: 31-Oct-2021
https://doi.org/10.3390/su132112050

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Better code, better sharing: on the need of analyzing jupyter notebooks

Benefits and Pitfalls of Jupyter Notebooks in the Classroom

Restoring reproducibility of Jupyter notebooks