research-article

UserFlow: A Tool for Visualizing Fine-grained Contextual Analytics in Teaching Documents

Authors:

Michael WybrowAuthors Info & Claims

ITiCSE '20: Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education

Pages 384 - 390

https://doi.org/10.1145/3341525.3387410

Published: 15 June 2020 Publication History

Abstract

The adoption of innovative online teaching tools in Computer Science (CS) courses provides opportunities for data-informed instruction as a regular teaching practice in CS classrooms. In this paper, we present a design study for an interactive visual analytics dashboard, called UserFlow, that supports feedback collection from teaching documents and assists instructors in interpreting feedback and acting on it in a timely manner. The design study is conducted with eight domain experts comprising of four teaching instructors, two learning analytics (LA) experts and two instructional designers. UserFlow offers a set of novel visualization designs for presenting the four interleaving aspects of document engagement (i.e., annotations, document traversal path, reading/focus time and student information). We evaluated UserFlow in an undergraduate computer science course with over 700 students. Our results demonstrate the usefulness and need for such a tool for CS educators to inform teaching approaches and courseware improvement.

References

[1]

Aneesha Bakharia, Linda Corrin, Paula De Barba, Gregor Kennedy, Dragan Gavs ević, Raoul Mulder, David Williams, Shane Dawson, and Lori Lockyer. 2016. A conceptual framework linking learning design with learning analytics. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge. ACM, 329--338.

Digital Library

[2]

Aaron Bangor, Philip Kortum, and James Miller. 2009. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies, Vol. 4, 3 (2009), 114--123.

Digital Library

[3]

Lecia Barker and Jane Gruning. 2014. The student prompt: Student feedback and change in teaching practices in postsecondary computer science. In 2014 IEEE Frontiers in Education Conference (FIE) Proceedings. IEEE, 1--8.

[4]

Vivien Beattie IV, Bill Collins, and Bill McInnes. 1997. Deep and surface learning: a simple or simplistic dichotomy? Accounting Education, Vol. 6, 1 (1997), 1--12.

[5]

Geoffrey Ellis and Alan Dix. 2007. A taxonomy of clutter reduction for information visualisation. IEEE transactions on visualization and computer graphics, Vol. 13, 6 (2007), 1216--1223.

[6]

Dragan Gavs ević, Shane Dawson, and George Siemens. 2015. Let's not forget: Learning analytics are about learning. TechTrends, Vol. 59, 1 (2015), 64--71.

[7]

Peter Goodyear and Yannis Dimitriadis. 2013. In medias res: reframing design for learning. Research in Learning Technology, Vol. 21 (2013).

[8]

Sandra G Hart. 2006. NASA-task load index (NASA-TLX); 20 years later. In Proceedings of the human factors and ergonomics society annual meeting, Vol. 50. Sage publications Sage CA: Los Angeles, CA, 904--908.

[9]

Clyde Freeman Herreid and Nancy A Schiller. 2013. Case studies and the flipped classroom. Journal of College Science Teaching, Vol. 42, 5 (2013), 62--66.

[10]

Mark Huxham, Phyllis Laybourn, Sandra Cairncross, Morag Gray, Norrie Brown, Judy Goldfinch, and Shirley Earl. 2008. Collecting student feedback: a comparison of questionnaire and other methods. Assessment & Evaluation in Higher Education, Vol. 33, 6 (2008), 675--686.

[11]

Jelena Jovanovic, Dragan Gasevic, Christopher Brooks, Vladan Devedzic, Marek Hatala, Timmy Eap, and Griff Richards. 2008. LOCO-Analyst: semantic web technologies in learning content usage analysis. International journal of continuing engineering education and life long learning, Vol. 18, 1 (2008), 54--76.

[12]

Jelena Jovanović, Dragan Gavs ević, Abelardo Pardo, Shane Dawson, and Alexander Whitelock-Wainwright. 2019. Introducing meaning to clicks: Towards traced-measures of self-efficacy and cognitive load. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge. ACM, 511--520.

Digital Library

[13]

Greg Kearsley and Ben Shneiderman. 1998. Engagement theory: A framework for technology-based teaching and learning. Educational technology, Vol. 38, 5 (1998), 20--23.

[14]

Mohammad Khalil and Martin Ebner. 2016. What massive open online course (MOOC) stakeholders can learn from learning analytics? Learning, design, and technology: An international compendium of theory, research, practice, and policy (2016), 1--30.

[15]

Danny YT Liu, Charlotte E Taylor, Adam J Bridgeman, Kathryn Bartimote-Aufflick, Abelardo Pardo, et al. 2016. Empowering Instructors Through Customizable Collection and Analyses of Actionable Information. In PCLA@ LAK. 3--9.

[16]

Martin Maguire. 2001. Methods to support human-centred design. International journal of human-computer studies, Vol. 55, 4 (2001), 587--634.

[17]

Riccardo Mazza and Vania Dimitrova. 2004. Visualising student tracking data to support instructors in web-based distance education. In Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters. ACM, 154--161.

Digital Library

[18]

Angela R Penny. 2003. Changing the agenda for research into students' views about university teaching: Four shortcomings of SRT research. Teaching in higher education, Vol. 8, 3 (2003), 399--411.

[19]

Patrick Riehmann, Manfred Hanfler, and Bernd Froehlich. 2005. Interactive sankey diagrams. In IEEE Symposium on Information Visualization, 2005. INFOVIS 2005. IEEE, 233--240.

[20]

Cristóbal Romero and Sebastián Ventura. 2010. Educational data mining: a review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol. 40, 6 (2010), 601--618.

Digital Library

[21]

Shaveen Singh. 2019. Exploring the Potential of Social Annotations for Predictive and Descriptive Analytics. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education. 247--248.

Digital Library

[22]

Shaveen Singh and Sunil Pranit Lal. 2013. Using feature selection and association rule mining to evaluate digital courseware. In 2013 Eleventh International Conference on ICT and Knowledge Engineering. IEEE, 1--7.

[23]

Shaveen Singh and Bernd Meyer. 2019. Using social annotations to augment the learning space and learner experience. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education. ACM, 527--533.

Digital Library

[24]

Marc Steen. 2011. Tensions in human-centred design. CoDesign, Vol. 7, 1 (2011), 45--60.

[25]

Alyssa Friend Wise, Jovita Maria Vytasek, Simone Hausknecht, and Yuting Zhao. 2016. Developing Learning Analytics Design Knowledge in the "Middle Space": The Student Tuning Model and Align Design Framework for Learning Analytics Use. Online Learning, Vol. 20, 2 (2016), 155--182.

[26]

Jonathan Witt Dr, Carey Bissonnette Dr, and E Mary. 2015. Student feedback from beginning to end: A new course evaluation model. (2015).

Cited By

Kaliisa RJivet IPrinsloo P(2023)A checklist to guide the planning, designing, implementation, and evaluation of learning analytics dashboardsInternational Journal of Educational Technology in Higher Education10.1186/s41239-023-00394-620:1Online publication date: 3-May-2023
https://doi.org/10.1186/s41239-023-00394-6

Index Terms

UserFlow: A Tool for Visualizing Fine-grained Contextual Analytics in Teaching Documents
1. Applied computing
  1. Education

Recommendations

Designing a learning analytics dashboard for twitter-facilitated teaching
L@S '18: Proceedings of the Fifth Annual ACM Conference on Learning at Scale

Social media sites are increasingly being adopted to support teaching practice in higher education. Learning Analytics (LA) dashboards can be used to reveal how students engage with course material and others in the class. However, research on the best ...
The Learning Analytics System that Improves the Teaching-Learning Experience of MOOC Instructors and Students
Learning Technologies and Systems
Abstract
Great learning opportunities are provided through MOOCs. However, MOOCs provide a number of challenges for students. Many students find it difficult to successfully finish MOOCs due to a variety of factors, including feelings of loneliness, a lack ...
Robotics to promote elementary education pre-service teachers' STEM engagement, learning, and teaching

We report a research project with a purpose of helping teachers learn how to design and implement science, technology, engineering, and mathematics (STEM) lessons using robotics. Specifically, pre-service teachers' STEM engagement, learning, and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ITiCSE '20: Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education

June 2020

615 pages

ISBN:9781450368742

DOI:10.1145/3341525

General Chairs:
Michail Giannakos
NTNU, Norway
,
Guttorm Sindre
Excited/NTNU, Norway
,
Program Chairs:
Andrew Luxton-Reilly
University of Auckland, NZ
,
Monica Divitini
NTNU, Norway

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 June 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ITiCSE '20

Sponsor:

SIGCSE

ITiCSE '20: Innovation and Technology in Computer Science Education

June 15 - 19, 2020

Trondheim, Norway

Acceptance Rates

Overall Acceptance Rate 552 of 1,613 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
290
Total Downloads

Downloads (Last 12 months)34
Downloads (Last 6 weeks)3

Reflects downloads up to 27 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kaliisa RJivet IPrinsloo P(2023)A checklist to guide the planning, designing, implementation, and evaluation of learning analytics dashboardsInternational Journal of Educational Technology in Higher Education10.1186/s41239-023-00394-620:1Online publication date: 3-May-2023
https://doi.org/10.1186/s41239-023-00394-6

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents