Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3631700.3665228acmconferencesArticle/Chapter ViewAbstractPublication PagesumapConference Proceedingsconference-collections
research-article
Open access

A Learning Analytics Dashboard for K-12 English Teachers - Bridging the Gap Between Student Process Data and Teacher Needs

Published: 28 June 2024 Publication History

Abstract

Educational technologies are being used more and more in secondary school settings. This increases the amount of students’ learning related data produced and stored. To keep up with this rise and to get most out of the collected data, teachers need digital tools that support and facilitate their pedagogical decision-making process. Learning analytics dashboards can be a good source to provide teachers with necessary insights into their students’ learning processes. However, for such tools to be effective and actionable, they have to be aligned with teachers’ needs and thus, provide and visualize data in a concise and structured way. We therefore conducted a survey study with 11 English teachers from K-12 secondary schools in Germany who evaluated the assumed usefulness of possible dashboard features. Based on these findings, we developed a teacher dashboard incorporating the most desired functionalities, such as a quickly accessible summary of strengths, weaknesses and support needs, or an overview of current misconceptions and competencies alongside additional metrics in order to support multiple teaching practices. The implementation and the underlying calculations are described, focusing on the importance of learners’ process data to provide teachers with a detailed and revealing view on their students’ and class learning states. In an evaluation study of the dashboard’s prototype with mock data, teachers (n=6) gave high ratings for the dashboard’s usability.

References

[1]
Fernando RH Andrade, Riichiro Mizoguchi, and Seiji Isotani. 2016. The bright and dark sides of gamification. In Intelligent Tutoring Systems: 13th International Conference, ITS 2016, Zagreb, Croatia, June 7-10, 2016. Proceedings 13. Springer, 176–186.
[2]
Benjamin Bach, Euan Freeman, Alfie Abdul-Rahman, Cagatay Turkay, Saiful Khan, Yulei Fan, and Min Chen. 2022. Dashboard design patterns. IEEE Transactions on Visualization and Computer Graphics 29, 1 (2022), 342–352.
[3]
Robert Bodily and Katrien Verbert. 2017. Review of research on student-facing learning analytics dashboards and educational recommender systems. IEEE Transactions on Learning Technologies 10, 4 (2017), 405–418.
[4]
Paul Caldarella, Ross AA Larsen, Leslie Williams, Kade R Downs, Howard P Wills, and Joseph H Wehby. 2020. Effects of teachers’ praise-to-reprimand ratios on elementary students’ on-task behaviour. Educational Psychology 40, 10 (2020), 1306–1322.
[5]
Leona Colling, Tanja Heck, and Detmar Meurers. 2023. Reconciling Adaptivity and Task Orientation in the Student Dashboard of an Intelligent Language Tutoring System. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023). 288–299.
[6]
Leona Colling, Ines Pieronczyk, Cora Parrisius, Heiko Holz, Stephen Bodnar, Florian Nuxoll, and Detmar Meurers. 2024. Towards Task-Oriented ICALL: A Criterion-Referenced Learner Dashboard Organising Digital Practice. In Proceedings of the 16th International Conference on Computer Supported Education - Volume 1: CSEDU. 668–679. https://doi.org/10.5220/0012753000003693
[7]
Mollie Dollinger, Danny Liu, Natasha Arthars, and Jason M Lodge. 2019. Working together in learning analytics towards the co-creation of value.Journal of Learning Analytics 6, 2 (2019), 10–26.
[8]
Raphael A Dourado, Rodrigo Lins Rodrigues, Nivan Ferreira, Rafael Ferreira Mello, Alex Sandro Gomes, and Katrien Verbert. 2021. A teacher-facing learning analytics dashboard for process-oriented feedback in online learning. In LAK21: 11th International Learning Analytics and Knowledge Conference. 482–489.
[9]
Hermann Ebbinghaus. 1885. Über das gedächtnis: untersuchungen zur experimentellen psychologie. Duncker & Humblot.
[10]
Roberta Galici, Tanja Käser, Gianni Fenu, and Mirko Marras. 2023. How close are predictive models to teachers in detecting learners at risk?. In Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization. 135–145.
[11]
Carolina Guzmán-Valenzuela, Carolina Gómez-González, Andrés Rojas-Murphy Tagle, and Alejandro Lorca-Vyhmeister. 2021. Learning analytics in higher education: a preponderance of analytics but very little learning?International Journal of Educational Technology in Higher Education 18 (2021). https://api.semanticscholar.org/CorpusID:233865710
[12]
Matthew R Hodgman. 2015. Student praise in the modern classroom: The use of praise notes as a productive motivational tool. Journal of Education and Training 2, 1 (2015), 41–47.
[13]
Nicole Hutchins and Gautam Biswas. 2023. Using Teacher Dashboards to Customize Lesson Plans for a Problem-Based, Middle School STEM Curriculum. In LAK23: 13th International Learning Analytics and Knowledge Conference. 324–332.
[14]
Ioana Jivet, Maren Scheffel, Marcus Specht, and Hendrik Drachsler. 2018. License to evaluate: Preparing learning analytics dashboards for educational practice. In Proceedings of the 8th international conference on learning analytics and knowledge. 31–40.
[15]
Rogers Kaliisa, Anna Gillespie, Christothea Herodotou, Anders Kluge, and Bart Rienties. 2021. Teachers’ perspectives on the promises, needs and challenges of learning analytics dashboards: Insights from institutions offering blended and distance learning. In Visualizations and dashboards for learning analytics. Springer, 351–370.
[16]
Rogers Kaliisa, Ioana Jivet, and Paul Prinsloo. 2023. A checklist to guide the planning, designing, implementation, and evaluation of learning analytics dashboards. International Journal of Educational Technology in Higher Education 20, 1 (2023), 28.
[17]
Sean HK Kang. 2016. Spaced repetition promotes efficient and effective learning: Policy implications for instruction. Policy Insights from the Behavioral and Brain Sciences 3, 1 (2016), 12–19.
[18]
Mareike Kholin, Leona Colling, Nina Lamberti, and Detmar Meurers. in preparation. Elevating K-12 Education: Innovations in Learning Analytics Teacher-Facing Dashboards and Evidence-Based Best Practices. (in preparation).
[19]
Hassan Khosravi, Simon Buckingham Shum, Guanliang Chen, Cristina Conati, Yi-Shan Tsai, Judy Kay, Simon Knight, Roberto Martinez-Maldonado, Shazia Sadiq, and Dragan Gašević. 2022. Explainable artificial intelligence in education. Computers and Education: Artificial Intelligence 3 (2022), 100074.
[20]
Bracha Kramarski. 2018. Teachers as agents in promoting students’ srl and performance: applications for teachers’ dual-role training program. In Handbook of self-regulation and of learning and performance. Routledge, 223–239.
[21]
Mareike Kunter and Silvia Ewald. 2016. Bedingungen und Effekte von Unterricht: Aktuelle Forschungsperspektiven aus der pädagogischen Psychologie. Bedingungen und Effekte guten Unterrichts 1 (2016).
[22]
Richard N Landers and Amy K Landers. 2014. An empirical test of the theory of gamified learning: The effect of leaderboards on time-on-task and academic performance. Simulation & Gaming 45, 6 (2014), 769–785.
[23]
Bettina Laugwitz, Theo Held, and Martin Schrepp. 2008. Construction and evaluation of a user experience questionnaire. In HCI and Usability for Education and Work: 4th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society, USAB 2008, Graz, Austria, November 20-21, 2008. Proceedings 4. Springer, 63–76.
[24]
Detmar Meurers, Kordula De Kuthy, Florian Nuxoll, Björn Rudzewitz, and Ramon Ziai. 2019. Scaling up intervention studies to investigate real-life foreign language learning in school. Annual Review of Applied Linguistics 39 (2019), 161–188.
[25]
Inge Molenaar and Carolien Knoop-van Campen. 2017. Teacher dashboards in practice: Usage and impact. In Data Driven Approaches in Digital Education: 12th European Conference on Technology Enhanced Learning, EC-TEL 2017, Tallinn, Estonia, September 12–15, 2017, Proceedings 12. Springer, 125–138.
[26]
Inge Molenaar and Carolien AN Knoop-van Campen. 2018. How teachers make dashboard information actionable. IEEE Transactions on Learning Technologies 12, 3 (2018), 347–355.
[27]
Tanya Nazaretsky, Mutlu Cukurova, and Giora Alexandron. 2022. An instrument for measuring teachers’ trust in AI-based educational technology. In LAK22: 12th international learning analytics and knowledge conference. 56–66.
[28]
Cora Parrisius, Ines Pieronczyk, Carolyn Blume, Katharina Wendebourg, Diana Pili-Moss, Mirjam Assmann, Sabine Beilharz, Stephen Bodnar, Leona Colling, Heiko Holz, 2022. Using an intelligent tutoring system within a task-based learning approach in English as a foreign language classes to foster motivation and learning outcome (Interact4School): Pre-registration of the study design. (2022).
[29]
Stanislav Pozdniakov, Roberto Martinez-Maldonado, Yi-Shan Tsai, Namrata Srivastava, Yuchen Liu, and Dragan Gasevic. 2023. Single or Multi-page Learning Analytics Dashboards? Relationships Between Teachers’ Cognitive Load and Visualisation Literacy. In European Conference on Technology Enhanced Learning. Springer, 339–355.
[30]
Martí Quixal, Björn Rudzewitz, Elizabeth Bear, and Detmar Meurers. 2021. Automatic annotation of curricular language targets to enrich activity models and support both pedagogy and adaptive systems. In Proceedings of the 10th Workshop on NLP for Computer Assisted Language Learning. 15–27.
[31]
Maria Rauschenberger, Martin Schrepp, Manuel Pérez Cota, Siegfried Olschner, and Jörg Thomaschewski. 2013. Efficient measurement of the user experience of interactive products. How to use the user experience questionnaire (UEQ). Example: Spanish language version. (2013).
[32]
Björn Rudzewitz, Ramon Ziai, Kordula De Kuthy, and Detmar Meurers. 2017. Developing a web-based workbook for English supporting the interaction of students and teachers. In Proceedings of the joint workshop on NLP for Computer Assisted Language Learning and NLP for Language Acquisition. 36–46.
[33]
Björn Rudzewitz, Ramon Ziai, Kordula De Kuthy, Verena Möller, Florian Nuxoll, and Detmar Meurers. 2018. Generating feedback for English foreign language exercises. In Proceedings of the thirteenth workshop on innovative use of NLP for building educational applications. 127–136.
[34]
Björn Rudzewitz, Ramon Ziai, Florian Nuxoll, Kordula De Kuthy, and Detmar Meurers. 2019. Enhancing a Web-based Language Tutoring System with Learning Analytics. In EDM (Workshops). 1–7.
[35]
Allen G. Schick, Lawrence A. Gordon, and S. Haka. 1990. Information overload: A temporal approach. Accounting Organizations and Society 15 (1990), 199–220. https://doi.org/10.1016/0361-3682(90)90005-F
[36]
Martin Schrepp, Jörg Thomaschewski, and Andreas Hinderks. 2017. Construction of a benchmark for the user experience questionnaire (UEQ). (2017).
[37]
Beat A Schwendimann, Maria Jesus Rodriguez-Triana, Andrii Vozniuk, Luis P Prieto, Mina Shirvani Boroujeni, Adrian Holzer, Denis Gillet, and Pierre Dillenbourg. 2016. Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies 10, 1 (2016), 30–41.
[38]
Gayane Sedrakyan, Jonna Malmberg, Katrien Verbert, Sanna Järvelä, and Paul A Kirschner. 2020. Linking learning behavior analytics and learning science concepts: Designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior 107 (2020), 105512.
[39]
Vinitra Swamy, Sijia Du, Mirko Marras, and Tanja Kaser. 2023. Trusting the explainers: teacher validation of explainable artificial intelligence for course design. In LAK23: 13th International Learning Analytics and Knowledge Conference. 345–356.
[40]
Natercia Valle, Pavlo Antonenko, Kara Dawson, and Anne Corinne Huggins-Manley. 2021. Staying on target: A systematic literature review on learner-facing learning analytics dashboards. British Journal of Educational Technology 52, 4 (2021), 1724–1748.
[41]
Anouschka Van Leeuwen and Nikol Rummel. 2020. Comparing teachers’ use of mirroring and advising dashboards. In Proceedings of the tenth international conference on learning analytics & knowledge. 26–34.
[42]
Maarten Van Someren, Yvonne F Barnard, and J Sandberg. 1994. The think aloud method: a practical approach to modelling cognitive. London: AcademicPress 11, 6 (1994).
[43]
M Wiedbusch. 2021. Intelligent teacher dashboards that support students’ self-regulated learning, engagement, and teachers’ decision making. 6 (2021).
[44]
Kimberly Williamson and Rene Kizilcec. 2022. A review of learning analytics dashboard research in higher education: Implications for justice, equity, diversity, and inclusion. In LAK22: 12th international learning analytics and knowledge conference. 260–270.
[45]
Françeska Xhakaj, Vincent Aleven, and Bruce M McLaren. 2017. Effects of a teacher dashboard for an intelligent tutoring system on teacher knowledge, lesson planning, lessons and student learning. In Data Driven Approaches in Digital Education: 12th European Conference on Technology Enhanced Learning, EC-TEL 2017, Tallinn, Estonia, September 12–15, 2017, Proceedings 12. Springer, 315–329.

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
UMAP Adjunct '24: Adjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization
June 2024
662 pages
ISBN:9798400704666
DOI:10.1145/3631700
This work is licensed under a Creative Commons Attribution-ShareAlike International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 June 2024

Check for updates

Author Tags

  1. computer-assisted language learning
  2. intelligent tutoring systems
  3. learning analytics
  4. teacher dashboard

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • aim - Akademie für Innovative Bildung und Management Heilbronn-Franken gemeinnützige GmbH
  • LEAD Graduate School & Research Network

Conference

UMAP '24
Sponsor:

Acceptance Rates

Overall Acceptance Rate 162 of 633 submissions, 26%

Upcoming Conference

UMAP '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 243
    Total Downloads
  • Downloads (Last 12 months)243
  • Downloads (Last 6 weeks)71
Reflects downloads up to 18 Nov 2024

Other Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media