research-article

How patterns of students dashboard use are related to their achievement and self-regulatory engagement

Authors:

Fatemeh Salehian Kia,

Stephanie D. Teasley,

Stuart A. Karabenick,

Matthew KayAuthors Info & Claims

LAK '20: Proceedings of the Tenth International Conference on Learning Analytics & Knowledge

Pages 340 - 349

https://doi.org/10.1145/3375462.3375472

Published: 23 March 2020 Publication History

Abstract

The aim of student-facing dashboards is to support learning by providing students with actionable information and promoting self-regulated learning. We created a new dashboard design aligned with SRL theory, called MyLA, to better understand how students use a learning analytics tool. We conducted sequence analysis on students' interactions with three different visualizations in the dashboard, implemented in a LMS, for a large number of students (860) in ten courses representing different disciplines. To evaluate different students' experiences with the dashboard, we computed chi-squared tests of independence on dashboard users (52%) to find frequent patterns that discriminate students by their differences in academic achievement and self-regulated learning behaviors. The results revealed discriminating patterns in dashboard use among different levels of academic achievement and self-regulated learning, particularly for low achieving students and high self-regulated learners. Our findings highlight the importance of differences in students' experience with a student-facing dashboard, and emphasize that one size does not fit all in the design of learning analytics tools.

References

[1]

Rakesh. Agrawal and Ramakrishnan Srikant. 1995. Mining sequential patterns. In Proceedings of the International Conference on Data Engeneering (ICDE), In P. S. Yu and A. L. P. Chen (Eds.). IEEE Computer Society, Taipei, Taiwan, 487--499.

[2]

Stephen J Aguilar. 2016. Perceived Motivational Affordances: Capturing and Measuring Students' Sense-Making Around Visualizations of their Academic Achievement Information. (2016).

[3]

Sanam Shirazi Beheshitha, Marek Hatala, Dragan Gašević, and Srećko Joksimović. 2016. The role of achievement goal orientations when studying effect of learning analytics visualizations. In Proceedings of the sixth international conference on learning analytics & knowledge. ACM, 54--63.

Digital Library

[4]

Robert A Bjork, John Dunlosky, and Nate Kornell. 2013. Self-regulated learning: Beliefs, techniques, and illusions. Annual review of psychology 64 (2013), 417--444.

[5]

Robert Bodily and Katrien Verbert. 2017. Review of research on student-facing learning analytics dashboards and educational recommender systems. IEEE Transactions on Learning Technologies (2017).

Digital Library

[6]

Robert Bodily and Katrien Verbert. 2017. Trends and issues in student-facing learning analytics reporting systems research. In Proceedings of the seventh international learning analytics & knowledge conference. ACM, 309--318.

Digital Library

[7]

Analía Cicchinelli, Eduardo Veas, Abelardo Pardo, Viktoria Pammer-Schindler, Angela Fessl, Carla Barreiros, and Stefanie Lindstädt. 2018. Finding traces of self-regulated learning in activity streams. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge. ACM, 191--200.

Digital Library

[8]

William S Cleveland and Robert McGill. 1984. Graphical perception: Theory, experimentation, and application to the development of graphical methods. Journal of the American statistical association 79, 387 (1984), 531--554.

[9]

Eva Durall and Begoña Gros. 2014. Learning Analytics as a Metacognitive Tool. In CSEDU (1). 380--384.

[10]

Erik Duval. 2011. Attention please!: learning analytics for visualization and recommendation. In Proceedings of the 1st international conference on learning analytics and knowledge. ACM, 9--17.

Digital Library

[11]

Andrew J Elliot and Kou Murayama. 2008. On the measurement of achievement goals: Critique, illustration, and application. Journal of Educational Psychology 100, 3 (2008), 613.

[12]

Stephen Few. 2006. Information dashboard design. (2006).

[13]

John Fritz. 2011. Classroom walls that talk: Using online course activity data of successful students to raise self-awareness of underperforming peers. The Internet and Higher Education 14, 2 (2011), 89--97.

[14]

Alexis Gabadinho, Gilbert Ritschard, Matthias Studer, and Nicalas S Müller. 2009. Mining sequence data in R with the TraMineR package: A users guide for version 1.2. Geneva: University of Geneva (2009).

[15]

Donald G Gardner, Larry L Cummings, Randall B Dunham, and Jon L Pierce. 1998. Single-item versus multiple-item measurement scales: An empirical comparison. Educational and psychological measurement 58, 6 (1998), 898--915.

[16]

Dragan Gašević, Shane Dawson, and George Siemens. 2015. Let's not forget: Learning analytics are about learning. TechTrends 59, 1 (2015), 64--71.

[17]

Dragan Gasevic, Jelena Jovanovic, Abelardo Pardo, and Shane Dawson. 2017. Detecting learning strategies with analytics: Links with self-reported measures and academic performance. Journal of Learning Analytics 4, 2 (2017), 113--128.

[18]

Dragan Gašević, Wannisa Matcha, Jelena Jovanović, Abelardo Pardo, Lisa-Angelique Lim, Sheridan Gentili, et al. 2019. Discovering Time Management Strategies in Learning Processes Using Process Mining Techniques. In European Conference on Technology Enhanced Learning. Springer, 555--569.

[19]

Robert J Grissom and John J Kim. 2012. Effect sizes for research: Univariate and multivariate applications. Routledge.

[20]

Marek Hatala and Sanam Shirazi Beheshitha. 2016. Associations Between Students ' Approaches to Learning and Learning Analytics Visualizations. In Proceedings of the LAK 2016 Workshop on Learning Analytics for Learners, Susan Bull, Blandine M. Ginon, Judy Kay, Michael D. Kickmeier-Rust, and Matthew D. Johnson (Eds.), Vol. 1596. CEUR-WS, Edinburgh, Scotland, 3--10. http://ceur-ws.org/Vol-1596/paper1.pdf

[21]

John Hattie and Helen Timperley. 2007. The power of feedback. Review of educational research 77, 1 (2007), 81--112.

[22]

Carl C Haynes, Stephanie D. Teasley, Stephanie Hayley, Meghan Oster, and John Whitmer. 2018. How am I Doing?: Student-Facing Performance Dashboards in Higher Education. In Companion Proceedings of the 8th international conference on learning analytics and knowledge. 274--275.

[23]

Ioana Jivet, Maren Scheffel, Hendrik Drachsler, and Marcus Specht. 2017. Awareness is not enough: pitfalls of learning analytics dashboards in the educational practice. In European Conference on Technology Enhanced Learning. Springer, 82--96.

[24]

Ioana Jivet, Maren Scheffel, Marcus Specht, and Hendrik Drachsler. 2018. License to evaluate: preparing learning analytics dashboards for educational practice. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge. ACM, 31--40.

Digital Library

[25]

Jelena Jovanović, Dragan Gašević, Shane Dawson, Abelardo Pardo, and Negin Mirriahi. 2017. Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher Education 33, 4 (2017), 74--85.

[26]

Steven Lonn, Stephen J Aguilar, and Stephanie D Teasley. 2015. Investigating student motivation in the context of a learning analytics intervention during a summer bridge program. Computers in Human Behavior 47 (2015), 90--97.

Digital Library

[27]

Jock Mackinlay. 1986. Automating the design of graphical presentations of relational information. Acm Transactions On Graphics (Tog) 5, 2 (1986), 110--141.

Digital Library

[28]

Wannisa Matcha, Dragan Gasevic, Abelardo Pardo, et al. 2019. A Systematic Review of Empirical Studies on Learning Analytics Dashboards: A Self-Regulated Learning Perspective. IEEE Transactions on Learning Technologies (2019).

[29]

Miriah Meyer, Michael Sedlmair, P Samuel Quinan, and Tamara Munzner. 2015. The nested blocks and guidelines model. Information Visualization 14, 3 (2015), 234--249.

[30]

Tamara Munzner. 2009. A nested model for visualization design and validation. IEEE transactions on visualization and computer graphics 15, 6 (2009), 921--928.

Digital Library

[31]

Lace M Padilla, Sarah H Creem-Regehr, Mary Hegarty, and Jeanine K Stefanucci. 2018. Decision making with visualizations: a cognitive framework across disciplines. Cognitive research: principles and implications 3, 1 (2018), 29.

[32]

Steven Pinker. 1990. A theory of graph comprehension. Artificial intelligence and the future of testing (1990), 73--126.

[33]

Paul R Pintrich, David AF Smith, Teresa Garcia, and Wilbert J McKeachie. 1993. Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and psychological measurement 53, 3 (1993), 801--813.

[34]

Paul R Pintrich and Akane Zusho. 2002. Student motivation and self-regulated learning in the college classroom. In Higher education: Handbook of theory and research. Springer, 55--128.

[35]

Gabriel Reimers, Anna Neovesky, and Akademie der Wissenschaften. 2015. Student focused dashboards. 7th International Conferenceon Computer Supported Education (Lisbon) (2015).

[36]

Ido Roll and Philip H Winne. 2015. Understanding, evaluating, and supporting self-regulated learning using learning analytics. Journal of Learning Analytics 2, 1 (2015), 7--12.

[37]

Sanne FE Rovers, Geraldine Clarebout, Hans HCM Savelberg, Anique BH de Bruin, and Jeroen JG van Merriënboer. 2019. Granularity matters: comparing different ways of measuring self-regulated learning. Metacognition and Learning 14, 1 (2019), 1--19.

[38]

Richard M Ryan and Edward L Deci. 2000. Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary educational psychology 25, 1 (2000), 54--67.

[39]

Pierangela Samarati and Latanya Sweeney. 1998. Protecting privacy when disclosing information: k-anonymity and its enforcement through generalization and suppression. Technical Report. technical report, SRI International.

[40]

Dale H Schunk and Jeffrey A Greene. 2017. Handbook of self-regulation of learning and performance. Routledge.

[41]

Beat A Schwendimann, Maria Jesus Rodriguez-Triana, Andrii Vozniuk, Luis P Prieto, Mina Shirvani Boroujeni, Adrian Holzer, Denis Gillet, and Pierre Dillenbourg. 2016. Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies 10, 1 (2016), 30--41.

Digital Library

[42]

Beat A Schwendimann, Maria Jesus Rodriguez-Triana, Andrii Vozniuk, Luis P Prieto, Mina Shirvani Boroujeni, Adrian Holzer, Denis Gillet, and Pierre Dillenbourg. 2017. Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies 10, 1 (2017), 30--41.

Digital Library

[43]

Gayane Sedrakyan, Jonna Malmberg, Katrien Verbert, Sanna Järvelä, and Paul A Kirschner. 2018. Linking learning behavior analytics and learning science concepts: designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior (2018).

[44]

Varshita Sher, Marek Hatala, and Dragan Gašević. 2019. On multi-device use: Using technological modality profiles to explain differences in students' learning. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge. ACM, 1--10.

Digital Library

[45]

Stephanie D Teasley. 2017. Student facing dashboards: One size fits all? Technology, Knowledge and Learning 22, 3 (2017), 377--384.

[46]

Katrien Verbert, Erik Duval, Joris Klerkx, Sten Govaerts, and José Luis Santos. 2013. Learning analytics dashboard applications. American Behavioral Scientist 57, 10 (2013), 1500--1509.

[47]

Katrien Verbert, Sten Govaerts, Erik Duval, Jose Luis Santos, Frans Van Assche, Gonzalo Parra, and Joris Klerkx. 2014. Learning dashboards: an overview and future research opportunities. Personal and Ubiquitous Computing 18, 6 (2014), 1499--1514.

Digital Library

[48]

Camilo Vieira, Paul Parsons, and Vetria Byrd. 2018. Visual learning analytics of educational data: A systematic literature review and research agenda. Computers & Education 122 (2018), 119--135.

[49]

Philip H Winne and Dianne Jamieson-Noel. 2002. Exploring students' calibration of self reports about study tactics and achievement. Contemporary Educational Psychology 27, 4 (2002), 551--572.

[50]

Philip H Winne and Nancy E Perry. 2000. Measuring self-regulated learning. In Handbook of self-regulation. Elsevier, 531--566.

[51]

Naomi E Winstone, Robert A Nash, Michael Parker, and James Rowntree. 2017. Supporting Learners' Agentic Engagement With Feedback: A Systematic Review and a Taxonomy of Recipience Processes. Educational Psychologist 52, 1 (2017), 17--37.

[52]

JR Young. 2016. What clicks from 70,000 courses reveal about student learning. Chronicle of Higher Education 63, 3 (2016).

[53]

Mingming Zhou and Philip H Winne. 2012. Modeling academic achievement by self-reported versus traced goal orientation. Learning and Instruction 22, 6 (2012), 413--419.

Cited By

صالح ق(2024)الكفاءة الذاتية لمدرسي التربية الرياضية في إدارة الصف وعلاقتها بالأنماط السلوكية المنظمة لطلاب المدراس الإعدادية في محافظة بابل أثناء درس الرياضة من وجهة نظرهممجلة دراسات وبحوث التربية الرياضية10.55998/jsrse.v34i1.454(131-148)Online publication date: 14-Jan-2024
https://doi.org/10.55998/jsrse.v34i1.454
Cheng NZhao WXu XLiu HTao J(2024)The influence of learning analytics dashboard information design on cognitive load and performanceEducation and Information Technologies10.1007/s10639-024-12606-129:15(19729-19752)Online publication date: 1-Apr-2024
https://doi.org/10.1007/s10639-024-12606-1
Paulsen LLindsay E(2024)Learning analytics dashboards are increasingly becoming about learning and not just analytics - A systematic reviewEducation and Information Technologies10.1007/s10639-023-12401-429:11(14279-14308)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s10639-023-12401-4
Show More Cited By

Index Terms

How patterns of students dashboard use are related to their achievement and self-regulatory engagement
1. Applied computing
  1. Education
    1. Interactive learning environments
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
  2. Visualization
    1. Visualization application domains
      1. Information visualization

Recommendations

Examining change in students’ self-regulated learning patterns after a formative assessment using process mining techniques
Abstract
The growing popularity of e-learning platforms, such as learning management systems, has foregrounded the role of self-regulated learning (SRL) in student success. In many e-learning environments, students typically complete learning assignments ...
Highlights
- Students engage in a loosely sequenced recursive self-regulated learning cycle.
- Students actively adjust self-regulated learning patterns in response to a formative assessment.
- There are group differences in the adjustment of self-...
Promoting self-regulated learning in web-based learning environments

Self-regulated learning with the Internet or hypermedia requires not only cognitive learning strategies, but also specific and general meta-cognitive strategies. The purposes of the Study2000 project, carried out at the TU Dresden, were to develop and ...
The use of Edpuzzle to support low-achiever's development of self-regulated learning and their learning of chemistry
ICETC '18: Proceedings of the 10th International Conference on Education Technology and Computers

Education professionals have identified that self-regulated learning skills is critical in fostering lifelong learning. Being academically at risk students, low-achievers can benefit academically if they develop self-regulated learning skills. This ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

LAK '20: Proceedings of the Tenth International Conference on Learning Analytics & Knowledge

March 2020

679 pages

ISBN:9781450377126

DOI:10.1145/3375462

General Chairs:
Christoph Rensing
Technische Universität Darmstadt, Germany
,
Hendrik Drachsler
DIPF Leibniz Institute for Research and Information in Education & University of Frankfurt, Germany
,
Program Chairs:
Vitomir Kovanović
University of South Australia, Australia
,
Niels Pinkwart
Humboldt-Universität zu Berlin, Germany
,
Maren Scheffel
Open Universiteit, Netherlands
,
Katrien Verbert
KU Leuven, Belgium

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 March 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

LAK '20

LAK '20: 10th International Conference on Learning Analytics and Knowledge

March 23 - 27, 2020

Frankfurt, Germany

Acceptance Rates

LAK '20 Paper Acceptance Rate 80 of 261 submissions, 31%;

Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
1,259
Total Downloads

Downloads (Last 12 months)151
Downloads (Last 6 weeks)15

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

صالح ق(2024)الكفاءة الذاتية لمدرسي التربية الرياضية في إدارة الصف وعلاقتها بالأنماط السلوكية المنظمة لطلاب المدراس الإعدادية في محافظة بابل أثناء درس الرياضة من وجهة نظرهممجلة دراسات وبحوث التربية الرياضية10.55998/jsrse.v34i1.454(131-148)Online publication date: 14-Jan-2024
https://doi.org/10.55998/jsrse.v34i1.454
Cheng NZhao WXu XLiu HTao J(2024)The influence of learning analytics dashboard information design on cognitive load and performanceEducation and Information Technologies10.1007/s10639-024-12606-129:15(19729-19752)Online publication date: 1-Apr-2024
https://doi.org/10.1007/s10639-024-12606-1
Paulsen LLindsay E(2024)Learning analytics dashboards are increasingly becoming about learning and not just analytics - A systematic reviewEducation and Information Technologies10.1007/s10639-023-12401-429:11(14279-14308)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s10639-023-12401-4
Tanaka TUeda M(2024)Predicting the Academic Performance of Programming Students Using Logs from an Online Learning System: Toward Providing Timely Guidance and Feedback to StudentsArtificial Intelligence for Supporting Human Cognition and Exploratory Learning in the Digital Age10.1007/978-3-031-66462-5_8(139-157)Online publication date: 9-Aug-2024
https://doi.org/10.1007/978-3-031-66462-5_8
Gujju KBandi SMoraes M(2024)Review of Learning Analytics Dashboards for StudentsProceedings of the Third International Conference on Innovations in Computing Research (ICR’24)10.1007/978-3-031-65522-7_27(300-312)Online publication date: 1-Aug-2024
https://doi.org/10.1007/978-3-031-65522-7_27
Sahin MIfenthaler D(2024)Foundations of Assessment AnalyticsAssessment Analytics in Education10.1007/978-3-031-56365-2_1(3-17)Online publication date: 8-May-2024
https://doi.org/10.1007/978-3-031-56365-2_1
Aghaei KHatala MMogharrab AHilliger IKhosravi HRienties BDawson S(2023)How Students’ Emotion and Motivation Changes After Viewing Dashboards with Varied Social Comparison Group: A Qualitative StudyLAK23: 13th International Learning Analytics and Knowledge Conference10.1145/3576050.3576107(663-669)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3576050.3576107
Martínez ÁMartinell A(2023)Application of Learning Analytics in Higher Education Institutions: A Systematic Literature Review2023 11th International Conference in Software Engineering Research and Innovation (CONISOFT)10.1109/CONISOFT58849.2023.00024(116-123)Online publication date: 6-Nov-2023
https://doi.org/10.1109/CONISOFT58849.2023.00024
Motz BMorrone A(2023)Wild brooms and learning analyticsJournal of Computing in Higher Education10.1007/s12528-023-09353-636:1(145-153)Online publication date: 24-Jan-2023
https://doi.org/10.1007/s12528-023-09353-6
Al-Shaikhli D(2023)The effect of the tracking technology on students’ perceptions of their continuing intention to use a learning management systemEducation and Information Technologies10.1007/s10639-022-11156-828:1(343-371)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1007/s10639-022-11156-8
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents