Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Where Is the Learning in Learning Analytics?

A Systematic Literature Review to Identify Measures of Affected Learning

  • Conference paper
  • First Online:
Lifelong Technology-Enhanced Learning (EC-TEL 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11082))

Included in the following conference series:

Abstract

Learning analytics is the analysis and visualization of student data with the purpose of improving education. Literature reporting on measures of the effects of data-driven pedagogical interventions on learning and the environment in which this takes place, allows us to assess in what way learning analytics actually improves learning. We conducted a systematic literature review aimed at identifying such measures of data-driven improvement. A review of 1034 papers yielded 38 key studies, which were thoroughly analyzed on aspects like objective, affected learning and their operationalization (measures). Based on prevalent learning theories, we synthesized a classification scheme comprised of four categories: learning process, student performance, learning environment, and departmental performance. Most of the analyzed studies relate to either student performance or learning process. Based on the results, we recommend to make deliberate decisions on the (multiple) aspects of learning one tries to improve by the application of learning analytics. Our classification scheme with examples of measures may help both academics and practitioners doing so, as it allows for structured positioning of learning analytics benefits.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Akçapınar, G.: How automated feedback through text mining changes plagiaristic behavior in online assignments. Comput. Educ. 87, 123–130 (2015)

    Article  Google Scholar 

  2. Arguedas, M., Daradoumis, T., Xhafa, F.: Analyzing the effects of emotion management on time and self-management in computer-based learning. Comput. Hum. Behav. 63, 517–529 (2016)

    Article  Google Scholar 

  3. Arnold, K.E., Pistilli, M.D.: Course signals at purdue: using learning analytics to increase student success. In: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, pp. 267–270. ACM (2012)

    Google Scholar 

  4. Beheshitha, S.S., Hatala, M., Gašević, D., Joksimović, S.: The role of achievement goal orientations when studying effect of learning analytics visualizations. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, pp. 54–63. ACM (2016)

    Google Scholar 

  5. Berland, M., Davis, D., Smith, C.P.: AMOEBA: designing for collaboration in computer science classrooms through live learning analytics. Int. J. Comput. Support. Collab. Learn. 10(4), 425–447 (2015)

    Article  Google Scholar 

  6. Biggs, J.B., Telfer, R.: The Process of Learning. McGraw-Hill/Appleton & Lange (1987)

    Google Scholar 

  7. Bodily, R., Verbert, K.: Trends and issues in student-facing learning analytics reporting systems research. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 309–318. ACM (2017)

    Google Scholar 

  8. Braungart, M., Braungart, R.: Applying learning theories to healthcare practice (2007)

    Google Scholar 

  9. Cheng, H., Liao, W.: Establishing an lifelong learning environment using IOT and learning analytics. In: 14th International Conference on Advanced Communication Technology (ICACT), pp. 1178–1183. IEEE (2012)

    Google Scholar 

  10. Clow, D.: The learning analytics cycle: closing the loop effectively. In: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, pp. 134–138. ACM (2012)

    Google Scholar 

  11. David, Y.B., Segal, A., Gal, Y.K.: Sequencing educational content in class rooms using Bayesian knowledge tracing. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, pp. 354–363. ACM (2016)

    Google Scholar 

  12. Davis, D., Jivet, I., Kizilcec, R.F., Chen, G., Hauff, C., Houben, G.: Follow the successful crowd: raising MOOC completion rates through social comparison at scale. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 454–463 (2017)

    Google Scholar 

  13. Dawson, S., Jovanovic, J., Gašević, D., Pardo, A.: From prediction to impact: evaluation of a learning analytics retention program. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 474–478. ACM (2017)

    Google Scholar 

  14. DePaul: Direct versus indirect assessment of student learning (n.d.). https://resources.depaul.edu/teaching-commons/teaching-guides/feedback-grading/Pages/direct-assessment.aspx. Accessed 28 Apr 2018

  15. Diana, N., Eagle, M., Stamper, J.C., Grover, S., Bienkowski, M.A., Basu, S.: An instructor dashboard for real-time analytics in interactive programming assignments. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 272–279 (2017)

    Google Scholar 

  16. Gašević, D., Dawson, S., Siemens, G.: Let’s not forget: learning analytics are about learning. TechTrends 59(1), 64–71 (2015)

    Article  Google Scholar 

  17. Grann, J., Bushway, D.: Competency map: visualizing student learning to promote student success. In: Proceedings of the Fourth International Conference on Learning Analytics and Knowledge, pp. 168–172. ACM (2014)

    Google Scholar 

  18. Groba, A.R., Barreiros, B.V., Lama, M., Gewerc, A., Mucientes, M.: Using a learning analytics tool for evaluation in self-regulated learning. In: Frontiers in Education Conference (FIE), pp. 1–8. IEEE (2014)

    Google Scholar 

  19. Harrison, S., Villano, R., Lynch, G., Chen, G.: Measuring financial implications of an early alert system. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, pp. 241–248. ACM (2016)

    Google Scholar 

  20. Herodotou, C., Rienties, B., Boroowa, A., Zdrahal, Z., Hlosta, M., Naydenova, G.: Implementing predictive learning analytics on a large scale: the teacher’s perspective. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 267–271. ACM (2017)

    Google Scholar 

  21. Holman, C., Aguilar, S.J., Levick, A., Stern, J., Plummer, B., Fishman, B.: Planning for success: how students use a grade prediction tool to win their classes. In: Proceedings of the Fifth International Conference on Learning Analytics and Knowledge, pp. 260–264. ACM (2015)

    Google Scholar 

  22. Jayaprakash, S.M., Moody, E.W., Lauría, E.J., Regan, J.R., Baron, J.D.: Early alert of academically at-risk students: an open source analytics initiative (2014)

    Article  Google Scholar 

  23. Khan, I., Pardo, A.: Data2U: scalable real time student feedback in active learning environments. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, pp. 249–253. ACM (2016)

    Google Scholar 

  24. Kolb, D.: Experiential Learning as the Science of Learning and Development. Prentice Hall, Englewood Cliffs (1984)

    Google Scholar 

  25. Kumar, V., Boulanger, D., Seanosky, J., Panneerselvam, K., Somasundaram, T.S.: Competence analytics. J. Comput. Educ. 1(4), 251–270 (2014)

    Article  Google Scholar 

  26. Lauría, E.J., Moody, E.W., Jayaprakash, S.M., Jonnalagadda, N., Baron, J.D.: Open academic analytics initiative: initial research findings. In: Proceedings of the Third International Conference on Learning Analytics and Knowledge, pp. 150–154. ACM (2013)

    Google Scholar 

  27. Lonn, S., Aguilar, S.J., Teasley, S.D.: Investigating student motivation in the context of a learning analytics intervention during a summer bridge program. Comput. Hum. Behav. 47, 90–97 (2015)

    Article  Google Scholar 

  28. Manske, S., Hoppe, H.U.: The “Concept Cloud”: supporting collaborative knowledge construction based on semantic extraction from learner-generated artefacts. In: IEEE 16th International Conference on Advanced Learning Technologies (ICALT), pp. 302–306. IEEE (2016)

    Google Scholar 

  29. Marcos-García, J., Martínez-Monés, A., Dimitriadis, Y.: DESPRO: a method based on roles to provide collaboration analysis support adapted to the participants in CSCL situations. Comput. Educ. 82, 335–353 (2015)

    Article  Google Scholar 

  30. Martinez-Maldonado, R., Yacef, K., Kay, J.: TSCL: a conceptual model to inform understanding of collaborative learning processes at interactive tabletops. Int. J. Hum Comput Stud. 83, 62–82 (2015)

    Article  Google Scholar 

  31. McKenzie, W.A., Perini, E., Rohlf, V., Toukhsati, S., Conduit, R., Sanson, G.: A blended learning lecture delivery model for large and diverse undergraduate cohorts. Comput. Educ. 64, 116–126 (2013). https://doi.org/10.1016/j.compedu.2013.01.009

    Article  Google Scholar 

  32. Melero, J., Hernández-Leo, D., Sun, J., Santos, P., Blat, J.: How was the activity? A visualization support for a case of location-based learning design. Br. J. Edu. Technol. 46(2), 317–329 (2015)

    Article  Google Scholar 

  33. Nunn, S., Avella, J.T., Kanai, T., Kebritchi, M.: Learning analytics methods, benefits, and challenges in higher education: a systematic literature review. Online Learn. 20, 2 (2016)

    Article  Google Scholar 

  34. Nussbaumer, A., Hillemann, E., Gütl, C., Albert, D.: A competence-based service for supporting self-regulated learning in virtual environments. J. Learn. Anal. 2(1), 101–133 (2015)

    Article  Google Scholar 

  35. Papamitsiou, Z.K., Economides, A.A.: Learning analytics and educational data mining in practice: a systematic literature review of empirical evidence. Educ. Technol. Soc. 17(4), 49–64 (2014)

    Google Scholar 

  36. Papoušek, J., Stanislav, V., Pelánek, R.: Evaluation of an adaptive practice system for learning geography facts. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, pp. 134–142. ACM (2016)

    Google Scholar 

  37. Perikos, I., Grivokostopoulou, F., Hatzilygeroudis, I.: Assistance and feedback mechanism in an intelligent tutoring system for teaching conversion of natural language into logic. Int. J. Artif. Intell. Educ. 27(3), 475–514 (2017)

    Article  Google Scholar 

  38. Rome, M.: Best practices in student learning and assessment: creating and implementing effective assessment for NYU schools, departments and programs (2011)

    Google Scholar 

  39. Ruiz-Calleja, A., Prieto, L.P., Ley, T., Rodríguez-Triana, M.J., Dennerlein, S.: Learning analytics for professional and workplace learning: a literature review. In: Lavoué, É., Drachsler, H., Verbert, K., Broisin, J., Pérez-Sanagustín, M. (eds.) EC-TEL 2017. LNCS, vol. 10474, pp. 164–178. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66610-5_13

    Chapter  Google Scholar 

  40. Sharma, K., Alavi, H.S., Jermann, P., Dillenbourg, P.: A gaze-based learning analytics model: in-video visual feedback to improve learner’s attention in MOOCs. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, pp. 417–421. ACM (2016)

    Google Scholar 

  41. Siadaty, M., Gašević, D., Hatala, M.: Associations between technological scaffolding and micro-level processes of self-regulated learning: a workplace study. Comput. Hum. Behav. 55, 1007–1019 (2016)

    Article  Google Scholar 

  42. Siadaty, M., Gašević, D., Hatala, M.: Measuring the impact of technological scaffolding interventions on micro-level processes of self-regulated workplace learning. Comput. Hum. Behav. 59, 469–482 (2016). https://doi.org/10.1016/j.chb.2016.02.025

    Article  Google Scholar 

  43. Siemens, G., Gasevic, D., Haythornthwaite, C., Dawson, S.P., Shum, S., Ferguson, R., Duval, E., Verbert, K., Baker, R.: Open learning analytics: an integrated & modularized platform (2011)

    Google Scholar 

  44. Siemens, G., Long, P.: Penetrating the fog: analytics in learning and education. EDUCAUSE Rev. 46(5), 30 (2011)

    Google Scholar 

  45. Smolin, D., Butakov, S.: Applying artificial intelligence to the educational data: an example of syllabus quality analysis. In: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, pp. 164–169. ACM (2012)

    Google Scholar 

  46. Steinmayr, R., Meißner, A., Weidinger, A.F., Wirthwein, L.: Academic achievement. Oxford Bibliographies (2014). https://doi.org/10.1093/obo/9780199756810-0108

  47. Tabuenca, B., Kalz, M., Drachsler, H., Specht, M.: Time will tell: the role of mobile learning analytics in self-regulated learning. Comput. Educ. 89, 53–74 (2015)

    Article  Google Scholar 

  48. Van Barneveld, A., Arnold, K.E., Campbell, J.P.: Analytics in higher education: establishing a common language. EDUCAUSE Learn. Initiative 1, 1–11 (2012)

    Google Scholar 

  49. Van Leeuwen, A., Janssen, J., Erkens, G., Brekelmans, M.: Teacher regulation of cognitive activities during student collaboration: effects of learning analytics. Comput. Educ. 90, 80–94 (2015)

    Article  Google Scholar 

  50. Van Leeuwen, A., Janssen, J., Erkens, G., Brekelmans, M.: Supporting teachers in guiding collaborating students: effects of learning analytics in CSCL. Comput. Educ. 79, 28–39 (2014)

    Article  Google Scholar 

  51. Whitelock, D., Twiner, A., Richardson, J.T., Field, D., Pulman, S.: OpenEssayist: a supply and demand learning analytics tool for drafting academic essays. In: Proceedings of the Fifth International Conference on Learning Analytics and Knowledge, pp. 208–212. ACM (2015)

    Google Scholar 

  52. Wise, A.F., Zhao, Y., Hausknecht, S.N.: Learning analytics for online discussions: embedded and extracted approaches. J. Learn. Anal. 1(2), 48–71 (2014)

    Article  Google Scholar 

  53. Xiong, X., Wang, Y., Beck, J.B.: Improving students’ long-term retention performance: a study on personalized retention schedules. In: Proceedings of the Fifth International Conference on Learning Analytics and Knowledge, pp. 325–329. ACM (2015)

    Google Scholar 

  54. Yamada, M., Kitamura, S., Matsukawa, H., Misono, T., Kitani, N., Yamauchi, Y.: Collaborative filtering for expansion of learner’s background knowledge in online language learning: does “top-down” processing improve vocabulary proficiency? Educ. Technol. Res. Dev. 62(5), 529–553 (2014)

    Article  Google Scholar 

  55. Yassine, S., Kadry, S., Sicilia, M.: A framework for learning analytics in moodle for assessing course outcomes. In: IEEE Global Engineering Education Conference (EDUCON), pp. 261–266. IEEE (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Research Centre for Learning and Innovation, HU University of Applied Sciences, P.O. Box 182, 3500 AD, Utrecht, The Netherlands

    Justian Knobbout & Esther van der Stappen

Authors
  1. Justian Knobbout
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Esther van der Stappen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Justian Knobbout .

Editor information

Editors and Affiliations

  1. Graz University of Technology, Graz, Austria

    Viktoria Pammer-Schindler

  2. Pontificia Universidad Católica de Chile, Providencia, Santiago de Chile, Chile

    Mar Pérez-Sanagustín

  3. DIPF | Leibniz Institute for Research and Information in Education, Frankfurt, Germany

    Hendrik Drachsler

  4. RayCom BV, Utrecht, Utrecht, The Netherlands

    Raymond Elferink

  5. Open University Netherlands, Heerlen, The Netherlands

    Maren Scheffel

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Knobbout, J., van der Stappen, E. (2018). Where Is the Learning in Learning Analytics?. In: Pammer-Schindler, V., Pérez-Sanagustín, M., Drachsler, H., Elferink, R., Scheffel, M. (eds) Lifelong Technology-Enhanced Learning. EC-TEL 2018. Lecture Notes in Computer Science(), vol 11082. Springer, Cham. https://doi.org/10.1007/978-3-319-98572-5_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-98572-5_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-98571-8

  • Online ISBN: 978-3-319-98572-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation