Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3636555.3636852acmotherconferencesArticle/Chapter ViewAbstractPublication PageslakConference Proceedingsconference-collections
research-article

Towards Comprehensive Monitoring of Graduate Attribute Development: A Learning Analytics Approach in Higher Education

Published: 18 March 2024 Publication History

Abstract

In response to the evolving demands of the contemporary workplace, higher education (HE) institutions are increasingly emphasising the development of transversal skills and graduate attributes (GAs). The development of GAs, such as effective communication, collaboration, and lifelong learning, are non-linear and follow distinct trajectories for individual learners. The ability to trace and measure the progression of GA remains a significant challenge. While previous studies have focused on empirical methods for measuring GAs in individual courses, a notable gap exists in understanding their longitudinal development within HE programs. To address this research gap, our study focuses on measuring and tracing the development of GAs in an Initial Teacher Education (ITE) undergraduate program at a large public university in Australia. By combining learning analytics (LA) with psychometric models, we analysed students’ assessment grades to measure learners’ GA development in each year of the ITE program. The resulting measurements enabled the identification of distinct profiles of GA attainment, as demonstrated by learners and their distinct pathways. The overall approach allows for a comprehensive representation of a learner's progress throughout the program of study. As such, the developed approach sets the grounds for more personalised learning support, program evaluation, and improvement of students’ GA attainment.

References

[1]
Abelha, M., Fernandes, S., Mesquita, D., Seabra, F. and Ferreira-Oliveira, A.T. 2020. Graduate Employability and Competence Development in Higher Education—A Systematic Literature Review Using PRISMA. Sustainability. 12, 15 (Jan. 2020), 5900.
[2]
Ajjawi, R. and Boud, D. 2023. Changing representations of student achievement: The need for innovation. Innovations in Education and Teaching International. 0, 0 (Mar. 2023), 1–11.
[3]
Barrie, S., Hughes, C. and Smith, C. 2009. The national graduate attributes project: Integration and assessment of graduate attributes in curriculum. Sydney: Australian Learning and Teaching Council. (2009).
[4]
Barthakur, A., Dawson, S. and Kovanovic, V. 2023. Advancing leaner profiles with learning analytics: A scoping review of current trends and challenges. Proceedings of the 13th International Conference on Learning Analytics & Knowledge (LAK’23) (New York, NY, USA, 2023), 606–612.
[5]
Barthakur, A., Joksimovic, S., Kovanovic, V., Corbett, F.C., Richey, M. and Pardo, A. 2022. Assessing the sequencing of learning objectives in a study program using evidence-based practice. Assessment & Evaluation in Higher Education. 47, 8 (Apr. 2022), 1429–1443.
[6]
Bennett, D. 2018. Graduate employability and higher education: Past, present and future. HERDSA Review of Higher Education. 5, (2018), 31–61.
[7]
Bouslimani, Y., Durand, G. and Belacel, N. 2016. EDUCATIONAL DATA MINING APPROACH FOR ENGINEERING GRADUATE ATTRIBUTES ANALYSIS. Proceedings of the Canadian Engineering Education Association (CEEA). (2016).
[8]
Brawley, S., Clark, J., Dixon, C., Ford, L., Ross, S., Upton, S. and Nielsen, E. 2013. Learning outcomes assessment and history: TEQSA, the After Standards Project and the QA/QI challenge in Australia. Arts and Humanities in Higher Education. 12, 1 (2013), 20–35.
[9]
Bullock, K. 2011. International Baccalaureate learner profile: Literature review.
[10]
Charrad, M., Ghazzali, N., Boiteau, V. and Niknafs, A. 2014. NbClust: An R Package for Determining the Relevant Number of Clusters in a Data Set. Journal of Statistical Software. 61, (Nov. 2014), 1–36.
[11]
Clifford, V. and Montgomery, C. 2017. Designing an internationationalised curriculum for higher education: embracing the local and the global citizen. Higher Education Research & Development. 36, 6 (Sep. 2017), 1138–1151.
[12]
Coetzee, M. 2014. Measuring student graduateness: Reliability and construct validity of the Graduate Skills and Attributes Scale. Higher Education Research & Development. 33, 5 (2014), 887–902.
[13]
Dunn†, J.C. 1974. Well-Separated Clusters and Optimal Fuzzy Partitions. Journal of Cybernetics. 4, 1 (Jan. 1974), 95–104.
[14]
ElAtia, S., Ipperciel, D., Zaiane, O., Bakhshinategh, B. and Thibaudeau, P. 2020. Graduate attributes assessment program. The International Journal of Information and Learning Technology. 38, 1 (2020), 117–134.
[15]
Fallows, S. and Steven, C. 2013. Integrating key skills in higher education: Employability, transferable skills and learning for life. Routledge.
[16]
George, A.C., Robitzsch, A., Kiefer, T., Groß, J. and Ünlü, A. 2016. The R Package CDM for Cognitive Diagnosis Models. Journal of Statistical Software. 74, 2 (2016).
[17]
Green, W., Hammer, S. and Star, C. 2009. Facing up to the challenge: why is it so hard to develop graduate attributes? Higher Education Research & Development. 28, 1 (Mar. 2009), 17–29.
[18]
Halibas, A.S., Mehtab, S., Al-Attili, A., Alo, B., Cordova, R. and Cruz, M.E.L.T. 2020. A thematic analysis of the quality audit reports in developing a framework for assessing the achievement of the graduate attributes. International Journal of Educational Management. 34, 5 (2020), 917–935.
[19]
Hammer, S., Ayriss, P. and McCubbin, A. 2021. Style or substance: how Australian universities contextualise their graduate attributes for the curriculum quality space. Higher Education Research & Development. 40, 3 (Apr. 2021), 508–523.
[20]
Haste, H. 2001. Ambiguity, autonomy and agency: psychological challenges to new competence. Defining and Selecting Key Competencies. (2001), 93–120.
[21]
Hill, J., Walkington, H. and France, D. 2016. Graduate attributes: implications for higher education practice and policy. Journal of Geography in Higher Education. 40, 2 (Apr. 2016), 155–163.
[22]
Huang, W., Lou, H. and Day, J. 2006. Is a high GPA still the most important factor for job opportunity? - An empirical investigation. Issues in Information Systems. VII, 1 (2006), 352–356.
[23]
Hughes, C. and Barrie, S. 2010. Influences on the assessment of graduate attributes in higher education. Assessment & Evaluation in Higher Education. 35, 3 (May 2010), 325–334.
[24]
Ipperciel, D. and ElAtia, S. 2014. Assessing Graduate Attributes: Building a Criteria-Based Competency Model. International Journal of Higher Education. 3, 3 (2014), 27–38.
[25]
Joksimovic, S., Siemens, G., Wang, Y.E., San Pedro, M.O.Z. and Way, J. 2020. Editorial: Beyond Cognitive Ability. Journal of Learning Analytics. 7, 1 (Apr. 2020), 1-4-1–4.
[26]
Kaupp, J., Frank, B., Brennan, R., McCahan, S., Narayanan, L., Ostafichuck, P., Sepehri, N. and Watts, K.C. 2012. A comparison of institutional approaches to CEAB graduate attribute requirements. Proceedings of the Canadian Engineering Education Association (CEEA). (2012).
[27]
Khosravi, H., Sadiq, S. and Amer-Yahia, S. 2023. Data management of AI-powered education technologies: Challenges and opportunities. Learning Letters. (2023).
[28]
Ma, C., Ouyang, J. and Xu, G. 2023. Learning Latent and Hierarchical Structures in Cognitive Diagnosis Models. Psychometrika. 88, 1 (Mar. 2023), 175–207.
[29]
Ma, C., de la Torre, J. and Xu, G. 2023. Bridging Parametric and Nonparametric Methods in Cognitive Diagnosis. Psychometrika. 88, 1 (Mar. 2023), 51–75.
[30]
Mahon, D. 2022. The Role of Graduate Attributes in Higher Education. A Review of the Issues Associated with Graduate Attributes and the case for their Measurement. Interchange. 53, 3–4 (2022), 353–369.
[31]
Malviya, T., Jain, S., Chaudhari, A. and Kotecha, R. 2020. Enhancing Attainment of Graduate Attributes through Data Mining.
[32]
Milligan, G.W. and Cooper, M.C. 1985. An examination of procedures for determining the number of clusters in a data set. Psychometrika. 50, 2 (Jun. 1985), 159–179.
[33]
Nunan, T., George, R. and McCausland, H. 2000. Implementing graduate skills at an Australian university. Integrating Key Skills in Higher Education. Routledge. 10.
[34]
O'Connell, M., Milligan, S. and Bentley, T. 2019. Beyond ATAR: a proposal for change. Koshland Innovation Fund.
[35]
Oliver, B. 2013. Graduate attributes as a focus for institution-wide curriculum renewal: innovations and challenges. Higher Education Research & Development. 32, 3 (Jun. 2013), 450–463.
[36]
Oliver, B. and Jorre de St Jorre, T. 2018. Graduate attributes for 2020 and beyond: recommendations for Australian higher education providers. Higher Education Research & Development. 37, 4 (Jun. 2018), 821–836.
[37]
Oliver, B., Tucker, B., Gupta, R. and Yeo, S. 2008. e VALUate: an evaluation instrument for measuring students’ perceptions of their engagement and learning outcomes. Assessment & Evaluation in Higher Education. 33, 6 (2008), 619–630.
[38]
Osmani, M., Weerakkody, V., Hindi, N.M., Al‐Esmail, R., Eldabi, T., Kapoor, K. and Irani, Z. 2015. Identifying the trends and impact of graduate attributes on employability: a literature review. Tertiary Education and Management. 21, 4 (Oct. 2015), 367–379.
[39]
Paek, S., Leong, P., Johnson, P. and Moore, C. 2021. Is GPA Enough? A Platform for Promoting Computer Science Undergraduates’ Pursuit of Career Related Extracurricular Activities. International Journal of Technology in Education and Science. 5, 1 (2021), 1–16.
[40]
Pandey, R.K., Obhan, J. and Kotecha, R. 2023. DATA MINING FOR ENHANCEMENT OF GRADUATE ATTRIBUTES. ICTACT Journal on Soft Computing. 13, 2 (2023).
[41]
Pellegrino, J.W. 2017. Teaching, learning and assessing 21st century skills. (Feb. 2017), 223–251.
[42]
Poquet, O., Kitto, K., Jovanovic, J., Dawson, S., Siemens, G. and Markauskaite, L. 2021. Transitions through lifelong learning: Implications for learning analytics. Computers and Education: Artificial Intelligence. 2, (Jan. 2021), 100039.
[43]
Qiu, W. and Joe, H. 2006. Separation index and partial membership for clustering. Computational Statistics & Data Analysis. 50, 3 (Feb. 2006), 585–603.
[44]
Quinlan, K.M. 2016. Developing student character through disciplinary curricula: an analysis of UK QAA subject benchmark statements. Studies in Higher Education. 41, 6 (Jun. 2016), 1041–1054.
[45]
Saqr, M., López-Pernas, S., Jovanović, J. and Gašević, D. 2023. Intense, turbulent, or wallowing in the mire: A longitudinal study of cross-course online tactics, strategies, and trajectories. The Internet and Higher Education. 57, (Apr. 2023), 100902.
[46]
Shum, S.B. and Crick, R.D. 2016. Learning Analytics for 21st Century Competencies. Journal of Learning Analytics. 3, 2 (Sep. 2016), 6–21.
[47]
Suleman, F. 2018. The employability skills of higher education graduates: insights into conceptual frameworks and methodological options. Higher Education. 76, 2 (Aug. 2018), 263–278.
[48]
Von Davier, M. and Lee, Y.-S. eds. 2019. Handbook of Diagnostic Classification Models: Models and Model Extensions, Applications, Software Packages. Springer International Publishing.
[49]
Wakelin, B. and Hanrahan, H. 2014. 25 Years of the Washington Accord. International Engineering Alliance.
[50]
Ward, J.H. 1963. Hierarchical Grouping to Optimize an Objective Function. Journal of the American Statistical Association. 58, 301 (1963), 236–244.
[51]
Winne, P.H. 2022. Modeling self-regulated learning as learners doing learning science: How trace data and learning analytics help develop skills for self-regulated learning. Metacognition and Learning. 17, 3 (Dec. 2022), 773–791.
[52]
Yorke, M. and Harvey, L. 2005. Graduate attributes and their development. New directions for institutional research. 2005, 128 (2005), 41–58.
[53]
Zhang, Y., Gong, Y. and Cui, W. 2022. Graduate Attribute Assessment for Each Student of Remote Sensing Science and Technology Program. 2022 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE) (Dec. 2022), 460–465.

Cited By

View all
  • (2024)A Systematic Review of Studies on Decision-Making Systems for Teaching and Learning in K-12Technology Enhanced Learning for Inclusive and Equitable Quality Education10.1007/978-3-031-72315-5_4(49-63)Online publication date: 16-Sep-2024

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences
LAK '24: Proceedings of the 14th Learning Analytics and Knowledge Conference
March 2024
962 pages
ISBN:9798400716188
DOI:10.1145/3636555
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 March 2024

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Higher education
  2. graduate attributes
  3. learner profile
  4. monitoring progression

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

LAK '24

Acceptance Rates

Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)153
  • Downloads (Last 6 weeks)15
Reflects downloads up to 13 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)A Systematic Review of Studies on Decision-Making Systems for Teaching and Learning in K-12Technology Enhanced Learning for Inclusive and Equitable Quality Education10.1007/978-3-031-72315-5_4(49-63)Online publication date: 16-Sep-2024

View Options

Get Access

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media