Using attention-based neural networks for predicting student learning outcomes in service-learning
Authors: 
Eugene Yujun Fu, 
Grace Ngai, Hong Va Leong, Stephen C.F. Chan, Daniel T.L. ShekAuthors Info & Claims
Pages 13763 - 13789
Abstract
As a high-impact educational practice, service-learning has demonstrated success in positively influencing students’ overall development, and much work has been done on investigating student learning outcomes from service-learning. A particular direction is to model students’ learning outcomes in the context of their learning experience, i.e., the various student, course, and pedagogical elements. It contributes to a better understanding of the learning process, a more accurate prediction of students’ attainments on the learning outcomes, and improvements in the design of learning activities to maximize student learning. However, most of the existing work in this area relies on statistical analysis that makes assumptions about attribute independence or simple linear dependence, which may not accurately reflect real-life scenarios. In contrast, the study described in this paper adopted a neural network-based approach to investigate the impact of students’ learning experience on different service-learning outcomes. A neural network with attention mechanisms was constructed to predict students’ service-learning outcomes by modeling the contextual information from their various learning experiences. In-depth evaluation experiments on a large-scale dataset collected from more than 10,000 students showed that this proposed model achieved better accuracy on predicting service-learning outcomes. More importantly, it could capture the interdependence between different aspects of student learning experience and the learning outcomes. We believe that this framework can be extended to student modeling for other types of learning activities.
References
[1]
Asif R, Merceron A, Ali SA, and Haider NG Analyzing undergraduate students’ performance using educational data mining Computers & Education 2017 113 177-194
[2]
Astin, A.W., Vogelgesang, L.J., Ikeda, E.K., & Yee, J.A. (2000). How service learning affects students. Higher Education, 144.
[3]
Azcona D, Hsiao IH, and Smeaton AF Detecting students-at-risk in computer programming classes with learning analytics from students’ digital footprints User Modeling and User-Adapted Interaction 2019 29 4 759-788
[4]
Ba, J., Mnih, V., & Kavukcuoglu, K. (2014). Multiple object recognition with visual attention. arXiv:14127755.
[5]
Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv:14090473.
[6]
Bosch, N. (2021). Identifying supportive student factors for mindset interventions: A two-model machine learning approach. Computers & Education, 104190.
[7]
Brinton, C.G., & Chiang, M. (2015). MOOC performance prediction via clickstream data and social learning networks. In 2015 IEEE conference on computer communications (INFOCOM) (pp. 2299–2307). IEEE.
[8]
Celio CI, Durlak J, and Dymnicki A A meta-analysis of the impact of service-learning on students Journal of Experiential Education 2011 34 2 164-181
[9]
Chan SC, Ngai G, and Kp Kwan Mandatory service learning at university: Do less-inclined students learn from it? Active Learning in Higher Education 2019 20 3 189-202
[10]
Chen SC, She HC, Chuang MH, Wu JY, Tsai JL, and Jung TP Eye movements predict students’ computer-based assessment performance of physics concepts in different presentation modalities Computers & Education 2014 74 61-72
[11]
Chen W, Brinton CG, Cao D, Mason-Singh A, Lu C, and Chiang M Early detection prediction of learning outcomes in online short-courses via learning behaviors IEEE Transactions on Learning Technologies 2018 12 1 44-58
[12]
Cheng, Z., Ding, Y., He, X., Zhu, L., Song, X., & Kankanhalli, M.S. (2018). Aˆ 3ncf: An adaptive aspect attention model for rating prediction. In IJCAI (pp. 3748–3754).
[13]
Covington, P., Adams, J., & Sargin, E. (2016). Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems (pp. 191–198).
[14]
Cui, Y., Chen, Z., Wei, S., Wang, S., Liu, T., & Hu, G. (2017). Attention-over-attention neural networks for reading comprehension. In Proceedings of the 55th annual meeting of the association for computational linguistics (Vol. 1: Long Papers) (pp. 593–602).
[15]
Frome, A., Corrado, G.S., Shlens, J., Bengio, S., Dean, J., Ranzato, M., & Mikolov, T. (2013). Devise: A deep visual-semantic embedding model. Advances in Neural Information Processing Systems, 26.
[16]
Fullerton A, Reitenauer VL, and Kerrigan SM A grateful recollecting: A qualitative study of the long-term impact of service-learning on graduates Journal of Higher Education Outreach and Engagement 2015 19 2 65-92
[17]
Gao, J., Wang, X., Wang, Y., Yang, Z., Gao, J., Wang, J., Tang, W., & Xie, X. (2019). Camp: Co-attention memory networks for diagnosis prediction in healthcare. In 2019 IEEE international conference on data mining (ICDM) (pp. 1036–1041). IEEE.
[18]
Geigle, C., & Zhai, C. (2017). Modeling MOOC student behavior with two-layer hidden Markov models. In Proceedings of the 4th (2017) ACM conference on learning@ scale (pp. 205–208).
[19]
Goo, C.W., Gao, G., Hsu, Y.K., Huo, C.L., Chen, T.C., Hsu, K.W., & Chen, Y.N. (2018). Slot-gated modeling for joint slot filling and intent prediction. In Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics: Human language technologies (Vol. 2 (Short Papers), pp 753–757).
[20]
Grawemeyer B, Mavrikis M, Holmes W, Gutiérrez-Santos S, Wiedmann M, and Rummel N Affective learning: Improving engagement and enhancing learning with affect-aware feedback User Modeling and User-Adapted Interaction 2017 27 1 119-158
[21]
Greenwood DA Outcomes of an academic service-learning project on four urban community colleges Journal of Education and Training Studies 2015 3 3 61-71
[22]
Hew KF, Hu X, Qiao C, and Tang Y What predicts student satisfaction with MOOCs: A gradient boosting trees supervised machine learning and sentiment analysis approach Computers & Education 2020 145 103724
[23]
Honnett, E.P., & Poulsen, S.J. (1989). Principals of good practice for combining service and learning. Guides 27.
[24]
Huang, X., Qian, S., Fang, Q., Sang, J., & Xu, C. (2018). CSAN: Contextual self-attention network for user sequential recommendation. In Proceedings of the 26th ACM international conference on Multimedia (pp. 447–455).
[25]
Hutt S, Krasich K, Mills C, Bosch N, White S, Brockmole JR, and D’Mello SK Automated gaze-based mind wandering detection during computerized learning in classrooms User Modeling and User-Adapted Interaction 2019 29 4 821-867
[26]
Kardan AA, Sadeghi H, Ghidary SS, and Sani MRF Prediction of student course selection in online higher education institutes using neural network Computers & Education 2013 65 1-11
[27]
Kingma, D.P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv:14126980.
[28]
Kruthiventi SS, Ayush K, and Babu RV Deepfix: A fully convolutional neural network for predicting human eye fixations IEEE Transactions on Image Processing 2017 26 9 4446-4456
[29]
Kuzilek J, Hlosta M, and Zdrahal Z Open university learning analytics dataset Scientific data 2017 4 170171
[30]
Lambright KT and Lu Y What impacts the learning in service learning? an examination of project structure and student characteristics Journal of Public Affairs Education 2009 15 4 425-444
[31]
Lemons G, Carberry A, Swan C, and Jarvin L The effects of service-based learning on metacognitive strategies during an engineering design task International Journal for Service Learning in Engineering Humanitarian Engineering and Social Entrepreneurship 2011 6 2 1-18
[32]
Liu, B., & Lane, I. (2016). Attention-based recurrent neural network models for joint intent detection and slot filling. arXiv:160901454.
[33]
Liu, Z., Winata, G.I., Lin, Z., Xu, P., & Fung, P. (2020). Attention-informed mixed-language training for zero-shot cross-lingual task-oriented dialogue systems. In Proceedings of the AAAI conference on artificial intelligence, (Vol. 34 pp. 8433–8440).
[34]
Lo, K.W.K., Ngai, G., Chan, S.C.F., & Kp, Kwan (2019). A computational approach to analyzing associations between students’ learning gains and learning experience in service-learning. In International association for research on service-learning and community engagement (IARSLCE).
[35]
Mabry JB Pedagogical variations in service-learning and student outcomes: How time, contact, and reflection matter Michigan Journal of Community Service Learning 1998 5 1 32-47
[36]
Minaei-Bidgoli, B., Kashy, D.A., Kortemeyer, G., & Punch, W.F. (2003). Predicting student performance: an application of data mining methods with an educational web-based system. In 33rd Annual Frontiers in Education, 2003. FIE 2003, (Vol. 1 pp. T2A–13). IEEE.
[37]
Moely BE and Ilustre V The impact of service-learning course characteristics on university students’ learning outcomes Michigan Journal of Community Service Learning 2014 21 1 5-16
[38]
Moriasi DN, Arnold JG, Van Liew MW, Bingner RL, Harmel RD, and Veith TL Model evaluation guidelines for systematic quantification of accuracy in watershed simulations Transactions of the ASABE 2007 50 3 885-900
[39]
Nadler JT, Weston R, and Voyles EC Stuck in the middle: the use and interpretation of mid-points in items on questionnaires The Journal of General Psychology 2015 142 2 71-89
[40]
Nayak, T, & Ng, HT (2019). Effective attention modeling for neural relation extraction. In Proceedings of the 23rd conference on computational natural language learning (CoNLL) (pp. 603–612).
[41]
Ngai G, Chan SC, and Kwan KP Challenge, meaning and preparation: Critical success factors influencing student learning outcomes from service-learning Journal of Higher Education Outreach and Engagement 2018 22 4 55-80
[42]
Novak JM, Markey V, and Allen M Evaluating cognitive outcomes of service learning in higher education: A meta-analysis Communication Research Reports 2007 24 2 149-157
[43]
Paechter M, Maier B, and Macher D Students’ expectations of, and experiences in e-learning: Their relation to learning achievements and course satisfaction Computers & Education 2010 54 1 222-229
[44]
Pelánek R Bayesian knowledge tracing, logistic models, and beyond: an overview of learner modeling techniques User Modeling and User-Adapted Interaction 2017 27 3 313-350
[45]
Prentice M and Robinson G Improving student learning outcomes with service learning Community College Journal 2010 10 2 1-16
[46]
Romero C, López MI, Luna JM, and Ventura S Predicting students’ final performance from participation in on-line discussion forums Computers & Education 2013 68 458-472
[47]
Rumelhart DE, Hinton GE, and Williams RJ Learning representations by back-propagating errors Nature 1986 323 6088 533-536
[48]
Simons L and Cleary B The influence of service learning on students’ personal and social development College Teaching 2006 54 4 307-319
[49]
Singh J, Knapp HV, Arnold J, and Demissie M Hydrological modeling of the Iroquois river watershed using HSPF and SWAT 1 JAWRA Journal of the American Water Resources Association 2005 41 2 343-360
[50]
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, and Salakhutdinov R Dropout: a simple way to prevent neural networks from overfitting The Journal of Machine Learning Research 2014 15 1 1929-1958
[51]
Tomasevic N, Gvozdenovic N, and Vranes S An overview and comparison of supervised data mining techniques for student exam performance prediction Computers & Education 2020 143 103676
[52]
Velez, P., & Ashworth, S.D. (2007). The impact of item readability on the endorsement of the midpoint response in surveys. Survey Research Methods, (Vol. 1 pp. 69–74).
[53]
Wang, X, He, X, Feng, F, Nie, L, & Chua, TS (2018). TEM: Tree-enhanced embedding model for explainable recommendation. In Proceedings of the 2018 World Wide Web conference (pp. 1543–1552).
[54]
Wang Z, Xia H, Chen S, and Chun G Joint representation learning with ratings and reviews for recommendation Neurocomputing 2021 425 181-190
[55]
Weber, J.E., & Weber, P.S. (2010). Service-learning: An empirical analysis of the impact of service-learning on civic mindedness. Journal of Business, Society and Government, Spring, pp 79–94.
[56]
Weiler L, Haddock S, Zimmerman TS, Krafchick J, Henry K, and Rudisill S Benefits derived by college students from mentoring at-risk youth in a service-learning course American Journal of Community Psychology 2013 52 3-4 236-248
[57]
Wu, C, Wu, F, An, M, Huang, J, Huang, Y, & Xie, X (2019). NPA: Neural news recommendation with personalized attention. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 2576–2584).
[58]
Wurr, A.J., & Hamilton, C.H. (2012). Leadership development in service-learning: An exploratory investigation. Journal of Higher Education Outreach and Engagement, 213–240.
[59]
Yan W, Wang D, Cao M, and Liu J Deep auto encoder model with convolutional text networks for video recommendation IEEE Access 2019 7 40333-40346
[60]
Yorio PL and Ye F A meta-analysis on the effects of service-learning on the social, personal, and cognitive outcomes of learning Academy of Management Learning & Education 2012 11 1 9-27
[61]
You, Q, Jin, H, Wang, Z, Fang, C, & Luo, J (2016). Image captioning with semantic attention. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4651–4659).
Recommendations
How does desktop virtual reality enhance learning outcomes? A structural equation modeling approach
This study examined how desktop virtual reality (VR) enhances learning and not merely does desktop VR influence learning. Various relevant constructs and their measurement factors were identified to examine how desktop VR enhances learning and the fit ...
A Course Redesign to Redefine Students Learning Outcomes: What's All the Hype?
ICIEI '18: Proceedings of the 3rd International Conference on Information and Education InnovationsSince the Commission for Academic Accreditation required all Higher Education institutions in the UAE to "develop a learning outcomes-based approach" there have been many attempts to modify the way student learning is measured. At the course level, it ...
Implementing Service-Learning Through an Online Graduate Course in Instructional Design
The following paper describes a collaborative service-learning project done in an online graduate class on instructional design. With the help of the university's service-learning program, a community issue was identified related to encouraging ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© The Author(s) 2023.
Publisher
Kluwer Academic Publishers
United States
Publication History
Published: 01 April 2023
Accepted: 09 January 2023
Received: 09 June 2021
Author Tags
Qualifiers
- Research-article
Funding Sources
- Hong Kong Research Grants Council and the Hong Kong Polytechnic University
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 23 Nov 2024
Other Metrics
Citations
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in