research-article

Engagement vs performance: using electronic portfolios to predict first semester engineering student retention

Authors:

Everaldo Aguiar,

Nitesh V. Chawla,

G. Alex Ambrose,

Victoria GoodrichAuthors Info & Claims

LAK '14: Proceedings of the Fourth International Conference on Learning Analytics And Knowledge

Pages 103 - 112

https://doi.org/10.1145/2567574.2567583

Published: 24 March 2014 Publication History

Abstract

As providers of higher education begin to harness the power of big data analytics, one very fitting application for these new techniques is that of predicting student attrition. The ability to pinpoint students who might soon decide to drop out of a given academic program allows those in charge to not only understand the causes for this undesired outcome, but it also provides room for the development of early intervention systems. While making such inferences based on academic performance data alone is certainly possible, we claim that in many cases there is no substantial correlation between how well a student performs and his or her decision to withdraw. This is specially true when the overall set of students has a relatively similar academic performance. To address this issue, we derive measurements of engagement from students' electronic portfolios and show how these features can be effectively used to augment the quality of predictions.

References

[1]

ABET: Accreditation board for engineering and technology. http://www.abet.org/. Accessed: 2013-10-16.

[2]

Digication e-Portfolios. http://www.digication.com/. Accessed: 2013-09-30.

[3]

M. Al-Atabi, A. S. Mahdi, O. Younis, and E. Chung. An integrated portfolio and advising system for undergraduate engineering students. Journal of Engineering Science and Technology, 6(5):532--541, 2011.

[4]

R. Alkhasawneh. Developing a Hybrid Model to Predict Student First Year Retention and Academic Success in STEM Disciplines Using Neural Networks. PhD thesis, Virginia Commonwealth University, July 2011.

[5]

A. W. Astin and H. S. Astin. Undergraduate science education: The impact of different college environments on the educational pipeline in the sciences. final report. 1992.

[6]

H. C. Barrett. Researching electronic portfolios and learner engagement: The reflect initiative. Journal of Adolescent & Adult Literacy, 50(6):436--449, 2007.

[7]

M. Bayes and M. Price. An essay towards solving a problem in the doctrine of chances. By the late Rev. Mr. Bayes, communicated by Mr. Price, in a letter to John Canton, M. A. and F. R. S. Philosophical Transactions (1683--1775), pages 370--418, 1763.

[8]

L. Breiman. Random forests. Machine learning, 45(1):5--32, 2001.

Digital Library

[9]

J. Burtner. The use of discriminant analysis to investigate the influence of non-cognitive factors on engineering school persistence. Journal of Engineering Education, 94(3):335--338, 2005.

[10]

N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer. SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16:321--357, 2002.

[11]

H. L. Chen and T. C. Black. Using e-portfolios to support an undergraduate learning career: An experiment with academic advising. Educause Quarterly, 33(4), 2010.

[12]

D. A. Cieslak, T. R. Hoens, N. V. Chawla, and W. P. Kegelmeyer. Hellinger distance decision trees are robust and skew-insensitive. Data Mining and Knowledge Discovery, 24(1):136--158, 2012.

Digital Library

[13]

J. Davis and M. Goadrich. The relationship between precision-recall and roc curves. In Proceedings of the 23rd international conference on Machine learning, pages 233--240. ACM, 2006.

Digital Library

[14]

M. S. DeBerard, G. Spielmans, and D. Julka. Predictors of academic achievement and retention among college freshmen: A longitudinal study. College Student Journal, 38(1):66--80, 2004.

[15]

G. Dekker, M. Pechenizkiy, and J. Vleeshouwers. Predicting students drop out: A case study. In International Conference on Educational Data Mining, pages 41--50, 2009.

[16]

D. Delen. A comparative analysis of machine learning techniques for student retention management. Decision Support Systems, 49(4):498--506, 2010.

Digital Library

[17]

D. Delen. Predicting student attrition with data mining methods. Journal of College Student Retention: Research, Theory and Practice, 13(1):17--35, 2011.

[18]

B. Eynon. Making connections: the laguardia eportfolio. Electronic portfolios, 2(0):59--68, 2009.

[19]

D. S. Fike and R. Fike. Predictors of first-year student retention in the community college. Community College Review, 36(2):68--88, 2008.

[20]

L. A. Flowers. Effects of living on campus on african american students' educational gains in college. NASPA Journal, 41(2), 2004.

[21]

S. Herzog. Estimating student retention and degree-completion time: Decision trees and neural networks vis-à-vis regression. New Directions for Institutional Research, 2006(131):17--33, 2006.

[22]

L. F. Johnson, A. Levine, R. Smith, and S. Stone. The horizon report: 2010 edition. 2010.

[23]

W. E. Knight, M. D. Hakel, and M. Gromko. The relationship between electronic portfolio participation and student success. Association for Institutional Research, 2008.

[24]

G. D. Kuh, S. Hu, and N. Vesper. "They shall be known by what they do": An activities-based typology of college students. Journal of College Student Development, 41(2):228--44, 2000.

[25]

E. J. Lauría, J. D. Baron, M. Devireddy, V. Sundararaju, and S. M. Jayaprakash. Mining academic data to improve college student retention: An open source perspective. In International Conference on Learning Analytics and Knowledge, pages 139--142. ACM, 2012.

Digital Library

[26]

Q. Li, H. Swaminathan, and J. Tang. Development of a classification system for engineering student characteristics affecting college enrollment and retention. Journal of Engineering Education, 98(4):361--376, 2009.

[27]

J. J. Lin, P. Imbrie, and K. J. Reid. Student retention modelling: An evaluation of different methods and their impact on prediction results. In Research in Engineering Education Sysmposium, pages 1--6, 2009.

[28]

S.-H. Lin. Data mining for student retention management. Journal of Computing Sciences in Colleges, 27(4):92--99, 2012.

Digital Library

[29]

O. Lopez-Fernandez and J. L. Rodriguez-Illera. Investigating university students' adaptation to a digital learner course portfolio. Computers & Education, 52(3):608--616, 2009.

Digital Library

[30]

J. Luna. Predicting student retention and academic success at New Mexico Tech. Master's thesis, New Mexico Institute of Mining and Technology, 2000.

[31]

J. MacGregor and B. Leigh Smith. Where are learning communities now? National leaders take stock. About Campus, 10(2):2--8, 2005.

[32]

G. Mendez, T. D. Buskirk, S. Lohr, and S. Haag. Factors associated with persistence in science and engineering majors: An exploratory study using classification trees and random forests. Journal of Engineering Education, 97(1):57--70, 2008.

[33]

A. Nandeshwar, T. Menzies, and A. Nelson. Learning patterns of university student retention. Expert Systems with Applications, 38(12):14984--14996, 2011.

Digital Library

[34]

E. T. Pascarella. How college affects students: Ten directions for future research. Journal of College Student Development, 47(5):508--520, 2006.

[35]

E. T. Pascarella, P. T. Terenzini, and G. S. Blimling. The impact of residential life on students. Realizing the educational potential of residence halls, pages 22--52, 1994.

[36]

K. Pittman. Comparison of data mining techniques used to predict student retention. ProQuest, 2011.

[37]

M. Price. Purpose, audience, and engagement in spelman college's efolio project. Handbook of Research on EPortfolios, page 259, 2006.

[38]

J. Quinlan. Induction of decision trees. Machine Learning, 1(1):81--106, 1986.

[39]

J. R. Quinlan. C4. 5: programs for machine learning, volume 1. Morgan kaufmann, 1993.

Digital Library

[40]

G. Ring, B. Weaver, J. J. Jones Jr, et al. Electronic portfolios: engaged students create multimedia-rich artifacts. Journal of the research center for educational technology, 4(2):103--114, 2009.

[41]

A. Seidman. College student retention: Formula for student success. Greenwood Publishing Group, 2005.

[42]

D. Thammasiri, D. Delen, P. Meesad, and N. Kasap. A critical assessment of imbalanced class distribution problem: The case of predicting freshmen student attrition. Expert Systems with Applications, 2013.

Digital Library

[43]

V. Tinto. Leaving college: Rethinking the causes and cures of student attrition. University of Chicago Press, 2012.

[44]

C. P. Veenstra, E. L. Dey, and G. D. Herrin. A model for freshman engineering retention. Advances in Engineering Education, 1(3):1--31, 2009.

[45]

M. Xenos, C. Pierrakeas, and P. Pintelas. A survey on student dropout rates and dropout causes concerning the students in the course of informatics of the hellenic open university. Computers & Education, 39(4):361--377, 2002.

[46]

S. K. Yadav, B. Bharadwaj, and S. Pal. Data mining applications: A comparative study for predicting student's performance. arXiv preprint arXiv:1202.4815, 2012.

[47]

K. B. Yancey. Reflection and electronic portfolios. BL Cambridge, S. Kahn, DP Tompkins, & KB Yancey Electronic portfolios, 2:5--16, 2009.

[48]

C. H. Yu, S. A. DiGangi, A. Jannasch-Pennell, W. Lo, and C. Kaprolet. A data-mining approach to differentiate predictors of retention. Online Submission, 2007.

[49]

G. Zhang, T. J. Anderson, M. W. Ohland, and B. R. Thorndyke. Identifying factors influencing engineering student graduation: A longitudinal and cross-institutional study. Journal of Engineering Education, 93(4):313--320, 2004.

[50]

Y. Zhang, S. Oussena, T. Clark, and K. Hyensook. Using data mining to improve student retention in higher education: a case study. In International Conerence on Enterprise Information Systems, June 2010.

Cited By

Jung SLee ELee BKim SPark C(2024)AI-Based Modeling for Predicting Open University Student RetentionJournal of Lifelong Learning Society10.26857/JLLS.2024.5.20.2.2720:2(27-52)Online publication date: 31-May-2024
https://doi.org/10.26857/JLLS.2024.5.20.2.27
Uguina-Gadella LEstévez-Ayres IFisteus JAlario-Hoyos CKloos C(2024)Analysis and Prediction of Students' Performance in a Computer-Based Course Through Real-Time EventsIEEE Transactions on Learning Technologies10.1109/TLT.2023.333143317(1794-1804)Online publication date: 2024
https://doi.org/10.1109/TLT.2023.3331433
Mallik SGangopadhyay A(2023)Proactive and reactive engagement of artificial intelligence methods for education: a reviewFrontiers in Artificial Intelligence10.3389/frai.2023.11513916Online publication date: 5-May-2023
https://doi.org/10.3389/frai.2023.1151391
Show More Cited By

Index Terms

Engagement vs performance: using electronic portfolios to predict first semester engineering student retention
1. Applied computing
  1. Education
  2. Enterprise computing
2. Social and professional topics
  1. Professional topics
    1. Computing education

Recommendations

Who, when, and why: a machine learning approach to prioritizing students at risk of not graduating high school on time
LAK '15: Proceedings of the Fifth International Conference on Learning Analytics And Knowledge

Several hundred thousand students drop out of high school every year in the United States. Interventions can help those who are falling behind in their educational goals, but given limited resources, such programs must focus on the right students, at ...
The Components of a Successful S-STEM Program: What Works at Appalachian State University
SIGCSE '18: Proceedings of the 49th ACM Technical Symposium on Computer Science Education

In 1999, the National Science Foundation created the "Computer Science, Engineering, and Mathematics Scholarships" (CSEMS) program to provide funding for institutions to supply scholarships for academically talented and financially needy students to ...
Educational data mining with Python and Apache spark: a hands-on tutorial
LAK '16: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge

Enormous amount of educational data has been accumulated through Massive Open Online Courses (MOOCs), as well as commercial and non-commercial learning platforms. This is in addition to the educational data released by US government since 2012 to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

LAK '14: Proceedings of the Fourth International Conference on Learning Analytics And Knowledge

March 2014

301 pages

ISBN:9781450326643

DOI:10.1145/2567574

Conference Chairs:
Mattew Pistilli
Purdue University, IN
,
James Willis
Purdue University, IN
,
Drew Koch
John N. Gardner Institute for Excellence in Undergraduate Education, NC
,
Kimberly Arnold
University of Wisconsin-Madison, WI
,
Program Chairs:
Stephanie Teasley
University of Michigan, MI
,
Abelardo Pardo
The University of Sydney, Australia

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

JNGI: John N. Gardner Institute for Excellence in Undergraduate Education
University of Wisc-Madison: University of Wisconsin-Madison
SoLAR: The Society for Learning Analytics Research
Purdue University: Purdue University

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 March 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

LAK '14

Sponsor:

JNGI
University of Wisc-Madison
SoLAR
Purdue University

LAK '14: Learning Analytics and Knowledge Conference 2014

March 24 - 28, 2014

Indiana, Indianapolis, USA

Acceptance Rates

LAK '14 Paper Acceptance Rate 13 of 44 submissions, 30%;

Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
901
Total Downloads

Downloads (Last 12 months)52
Downloads (Last 6 weeks)2

Reflects downloads up to 30 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jung SLee ELee BKim SPark C(2024)AI-Based Modeling for Predicting Open University Student RetentionJournal of Lifelong Learning Society10.26857/JLLS.2024.5.20.2.2720:2(27-52)Online publication date: 31-May-2024
https://doi.org/10.26857/JLLS.2024.5.20.2.27
Uguina-Gadella LEstévez-Ayres IFisteus JAlario-Hoyos CKloos C(2024)Analysis and Prediction of Students' Performance in a Computer-Based Course Through Real-Time EventsIEEE Transactions on Learning Technologies10.1109/TLT.2023.333143317(1794-1804)Online publication date: 2024
https://doi.org/10.1109/TLT.2023.3331433
Mallik SGangopadhyay A(2023)Proactive and reactive engagement of artificial intelligence methods for education: a reviewFrontiers in Artificial Intelligence10.3389/frai.2023.11513916Online publication date: 5-May-2023
https://doi.org/10.3389/frai.2023.1151391
McConvey KGuha SKuzminykh A(2023)A Human-Centered Review of Algorithms in Decision-Making in Higher EducationProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580658(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580658
Matz SBukow CPeters HDeacons CDinu AStachl C(2023)Using machine learning to predict student retention from socio-demographic characteristics and app-based engagement metricsScientific Reports10.1038/s41598-023-32484-w13:1Online publication date: 7-Apr-2023
https://doi.org/10.1038/s41598-023-32484-w
Li CHerbert NYeom SMontgomery J(2022)Retention Factors in STEM Education Identified Using Learning Analytics: A Systematic ReviewEducation Sciences10.3390/educsci1211078112:11(781)Online publication date: 3-Nov-2022
https://doi.org/10.3390/educsci12110781
Naseem MChaudhary KSharma B(2022)Predicting Freshmen Attrition in Computing Science using Data MiningEducation and Information Technologies10.1007/s10639-022-11018-327:7(9587-9617)Online publication date: 4-Apr-2022
https://doi.org/10.1007/s10639-022-11018-3
Hall MWorsham RReavis G(2021)The Effects of Offering Proactive Student-Success Coaching on Community College Students’ Academic Performance and PersistenceCommunity College Review10.1177/009155212098203049:2(202-237)Online publication date: 15-Jan-2021
https://doi.org/10.1177/0091552120982030
Mai TBezbradica MCrane M(2021)Learning behaviours data in programming education: Community analysis and outcome prediction with cleaned dataFuture Generation Computer Systems10.1016/j.future.2021.08.026Online publication date: Sep-2021
https://doi.org/10.1016/j.future.2021.08.026
Kew STasir Z(2021)Learning Analytics in Online Learning Environment: A Systematic Review on the Focuses and the Types of Student-Related Analytics DataTechnology, Knowledge and Learning10.1007/s10758-021-09541-227:2(405-427)Online publication date: 30-Jun-2021
https://doi.org/10.1007/s10758-021-09541-2
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents