Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Success, failure and emotions: examining the relationship between performance feedback and emotions in diagnostic reasoning

  • Development Article
  • Published:
Educational Technology Research and Development Aims and scope Submit manuscript

Abstract

Students experience a variety of emotions following achievement outcomes which stand to influence how they learn and perform in academic settings. However, little is known about the link between student outcome emotions and dimensions of performance feedback in computer-based learning environments (CBLEs). Understanding the dynamics of this relationship is particularly important for high-stakes, competency-based domains such as medical education. In this study, we examined the relationship between medical students’ (N = 30) outcome emotion profiles and their performance on a diagnostic reasoning task in the CBLE, BioWorld. We found that participants could be organized into distinct emotion groups using k-means cluster analyses based on their self-reported outcome emotion profiles: an expected positive emotion cluster and negative emotion cluster and an unexpected low intensity emotion cluster. A clear relationship was found between emotion clusters and diagnostic performance such that participants classified to the positive emotion cluster had the highest performance; those classified to the negative emotion cluster had the lowest performance; and those classified to the low intensity emotion cluster had performance outcomes that fell between the other two. Our discussion focuses on the theoretical implications of emotion classification and design recommendations for learning environments and emotional interventions in computer-based contexts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Antonenko, P. D., Toy, S., & Niederhauser, D. S. (2012). Using cluster analysis for data mining in educational technology research. Educational Technology Research and Development, 60, 383–398.

    Article  Google Scholar 

  • Arroyo, I., Muldner, K., Burleson, W., & Woolf, B. (2014). Adaptive interventions to address students’ negative activating and deactivating emotions during learning activities. Design Recommendations for Adaptive ITS (pp. 79–92). Adelphi, MD: U.S. Army Research Laboratory.

  • Artino, A. R., Jr., Hemmer, P. A., & Durning, S. J. (2011). Using self-regulated learning theory to understand the beliefs, emotions, and behaviors of struggling medical students. Academic Medicine, 86(10), S35–S38.

    Article  Google Scholar 

  • Artino, A. R., Jr., Holmboe, E. S., & Durning, S. J. (2012). Can achievement emotions be used to better understand motivation, learning, and performance in medical education? Medical Teacher, 34(3), 240–244.

    Article  Google Scholar 

  • Artino, A. R., La Rochelle, J. S., & Durning, S. J. (2010). Second-year medical students’ motivational beliefs, emotions, and achievement. Medical Education, 44, 1203–1212.

    Article  Google Scholar 

  • Azevedo, R., Witherspoon, A., Chauncey, A., Burkett, C., & Fike, A. (2009). MetaTutor: A MetaCognitive tool for enhancing self-regulated learning. In R. Pirrone, R. Azevedo, & G. Biswas (Eds.), Proceedings of the AAAI Fall Symposium on Cognitive and Metacognitive Educational Systems (pp. 14–19). Menlo Park, CA: Association for the Advancement of Artificial Intelligence (AAAI) Press.

  • Bearman, M. (2003). Is virtual the same as real? Medical students’ experiences of a virtual patient. Academic Medicine, 78(5), 538–545.

    Article  Google Scholar 

  • Beck, G. (2011). Investigation of the relationship between achievement emotions and academic performance in medical students. Unpublished doctoral dissertation. Capella University, Minneapolis, MN, USA.

  • Bland, J. M., & Altman, D. G. (1997). Statistics notes: Cronbach’s alpha. British Medical Journal, 314(7080), 572.

    Article  Google Scholar 

  • Boese, G. D., Stewart, T. L., Perry, R. P., & Hamm, J. M. (2013). Assisting failure-prone individuals to navigate achievement transitions using a cognitive motivation treatment (attributional retraining). Journal of Applied Social Psychology, 43(9), 1946–1955.

    Article  Google Scholar 

  • Bouchet, F., Harley, J. M., Trevors, G. J., & Azevedo, R. (2013). Clustering and profiling students according to their interactions with an intelligent tutoring system fostering self-regulated learning. Journal of Educational Data Mining, 5(1), 104–146.

    Google Scholar 

  • Carver, C. S., & Scheier, M. F. (2014). Dispositional optimism. Trends in Cognitive Sciences, 18, 293–299.

    Article  Google Scholar 

  • Courteille, O., Josephson, A., & Larsson, L. O. (2014). Interpersonal behaviors and socioemotional interaction of medical students in a virtual clinical encounter. BMC Medical Education, 14(1), 1.

    Article  Google Scholar 

  • Craig, S. D., D’Mello, S., Witherspoon, A., & Graesser, A. (2008). Emote aloud during learning with AutoTutor: Applying the facial action coding system to cognitive–affective states during learning. Cognition and Emotion, 22(5), 777–788.

    Article  Google Scholar 

  • D’Mello, S. K. (2013). A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. Journal of Educational Psychology, 105(4), 1082–1099.

    Article  Google Scholar 

  • D’Mello, S. K., Craig, S. D., Witherspoon, A., Mcdaniel, B., & Graesser, A. (2008). Automatic detection of learner’s affect from conversational cues. User Modeling and User-Adapted Interaction, 18(1–2), 45–80.

    Article  Google Scholar 

  • D’Mello, S. K., & Graesser, A. C. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2), 145–157.

    Article  Google Scholar 

  • D’Mello, S., Lehman, B., Pekrun, R., & Graesser, A. (2014). Confusion can be beneficial for learning. Learning and Instruction, 29, 153–170.

    Article  Google Scholar 

  • Daniels, L. M., Haynes, T. L., Stupnisky, R. H., Perry, R. P., Newall, N. E., & Pekrun, R. (2008). Individual differences in achievement goals: A longitudinal study of cognitive, emotional, and achievement outcomes. Contemporary Educational Psychology, 33(4), 584–608.

    Article  Google Scholar 

  • Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum-likelihood from incomplete data via the EM algorithm. Journal of Royal Statistical Society B, 39, 1–38.

    Google Scholar 

  • Duffy, M. C., Azevedo, R., Sun, N. Z., Griscom, S. E., Stead, V., Crelinsten, L., et al. (2015). Team regulation in a simulated medical emergency: An in-depth analysis of cognitive, metacognitive, and affective processes. Instructional Science, 43, 401–426.

    Article  Google Scholar 

  • Dunphy, B. C., Cantwell, R., Bourke, S., Fleming, M., Smith, B., Joseph, K. S., et al. (2010). Cognitive elements in clinical decision-making. Advances in Health Sciences Education, 15(2), 229–250.

    Article  Google Scholar 

  • Gauthier, G., & Lajoie, S. P. (2014). Do expert clinical teachers have a shared understanding of what constitutes a competent reasoning performance in case-based teaching? Instructional Science, 42(4), 579–594.

    Article  Google Scholar 

  • Ghosh, D., & Vogt, A. (2012). Outliers: An evaluation of methodologies. In Joint Statistical Meetings (pp. 3455–3460). San Diego, CA: American Statistical Association.

  • Gross, J. J. (2013). Emotion regulation: Taking stock and moving forward. Emotion, 13(3), 359–365.

    Article  Google Scholar 

  • Harley, J. M. (2015). Measuring emotions: A survey of cutting-edge methodologies used in computer-based learning environment research. In S. Tettegah & M. Gartmeier (Eds.), Emotions, technology, design, and learning (pp. 89–114). London: Academic Press.

    Google Scholar 

  • Harley, J. M., & Azevedo, R. (2014). Toward a feature-driven understanding of students’ emotions during interactions with agent-based learning environments: A selective review. International Journal of Games and Computer Mediated Simulation, 6(3), 17–34.

    Article  Google Scholar 

  • Harley, J. M., Bouchet, F., & Azevedo, R. (2013). Aligning and comparing data on learners’ emotions experienced with MetaTutor. In C. H. Lane, K. Yacef, J. Mostow, P. Pavik (Eds.), Lecture Notes in Computer Science: Vol. 7926. Artificial Intelligence in Education (pp. 61–70). Berlin: Springer.

  • Harley, J. M., Bouchet, F., Hussain, S., Azevedo, R., & Calvo, R. (2015). A multi-componential analysis of emotions during complex learning with an intelligent multi-agent system. Computers in Human Behavior, 48, 615–625.

    Article  Google Scholar 

  • Harley, J. M., Carter, C. K., Papaionnou, N., Bouchet, F., Azevedo, R., Landis, R. L., et al. (2016a). Examining the predictive relationship between personality and emotion traits and students’ agent-directed emotions: Towards emotionally-adaptive agent-based learning environments. User Modeling and User-Adapted Interaction, 26, 177–219.

    Article  Google Scholar 

  • Harley, J. M., Lajoie, S. P., Frasson, C., & Hall, N. C. (2016b). Developing emotion-aware, advanced learning technologies: A taxonomy of approaches and features. International Journal of Artificial Intelligence in Education. doi:10.1007/s40593-016-0126-8.

    Google Scholar 

  • Hartigan, J. A., & Wong, M. A. (1979). A K-means clustering algorithm. Applied Statistics, 28, 100–108.

    Article  Google Scholar 

  • Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65–70.

    Google Scholar 

  • Howell, D. C. (2013). Statistical methods for psychology. Belmont, CA: Wadsworth Cengage Learning.

    Google Scholar 

  • Inventado, P. S., Legaspi, R., Suarez, M., & Numao, M. (2011). Predicting student emotions resulting from appraisal of ITS feedback. Research and Practice in Technology Enhanced Learning, 6(2), 107–133.

    Google Scholar 

  • Jarrell, A., Harley, J. M., & Lajoie, S. P. (2016). The link between achievement emotions, appraisals and task performance: Pedagogical considerations for emotions in CBLEs. Journal of Computers in Education, 3(3), 289–307.

    Article  Google Scholar 

  • Kernis, M. H., & Johnson, E. K. (1990). Current and typical self-appraisals: Differential responsiveness to evaluative feedback and implications for emotions. Journal of Research in Personality, 24(2), 241–257.

    Article  Google Scholar 

  • Lajoie, S. (2009). Developing professional expertise with a cognitive apprenticeship model: Examples from avionics and medicine. In K. A. Ericsson (Ed.), Development of professional expertise: Toward measurement of expert performance and design of optimal learning environments (pp. 61–83). Cambridge, UK: Cambridge University Press.

    Chapter  Google Scholar 

  • Lajoie, S. P., & Azevedo, R. (2006). Teaching and learning in technology-rich environments. In P. Alexander & P. Winne (Eds.), Handbook of educational psychology (2nd ed., pp. 803–821). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Lajoie, S., Lee, L., Poitras, E., Bassiri, M., Kazemitabar, M., Cruz-Panesso, I., Hmelo-Silver, C., Wiseman, J., Chan, L., & Lu, J. (2015). The role of regulation in medical student learning in small groups: Regulating oneself and others’ learning and emotions. In Järvelä, S. & Hadwin, A. (Eds.) Special issue: Examining the emergence and outcomes of regulation in CSCL. Journal of Computer and Human Behavior.

  • Lazarus, R. S. (2006). Emotions and interpersonal relationships: Toward a person-centered conceptualization of emotions and coping. Journal of Personality, 74(1), 9–46.

    Article  Google Scholar 

  • Leelawong, K., & Biswas, G. (2008). Designing learning by teaching agents: The Betty’s brain system. International Journal of Artificial Intelligence in Education, 18, 181–208.

    Google Scholar 

  • Martinent, G., Nicolas, M., Gaudreau, P., & Campo, M. (2013). A cluster analysis of affective states before and during competition. Journal of Sport & Exercise Psychology, 35(6), 600–611.

    Article  Google Scholar 

  • McHugh, M. L. (2013). The Chi square test of independence. Biochemia Medica, 23(2), 143–149.

    Article  Google Scholar 

  • McQuiggan, S. W., Robison, J. L., & Lester, J. C. (2010). Affective transitions in narrative-centered learning environments. Educational Technology & Society, 13(1), 40–53.

    Google Scholar 

  • Meyers, L., Gamst, G., & Guarino, A. (2013). Applied multivariate research: Design and interpretation. Thousand Oaks, CA: SAGE.

    Google Scholar 

  • Mooi, E., & Sarstedt, M. (2011). Cluster analysis (pp. 237–284). Berlin: Springer.

    Google Scholar 

  • Naismith, L. M. (2013). Examining motivational and emotional influences on medical students’ attention to feedback in a technology-rich environment for learning clinical reasoning. Unpublished doctoral dissertation. Department of Educational and Counselling Psychology, McGill University, Montreal, Quebec, Canada.

  • Nielsen, P. A. (1991). Approaches to appreciate information systems methodologies: A soft system survey. Scandinavian Journal of Information Systems, 9, 43–60.

    Google Scholar 

  • Pekrun, R. (2006). The control-value theory of achievement emotions. Educational Psychology Review, 18(4), 315–341.

    Article  Google Scholar 

  • Pekrun, R., & Hofmann, H. (1996). Affective and motivational processes: Contrasting interindividual and intraindividual perspectives. Paper presented at the annual meeting of the American Educational Research Association, New York.

  • Pekrun, R., & Perry P. P. (2014). Control-value theory of achievement emotions. In International handbook of emotions in education (pp. 120–141). Routledge: New York.

  • Pekrun, R., Goetz, T., Frenzel, A. C., Barchfeld, P., & Perry, R. P. (2011). Measuring emotions in students’ learning and performance. Contemporary Educational Psychology, 36, 36–48.

    Article  Google Scholar 

  • Pekrun, R., Goetz, T., Perry, R. P., Kramer, K., Hochstadt, M., & Molfenter, S. (2004). Beyond test anxiety: Development and validation of the test emotions questionnaire (TEQ). Anxiety, Stress & Coping, 17, 287–316.

    Article  Google Scholar 

  • Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative research. Educational Psychologist, 37(2), 91–105.

    Article  Google Scholar 

  • Perry, R. P., Stupnisky, R. H., Hall, N. C., Chipperfield, J. G., & Weiner, B. (2010). Bad starts and better finishes: Attributional retraining and initial performance in competitive achievement settings. Journal of Social and Clinical Psychology, 29(6), 668–700.

    Article  Google Scholar 

  • Peterson, E. R., Brown, G. T., & Jun, M. C. (2015). Achievement emotions in higher education: A diary study exploring emotions across an assessment event. Contemporary Educational Psychology, 42, 82–96.

    Article  Google Scholar 

  • Pour, P. A., Hussain, M. S., AlZoubi, O., D’Mello, S., & Calvo, R. A. (2010). The impact of system feedback on learners’ affective and physiological states. In Intelligent tutoring systems (pp. 264–273). Berlin: Springer.

  • Rowe, A. D., Fitness, J., & Wood, L. N. (2014). The role and functionality of emotions in feedback at university: A qualitative study. The Australian Educational Researcher, 41(3), 283–309.

    Article  Google Scholar 

  • Rowe, J., Mott, B., McQuiggan, S., Robison, J., Lee, S., & Lester, J. (2009). Crystal island: A narrative-centered learning environment for eighth grade microbiology. In Workshop on intelligent educational games at the 14th international conference on artificial intelligence in education, Brighton (pp. 11–20).

  • Russell, J. A., Weiss, A., & Mendelsohn, G. A. (1989). Affect grid: A single-item scale of pleasure and arousal. Journal of Personality and Social Psychology, 57(3), 493–502.

    Article  Google Scholar 

  • Turner, J. C., Thorpe, P. K., & Meyer, D. K. (1998). Students’ reports of motivation and negative affect: A theoretical and empirical analysis. Journal of Educational Psychology, 90(4), 758.

    Article  Google Scholar 

Download references

Funding

This research was supported in part by a Master’s scholarship from SSHRC awarded to Amanda Jarrell, a Postdoctoral fellowship from FQRSC awarded to Dr. Jason Harley, grants from a SSHRC partnership (895-2011-1006), CRC (950-223504) and CF1 (30473) awarded to Dr. Susanne Lajoie, and doctoral fellowships from Richard H. Tomlinson (through McGill University) and SSHRC awarded to Dr. Laura Naismith.

Author information

Authors and Affiliations

  1. Department of Educational and Counselling Psychology, McGill University, 3700 McTavish Street 614, Montréal, QC, H3A 1Y2, Canada

    Amanda Jarrell & Susanne Lajoie

  2. Department of Educational Psychology, 6-102 Education North, University of Alberta, Edmonton, AB, T6G 2G5, Canada

    Jason M. Harley

  3. Toronto Western Hospital, University Health Network, East Wing 8E-427C, 399 Bathurst Street, Toronto, ON, M5T 2S8, Canada

    Laura Naismith

Authors
  1. Amanda Jarrell
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jason M. Harley
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Susanne Lajoie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Laura Naismith
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amanda Jarrell.

Ethics declarations

Conflict of interest

Dr. Jason Harley was a former students of Dr. Azevedo (ETR&D consulting editor). Dr. Azevedo was also a student of Dr. Susanne Lajoie, and is currently a co-investigator of an active grant with her. The authors declare that they have no other known conflicts of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the McGill Research Ethics Board and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jarrell, A., Harley, J.M., Lajoie, S. et al. Success, failure and emotions: examining the relationship between performance feedback and emotions in diagnostic reasoning. Education Tech Research Dev 65, 1263–1284 (2017). https://doi.org/10.1007/s11423-017-9521-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-017-9521-6

Keywords