research-article

Video outperforms illustrated text: : Do old explanations for the modality effect apply in a learner-paced fifth-grade classroom context?

Authors:

Mikko Haavisto,

Janne LepolaAuthors Info & Claims

Volume 199, Issue C

https://doi.org/10.1016/j.compedu.2023.104775

Published: 01 July 2023 Publication History

Abstract

The modality effect occurs when people learn better from a combination of pictures and narration than from a combination of pictures and written text. Despite the strong empirical results in earlier studies, the modality effect has been less prominent in later studies of children in learner-paced settings. However, the generalizability of these results in practice may be limited because the studies included notable differences compared to a classroom context. The present study examined the modality effect in a learner-paced classroom context. In a within-subjects experiment, fifth graders learned from illustrated texts and videos and completed pre-, post-, and delayed tests on two science topics. The video group outperformed the illustrated text group in retention, delayed retention, cognitive load, and efficiency measures but there were no statistical differences in transfer. In both learning conditions, the cognitive load was moderate and did not correlate with any learning outcomes. The results suggest that while the modality effect can occur in a learner-paced classroom context, it may not be based on the avoidance of cognitive overload. Alternative explanations concerning the differences in settings and materials between classroom contexts and modality effect research are discussed.

Graphical abstract

Display Omitted

Highlights

•

Modality effect (ME) has not been evident in learner-paced (LP) settings and children.

•

Fifth graders studied videos (V) and illustrated texts (IT) in a LP classroom context.

•

V led to better learning and less cognitive load than IT.

•

Unexpectedly, cognitive load did not affect learning outcomes in either condition.

•

The influences of the research settings and materials on ME are discussed.

References

[1]

Ø. Anmarkrud, A. Andresen, I. Bråten, Cognitive load and working memory in multimedia learning: Conceptual and measurement issues, Educational Psychologist 54 (2) (2019) 61–83,.

[2]

P. Ayres, Using subjective measures to detect variations of intrinsic cognitive load within problems, Learning and Instruction 16 (5) (2006) 389–400,.

[3]

A. Baddeley, Working memory, Science 255 (5044) (1992) 556–559,.

[4]

D. Bates, M. Mächler, B. Bolker, S. Walker, Fitting linear mixed-effects models using lme4, Journal of Statistical Software 67 (1) (2015),.

[5]

M. Brysbaert, How many words do we read per minute? A review and meta-analysis of reading rate, Journal of Memory and Language 109 (2019),.

[6]

K.R. Butcher, The multimedia principle, in: R.E. Mayer (Ed.), The Cambridge handbook of multimedia learning, 2nd ed., Cambridge University Press, 2014, pp. 174–205,.

[7]

C.T. Chambers, C. Johnston, Developmental differences in children's use of rating scales, Journal of Pediatric Psychology 27 (1) (2002) 27–36,.

[8]

P. Chandler, J. Sweller, The split-attention effect as a factor in the design of instruction, British Journal of Educational Psychology 62 (2) (1992) 233–246,.

[9]

F. Chen, J. Zhou, Y. Wang, K. Yu, S.Z. Arshad, A. Khawaji, D. Conway, Robust multimodal cognitive load measurement, Springer International Publishing, 2016,.

[10]

J. Cheon, S. Crooks, S. Chung, Does segmenting principle counteract modality principle in instructional animation?, British Journal of Educational Technology 45 (1) (2014) 56–64,.

[11]

H.-H. Choi, J.J.G. van Merriënboer, F. Paas, Effects of the physical environment on cognitive load and learning: Towards a new model of cognitive load, Educational Psychology Review 26 (2) (2014) 225–244,.

[12]

Y.M. Cycowicz, Orienting and memory to unexpected and/or unfamiliar visual events in children and adults, Developmental Cognitive Neuroscience 36 (2019),.

[13]

G.W. Evans, Child development and the physical environment, Annual Review of Psychology 57 (1) (2006) 423–451,.

[14]

M.W. Eysenck, M.G. Calvo, Anxiety and performance: The processing efficiency theory, Cognition & Emotion 6 (6) (1992) 409–434,.

[15]

F. Faul, E. Erdfelder, A.-G. Lang, A. Buchner, G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences, Behavior Research Methods 39 (2) (2007) 175–191,.

[16]

Finnish National Board on Research Integrity TENK, The ethical principles of research with human participants and ethical review in the human sciences in Finland (No. 3, Finnish National Board on Research Integrity TENK Guidelines, 2019, 2019 https://tenk.fi/sites/default/files/2021-01/Ethical_review_in_human_sciences_2020.pdf.

[17]

J. Fox, Regression diagnostics, 2nd ed., SAGE Publications, Inc, 2020,.

[18]

I. Friso-van den Bos, E. van de Weijer-Bergsma, Classroom versus individual working memory assessment: Predicting academic achievement and the role of attention and response inhibition, Memory 28 (1) (2020) 70–82,.

[19]

P. Ginns, Meta-analysis of the modality effect, Learning and Instruction 15 (4) (2005) 313–331,.

[20]

T. van Gog, F. Kirschner, L. Kester, F. Paas, Timing and frequency of mental effort measurement: Evidence in favour of repeated measures, Applied Cognitive Psychology 26 (6) (2012) 833–839,.

[21]

T. van Gog, F. Paas, Instructional efficiency: Revisiting the original construct in educational research, Educational Psychologist 43 (1) (2008) 16–26,.

[22]

D. Gopher, R. Braune, On the psychophysics of workload: Why bother with subjective measures?, Human Factors: The Journal of the Human Factors and Ergonomics Society 26 (5) (1984) 519–532,.

[23]

G. Hatsidimitris, S. Kalyuga, Guided self-management of transient information in animations through pacing and sequencing strategies, Educational Technology Research & Development 61 (1) (2013) 91–105,.

[24]

S. Herrlinger, T.N. Höffler, M. Opfermann, D. Leutner, When do pictures help learning from expository text? Multimedia and modality effects in primary schools, Research in Science Education 47 (3) (2017) 685–704,.

[25]

S. Higgins, E. Hall, K. Wall, P. Woolner, C. Mccaughey, The impact of school environments: A literature review, The Centre for Learning and Teaching, School of Education, Communication and Language Science, University of Newcastle, 2005, https://www.researchgate.net/publication/232607630_The_Impact_of_School_Environments_A_Literature_Review.

[26]

Y.-M. Huang, R. Shadiev, W.-Y. Hwang, Investigating the effectiveness of speech-to-text recognition applications on learning performance and cognitive load, Computers & Education 101 (2016) 15–28,.

Digital Library

[27]

T. Hyvönen, P. Niemi-Sampan, A. Koli, Materiaalit ja tekijänoikeudet opetuksessa, Kuulas Helsinki, 2018, https://www.kopiosto.fi/kopiosto/tekijanoikeustietoa/tutkimustoiminta/.

[28]

F.A. Inan, S.M. Crooks, J. Cheon, F. Ari, R. Flores, M. Kurucay, D. Paniukov, The reverse modality effect: Examining student learning from interactive computer-based instruction, British Journal of Educational Technology 46 (1) (2015) 123–130,.

[29]

D. Jiang, S. Kalyuga, Confirmatory factor analysis of cognitive load ratings supports a two-factor model, The Quantitative Methods for Psychology 16 (3) (2020) 216–225,.

[30]

S. Kalyuga, Expertise reversal effect and its implications for learner-tailored instruction, Educational Psychology Review 19 (4) (2007) 509–539,.

[31]

S. Kalyuga, Cognitive load theory: How many types of load does it really need?, Educational Psychology Review 23 (1) (2011) 1–19,.

[32]

S. Kalyuga, The expertise reversal principle in multimedia learning, in: R.E. Mayer (Ed.), The Cambridge handbook of multimedia learning, 2nd ed., Cambridge University Press, 2014, pp. 576–597,.

[33]

A. Kingstone, D. Smilek, J.D. Eastwood, Cognitive ethology: A new approach for studying human cognition, British Journal of Psychology 99 (3) (2008) 317–340,.

[34]

R.E. Kirk, Experimental design, in: R. Millsap, A. Maydeu-Olivares (Eds.), The SAGE handbook of quantitative methods in psychology, SAGE Publications Ltd, 2009, pp. 23–45,.

[35]

J. Klein, Improving the reproducibility of findings by updating research methodology, Quality and Quantity 56 (3) (2022) 1597–1609,.

[36]

C.A.N. Knoop-van Campen, E. Segers, L. Verhoeven, The modality and redundancy effects in multimedia learning in children with dyslexia, Dyslexia 24 (2) (2018) 140–155,.

[37]

C.A.N. Knoop-van Campen, E. Segers, L. Verhoeven, Modality and redundancy effects, and their relation to executive functioning in children with dyslexia, Research in Developmental Disabilities 90 (2019) 41–50,.

[38]

T.K. Koo, M.Y. Li, A guideline of selecting and reporting intraclass correlation coefficients for reliability research, Journal of Chiropractic Medicine 15 (2) (2016) 155–163,.

[39]

W. Leahy, P. Chandler, J. Sweller, When auditory presentations should and should not be a component of multimedia instruction, Applied Cognitive Psychology 17 (4) (2003) 401–418,.

[40]

W. Leahy, J. Sweller, Cognitive load theory, modality of presentation and the transient information effect, Applied Cognitive Psychology 25 (6) (2011) 943–951,.

[41]

W. Leahy, J. Sweller, Cognitive load theory and the effects of transient information on the modality effect, Instructional Science 44 (1) (2016) 107–123,.

[42]

T. Liu, Y. Lin, C. Hsu, C. Hsu, F. Paas, Learning from animations and computer simulations: Modality and reverse modality effects, British Journal of Educational Technology 52 (1) (2021) 304–317,.

[43]

R. Low, J. Sweller, The modality principle in multimedia learning, in: R.E. Mayer (Ed.), The Cambridge handbook of multimedia learning, 2nd ed., Cambridge University Press, 2014, pp. 227–246,.

[44]

D. Lüdecke, M. Ben-Shachar, I. Patil, P. Waggoner, D. Makowski, performance: An r package for assessment, comparison and testing of statistical models, Journal of Open Source Software 6 (60) (2021) 3139,.

[45]

H.W. Marsh, Negative item bias in ratings scales for preadolescent children: A cognitive-developmental phenomenon, Developmental Psychology 22 (1) (1986) 37–49,.

[46]

R.E. Mayer, Cognitive theory of multimedia learning, in: R.E. Mayer (Ed.), The Cambridge handbook of multimedia learning, 2nd ed., Cambridge University Press, 2014, pp. 43–71,.

[47]

R.E. Mayer, Multimedia learning, 3rd ed., Cambridge University Press, 2020,.

[48]

R.E. Mayer, L. Fiorella, Principles for reducing extraneous processing in multimedia learning: Coherence, signaling, redundancy, spatial contiguity, and temporal contiguity principles, in: R.E. Mayer (Ed.), The Cambridge handbook of multimedia learning, 2nd ed., Cambridge University Press, 2014, pp. 279–315,.

[49]

M. Merkt, A. Hoppe, G. Bruns, R. Ewerth, M. Huff, Pushing the button: Why do learners pause online videos?, Computers & Education 176 (2022),.

[50]

M. Merkt, S. Weigand, A. Heier, S. Schwan, Learning with videos vs. learning with print: The role of interactive features, Learning and Instruction, 2011,. S0959475211000247.

[51]

R. Moreno, R.E. Mayer, Cognitive principles of multimedia learning: The role of modality and contiguity, Journal of Educational Psychology 91 (2) (1999) 358–368,.

[52]

D. Mutlu-Bayraktar, V. Cosgun, T. Altan, Cognitive load in multimedia learning environments: A systematic review, Computers & Education 141 (2019),.

Digital Library

[53]

S. Nakagawa, H. Schielzeth, A general and simple method for obtaining R ² from generalized linear mixed-effects models, Methods in Ecology and Evolution 4 (2) (2013) 133–142,.

[54]

Official Statistics of Finland, Students and qualifications, 2021, [e-publication]. Retrieved from https://www.stat.fi/til/opiskt/meta_en.html Accessed.

[55]

F. Paas, Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach, Journal of Educational Psychology 84 (4) (1992) 429–434,.

[56]

F. Paas, J. Sweller, Implications of cognitive load theory for multimedia learning, in: R.E. Mayer (Ed.), The Cambridge handbook of multimedia learning, 2nd ed., Cambridge University Press, 2014, pp. 27–42,.

[57]

F. Paas, J.E. Tuovinen, H. Tabbers, P.W.M. Van Gerven, Cognitive load measurement as a means to advance cognitive load theory, Educational Psychologist 38 (1) (2003) 63–71,.

[58]

F. Paas, J.J.G. Van Merriënboer, The efficiency of instructional conditions: An approach to combine mental effort and performance measures, Human Factors 35 (4) (1993) 737–743,.

[59]

F. Paas, J.J.G. van Merriënboer, J.J. Adam, Measurement of cognitive load in instructional research, Perceptual & Motor Skills 79 (1) (1994) 419–430,.

[60]

C.G. Penney, Modality effects and the structure of short-term verbal memory, Memory & Cognition 17 (4) (1989) 398–422,.

[61]

R. Ploetzner, Pictorial representations and learning, in: N.M. Seel (Ed.), Encyclopedia of the sciences of learning, Springer US, 2012, pp. 2636–2638,.

[62]

R Core Team, R: A language and environment for statistical computing, R Foundation for Statistical Computing, 2021, https://www.r-project.org/.

[63]

J. Reinwein, Does the modality effect exist? And if so, which modality effect?, Journal of Psycholinguistic Research 41 (1) (2012) 1–32,.

[64]

J. Reinwein, S. Tassé, Modality effects examined by means of an online sentence-picture comparison task, Journal of Psycholinguistic Research 51 (3) (2022) 521–542,.

[65]

K. Scheiter, A. Schüler, P. Gerjets, T. Huk, F.W. Hesse, Extending multimedia research: How do prerequisite knowledge and reading comprehension affect learning from text and pictures, Computers in Human Behavior 31 (2014) 73–84,.

Digital Library

[66]

A. Schmeck, M. Opfermann, T. van Gog, F. Paas, D. Leutner, Measuring cognitive load with subjective rating scales during problem solving: Differences between immediate and delayed ratings, Instructional Science 43 (1) (2015) 93–114,.

[67]

M.A. Schmuckler, What is ecological validity? A dimensional analysis, Infancy 2 (4) (2001) 419–436,.

[68]

A. Schüler, K. Scheiter, R. Rummer, P. Gerjets, Explaining the modality effect in multimedia learning: Is it due to a lack of temporal contiguity with written text and pictures?, Learning and Instruction 22 (2) (2012) 92–102,.

[69]

E. Segers, L. Verhoeven, N. Hulstijn-Hendrikse, Cognitive processes in children's multimedia text learning, Applied Cognitive Psychology 22 (3) (2008) 375–387,.

[70]

A.-M. Singh, N. Marcus, P. Ayres, The transient information effect: Investigating the impact of segmentation on spoken and written text, Applied Cognitive Psychology 26 (6) (2012) 848–853,.

[71]

S.M. Smith, E. Vela, Environmental context-dependent memory: A review and meta-analysis, Psychonomic Bulletin & Review 8 (2) (2001) 203–220,.

[72]

J. Snell, J. Grainger, Readers are parallel processors, Trends in Cognitive Sciences 23 (7) (2019) 537–546,.

[73]

K.D. Stiller, A. Freitag, P. Zinnbauer, C. Freitag, How pacing of multimedia instructions can influence modality effects: A case of superiority of visual texts, Australasian Journal of Educational Technology 25 (2) (2009),.

[74]

J. Sweller, J.J.G. van Merriënboer, F. Paas, Cognitive architecture and instructional design: 20 years later, Educational Psychology Review 31 (2) (2019) 261–292,.

[75]

H.K. Tabbers, B. de Koeijer, Learner control in animated multimedia instructions, Instructional Science 38 (5) (2010) 441–453,.

[76]

H.K. Tabbers, R.L. Martens, J.J.G. Van Merrienboer, The modality effect in multimedia instructions, Proceedings of the Annual Meeting of the Cognitive Science Society 23 (23) (2001) https://escholarship.org/uc/item/8fd0d9x5.

[77]

H.K. Tabbers, R.L. Martens, J.J.G. Van Merrienboer, Multimedia instructions and cognitive load theory: Effects of modality and cueing, British Journal of Educational Psychology 74 (1) (2004) 71–81,.

[78]

K.S. Taber, The use of Cronbach's alpha when developing and reporting research instruments in science education, Research in Science Education 48 (6) (2018) 1273–1296,.

[79]

M.J. Witteman, E. Segers, The modality effect tested in children in a user-paced multimedia environment, Journal of Computer Assisted Learning 26 (2) (2010) 132–142,.

[80]

A. Wong, W. Leahy, N. Marcus, J. Sweller, Cognitive load theory, the transient information effect and e-learning, Learning and Instruction 22 (6) (2012) 449–457,.

[81]

B. Xie, G. Salvendy, Review and reappraisal of modelling and predicting mental workload in single- and multi-task environments, Work & Stress 14 (1) (2000) 74–99,.

[82]

F.-Y. Yang, C.-Y. Chang, W.-R. Chien, Y.-T. Chien, Y.-H. Tseng, Tracking learners' visual attention during a multimedia presentation in a real classroom, Computers & Education 62 (2013) 208–220,.

[83]

J. Zhu, K. Dawson, A.D. Ritzhaupt, Investigating how multimedia and modality design principles influence student learning performance, satisfaction, mental effort, and visual attention, Journal of Educational Multimedia and Hypermedia 29 (3) (2020) 265–284.

Recommendations

The WebQuest model effects on mathematics curriculum learning in elementary school students

Since its inception in 1995, the WebQuest instructional model has received substantial attention from educators who have applied it to teaching activities. However, WebQuest has seldom been applied to mathematical teaching. Therefore, exploring ...
An experience of a three-year study on the development of critical thinking skills in flipped secondary classrooms with pedagogical and technological support

The aim of this study was attempting to investigate the outcome of critical thinking achievement of learners when its development is infused into subject teaching with pedagogical and technological support. A total of 124 junior secondary students ...
An evaluation study of the use of a cognitive tool in a one-to-one classroom for promoting classroom-based dialogic interaction

An evaluation study was conducted to investigate the effect of the use of a computer-supported cognitive tool (CT) in the one-to-one classroom setting on student engagement, attainment and perceptions in learning common fractions. Two Primary 4 classes ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computers & Education

Computers & Education Volume 199, Issue C

Jul 2023

191 pages

ISSN:0360-1315

Issue’s Table of Contents

The Authors.

Publisher

Elsevier Science Ltd.

United Kingdom

Publication History

Published: 01 July 2023

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Dec 2024

Other Metrics

View Author Metrics

Citations

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents