LJMU Research Online Santana-Monagas, E, Putwain, DW, Núñez, JL, Loro, JF and León, J Do teachers’ engaging messages predict motivation to learn and performance? http://researchonline.ljmu.ac.uk/id/eprint/16756/ Article Citation (please note it is advisable to refer to the publisher’s version if you intend to cite from this work) Santana-Monagas, E, Putwain, DW, Núñez, JL, Loro, JF and León, J (2022) Do teachers’ engaging messages predict motivation to learn and performance? Revista de Psicodidactica, 27 (1). pp. 86-95. ISSN 1136-1034 LJMU has developed LJMU Research Online for users to access the research output of the University more effectively. Copyright © and Moral Rights for the papers on this site are retained by the individual authors and/or other copyright owners. Users may download and/or print one copy of any article(s) in LJMU Research Online to facilitate their private study or for non-commercial research. You may not engage in further distribution of the material or use it for any profit-making activities or any commercial gain. The version presented here may differ from the published version or from the version of the record. Please see the repository URL above for details on accessing the published version and note that access may require a subscription. For more information please contact researchonline@ljmu.ac.uk http://researchonline.ljmu.ac.uk/ MOTIVATION MESSAGES 1 Do Teachers Engaging Messages Predict Motivation to Learn and Performance? MOTIVATION MESSAGES 2 Introduction “If you work hard you will learn interesting facts”. “Unless you work hard you will get into trouble”. These are examples of engaging messages that teachers use to encourage engagement among their students. If these messages are read carefully, it can be noticed that they support different kinds of motivations (i.e., motivational appeals; Santana-Monagas et al., 2022), the first is intrinsic to oneself (interest) and the second is external (punishment). It can also be observed that the messages are framed differently: gain-framed messages highlighting positive consequences and loss-framed messages highlighting negative consequences. In educational contexts, different teacher messages (e.g., reprimands, praise, fear appeals, etc.) have shown to be relevant for many student outcomes such as attention capacity, motivation, performance and engagement (Caldarella et al., 2020; Putwain et al., 2017, 2019; Putwain & Remedios, 2014). However, it could be that teachers can be relying on and integrating different kinds of messages within their speech. Thus, the present work approaches the study of teachers’ engaging messages as a construct derived from the combination of message framing theory (MFT: Rothman & Salovey, 1997), and selfdetermination theory (SDT: Ryan & Deci, 2000, 2020) and aims to examine how messages integrating motivational appeals and frames (gain vs. loss) relate to students’ motivation to learn and academic performance. Message Framing Theory Teachers’ engaging messages encompass both the frame and the motivational appeals within it. Regarding the frame, messages can prompt different responses depending on where the emphasis is located (Rothman & Salovey, 1997). This can highlight the benefits of engaging in an activity (gain-frame) or the cost of not doing so (loss-frame). In educational contexts, teachers can tell their students to study, work hard, and pay attention in class to obtain higher grades (gain-framed message) or they can tell them that if they don’t do so, they will fail their subject (loss-framed message). Both kinds of messages use the same stimuli to promote motivation, but with a different emphasis. Research following the MFT under educational contexts is scarce, but relevant. Studies following this theory have gathered evidence towards the negative effects that lossframed messages can have on students (Putwain et al., 2019). For instance, it has been found that messages that focus on fear of failure, namely loss-framed messages, trigger anxiety among students (Putwain & Symes, 2011), relate to low behavioural engagement, and worse performance (Putwain et al., 2017). Thus, given the non-adaptive outcomes such messages can elicit, teachers should be aware of such phenomena. Contrastingly, the possible outcomes related to the use of gain-framed messages remain largely unexamined. Furthermore, the few studies examining both messages together have not directly measured the use of these by teachers in natural contexts, but instead under artificial settings or under hypothetical contexts. These studies have shown mixed results. For instance, in (Symes & Putwain, 2016), message frame did not influence message appraisal, whereas, on another study by the same authors, gain-framed messages were related to a greater likelihood of disregarding the message when subjective task value and expectancy of success were high, compared to loss-framed messages (Putwain & Symes, 2016). These diverse results along with the lack of knowledge available regarding gain-framed messages underlines a gap in the literature aimed to be addressed with the present study. Self-Determination Theory Turning to motivational appeals, researchers following a SDT approach (Ryan & Deci, 2020) have identified four types of motivations that drive students to engage or not in MOTIVATION MESSAGES 3 certain activities. Motivational appeals can be defined as messages used by teachers that highlight students’ different motivations for engaging in a task. Motivations are commonly classified into autonomous forms of motivations (i.e., intrinsic and identified) and controlled forms of motivation (i.e., introjected and extrinsic; Deci & Ryan, 2008; Howard et al., 2021). Autonomous motivation concerns acting with willingness and choice. Contrastingly, controlled forms of motivations concern acting moved by external demands or forces (Deci & Ryan, 2008). For instance, when teachers appeal to a controlled motivation, students’ behaviour would be driven by rewards or punishments (e.g., doing homework to avoid detention) or by internal sources such as guilt or self-esteem (e.g., studying to make one’s parents feel proud). Moreover, when teachers appeal to autonomous forms of motivation, students engage in an activity purposely and because they think it is worth it (e.g., working hard because they think it is important to obtain a job in the future) or for the enjoyment they experience when doing so (Deci & Ryan, 2016). Nevertheless, in certain circumstances students might feel none of these motivations but instead feel completely amotivated, that is, a lack of intention to act (Behzadnia et al., 2018). Amotivation can result from students feeling a lack of competence, lack of interest or value, or a lack of contingency between a behaviour and it’s expected outcome (Deci & Ryan, 2008). It has commonly been identified as a distinctive negative predictor of engagement, learning processes, and well-being (Ryan & Deci, 2020). When students are autonomously motivated their performance is enhanced and, they feel fulfilled and content (Jang et al., 2016; León et al., 2015). For instance, in Taylor's et al. (2014) meta-analysis, results indicated that autonomous motivations (i.e., intrinsic and identified) were positively related with students’ school achievement, whereas controlled motivations (i.e., introjected and external) related negatively with amotivation having the strongest negative relation with achievement. Moreover, Froiland and Worrell (2016) showed that an intrinsic motivation to learn predicted students’ engagement. Thus, fostering autonomous forms of motivation (e.g., intrinsic or identified) among students would result of great importance given its substantial effect on student outcomes. Ways teachers can promote this type of motivation is through their need-supportive teaching and their instructional practices (León et al., 2017). Regarding need-supportive teaching, SDT researchers have examined and described a different set of teaching behaviours that foster one type of motivation or another (Collie et al., 2019; Vansteenkiste et al., 2012). Such behaviours support students’ innate basic needs for autonomy (the sense of willingness to actively participate in a certain activity), relatedness (feel truly bonded and connected with others), and competence (interacting effectively with the environment; Vansteenkiste et al., 2020) which result essential for growth and optimal functioning (Ryan & Deci, 2000). Autonomy-supportive teaching practices include offering choice, providing informative feedback, and showing care and attention to students' concerns, among others (Reeve, 2009). These practices have been related with students’ well-being (Behzadnia, 2020), engagement (Leo et al., 2020), motivation (Haerens et al., 2015), learning and behavior (Vansteenkiste et al., 2012). Among these behaviours, the study of teacher messages has been approached as a way of displaying an informative or controlling language (Legate et al., 2021; León et al., 2017; Reeve, 2009). However, this way of measuring teachers’ communications does not differentiate between different types of motivation that could be communicated in a more or less forceful way. Thus, examining teachers’ engaging messages from the present study perspective, as an approach to motivate students, might help to better understand teaching practices. From a practical point of view, this approach might be beneficial for teachers as it examines the exact messages they can rely on (i.e., “If you MOTIVATION MESSAGES 4 work hard, you will learn interesting facts”) instead of referring to a certain language which could seem vague (i.e., “my teacher uses forceful language”; Jang et al., 2016). Although research under the SDT has originated a strong body of evidence to reflect teacher’s capacity to motivate and engage students (Ryan & Deci, 2020), researchers are still highlighting the continuing decline in students’ academic interest (Lazarides et al., 2019) and intrinsic motivation (Scherrer & Preckel, 2019) throughout adolescence. This fact underpins the importance of the need to persist conducting research on new ways teachers can foster students’ motivation to learn. Teachers, as key agents for students’ learning (León et al., 2015; Ruiz-Alfonso & León, 2017), must be aware of the power they have to motivate students and raise their academic interest. A teacher capable to do so would not only be essential for students’ engagement and academic performance, but it would also have many other beneficial implications, such as need satisfaction, enhanced experiences of well-being (Behzadnia et al., 2018; Liu et al., 2017) and less maladaptive behavior (Oostdam et al., 2019). SDT and MFT Following Busemeyer’s (2017) and Gigerenzer’s (2017) recommendations, it is essential to not just rely on one macro-theory but also to rely on distinctive theories to accomplish a more accurate approximation to the study of human learning and behaviour. This approach may serve as a pathway for researchers to advance and gather new insight (Mayer & Sparrowe, 2013) on fields that, a priori, may seem unrelated. The following work relies on both the SDT and the MFT to enhance the study of teachers’ engaging messages as both theories could complement each other as well as counteract their weaknesses. In other words, following both of these theories would allow us to consider what neither theory could separately. For instance, MFT does not examine the types of motivation contained within the message focussing only on its frame, when in fact the motivation could determine students’ outcomes. Likewise, the SDT does not consider the frame of the message when teachers appeal to a certain motivation, despite its implication on student outcomes, as proven previously by researchers (Nicholson et al., 2019; Putwain et al., 2019; Putwain & Remedios, 2014). Together, this synthesis would lead to a better understanding of how each element of teacher messages (i.e., motivational appeals or message frame) contributes to its effect on students. It could help us acknolwedge whether a certain frame can diminish or reinforce the effect of a certain motivational appeal and viceversa. Figure 1 displays examples of the different messages that result when relying on both theories. Figure 1. Engaging Messages Multilevel Approach MOTIVATION MESSAGES 5 Teachers could use the same, or similar, engaging messages with the whole class (e.g., items could ask “My teacher tells the class that unless we work hard, we will miss our break”). Alternatively, they could direct, or adapt, engaging messages to specific students (e.g., items could ask “My teacher tells me that unless I work hard, I will miss my break). The present study used the latter approach to ask students about the teacher messages directed towards them specifically and not the whole class. Our rationale for adopting this approach is that teachers have reported adapting messages to specific students (Flitcroft et al., 2017). For example, a teacher might tend to rely mostly on intrinsic motivational appeals to encourage their students to work hard. However, this same teacher might notice that a certain student works harder when rewarded and hence might rely more on external motivational appeals. In this case, we can obtain two indicators with different meanings: the message the teacher uses with each student and the teacher’s tendency towards a particular message. That is, the most common messages the teacher uses with students in the same class. Thus, we can find data located at different levels, Level 1 data (L1 or student-level) refers to messages directed to specific students and Level 2 data (L2 or teacher-level) refers to the teacher’s tendency (Stapleton et al., 2016). When considering the multilevel nature of the data, researchers can approach a more thorough understanding of the effect these messages have on students. The present study The aim of the present study was to examine, relying on the SDT and the MFT, how teacher engaging messages relate with students’ motivation to learn and academic performance. Based on the aforementioned studies showing that negative outcomes related to loss-framed messages and positive outcomes related to autonomous forms of motivation (Froiland & Worrell, 2016; Nicholson et al., 2019; Putwain et al., 2019; Taylor et al., 2014), the following hypothesis were reached: students’ perceptions of teacher’s engaging messages characterized by a gain-frame and by autonomous motivational appeals will relate positively with students’ autonomous motivation to learn, whereas students’ perceptions of teacher’s amotivation messages will relate positively with amotivation among students (H1). Autonomous motivation to learn among students would positively relate with their academic performance, whereas amotivation will negatively relate with their academic performance (H2). Finally, it is expected that students’ perceptions of teacher’s engaging messages relate indirectly with students’ academic performance via motivation to learn (H3) (see Figure 2). Figure 2. Proposed ML-SEM. Method MOTIVATION MESSAGES 6 Participants The sample of the present study comprised 1209 students (600 females, 591 males, and 18 not reported; Mean age = 15.86, SD = 1.45) between grades 8-12. In total 49 teachers were evaluated (29 females, 19 males; Mean age = 46.38, SD = 8.07) by their corresponding students that were drawn from 63 classes from ten different secondary schools on the island of Gran Canaria (Spain) from both rural and urban environments. Students came mostly from middle-class families. The sampled schools presented no potential ethnic differences as most of the students were from the Canary Islands. Measures Teachers’ Engaging Messages In the absence of an existing instrument, new items were developed to measure teachers’ engaging messages. This new instrument is based on the Teachers Use of Fear Appeals Questionnaire (TUFAQ: Putwain et al., 2019) and incorporates new items framed by SDT and MFT to examine a wider variety of teacher messages. The instrument is composed of a total of 36 items preceded by the stem “My teacher tells me that...”. Items were grouped into nine factors. Eight of the factors corresponded to the four types of self-determined motivation (intrinsic, identified, introjected, and external) and its frame (gain vs. loss). The ninth factor was amotivation which was not classified by frame as it completely lacked one. Example items are displayed in Figure 1. Factors showed a high internal consistency with only gain-framed external showing a moderate reliability (see table 1). Different multilevel confirmatory factor analyses (CFAs) were run to compare the hypothesized model against plausible alternates. The hypothesized model displayed better fit indices than the plausible alternates considering the frame and motivational appeals independently (see supplementary material). Items were rated according to a seven-point Likert scale (1 = does not correspond at all to me to 7 = fully corresponds to me). Model fit indices for the CFA were as follows: χ² (1143) = 1873.427, p < .001, RMSEA = .028, CFI = .971, TLI = 968, SRMR-W = .049, SRMR-B = .138. Motivation to Learn Motivation to learn was measured using five of the seven subscales of the Spanish version of the Échelle de Motivation en Éducation (Núñez et al., 2005). Each subscale was composed of 4 items preceded by the stem “Why do you study?”. The subscales used were: amotivation, external motivation, introjected motivation, identified motivation and the subscale of intrinsic motivation (see supplementary material for example items). Similar to prior studies (León et al., 2015), factors displayed a high internal consistency (see table 1). Items were rated according to a seven-point Likert scale (1 = does not correspond at all to me to 7 fully corresponds to me). Model fit indices for the CFA were as follows: χ² (120) = 12195.584, p < .001, RMSEA = .056, CFI = .900, TLI = .881, SRMR-W = .056, SRMR-B = .409. Academic Performance Students’ academic performance was measured using teacher-estimated grades in maths, obtained from official school records. Grades ranged between 0-10, being 10 the highest possible mark. In the Spanish education system grades are assigned by teachers according to different rubrics provided by the government. These grades are of great importance as they define the universities and degrees students can have access to. Procedure MOTIVATION MESSAGES 7 We first contacted the different schools and requested their collaboration. Questionnaires were administered individually by researchers during a teaching period where participants’ assessed teacher was not present. Items were made specific to one compulsory subject, namely mathematics. For engaging messages, students were asked to think about their current mathematics teacher. The objectives of the research were explained to participants, emphasizing the voluntary and confidential nature of their participation. All participants provided informed consent to participate. The study was conducted in accordance with the ethical guidelines of the Declaration of Helsinki and was approved by the University Human Research Ethics Committee. Data Analytic Plan As mentioned, when following a multilevel approach, students’ ratings can be aggregated to serve as a measure of teachers’ tendency. Similar answers among students would indicate that what is been measure is, in fact, teacher’s messages and not students’ impressions (Marsh et al., 2012). Researchers can rely on ICC1 statistic, which represents the proportion of variance in the data attributable to the class level, to inform about the similarity observed across students’ ratings in a same class (Lüdtke et al., 2009; Marsh et al., 2012). For variables in which students rate a characteristic of the teacher, these values are found typically between .10 and .30, whereas for variables that are specific to each student these values are larger (Marsh et al., 2008). Then, to examine if teacher’s engaging messages predict students’ motivation to learn and performance, nine multilevel structural equation models (ML-SEMs; one for each kind of engaging message) were estimated. This approach allows to identify the total effect that a single message has on a student, instead of freely estimating all possible correlations among all constructs (Arens & Morin, 2016). The fit indices used to compare the models and the CFA of the instruments were the following: The root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR), comparative fit index (CFI) and the Tucker–Lewis index (TLI). To the best of our knowledge, there are no current guidelines to interpret multilevel models, therefore, Hu and Bentler's (1999) guidelines for single level models were followed. Models show a good fit when they meet the following criteria: RMSEA < .05, SRMR < .08, and CFI and TLI > .95. However, when working with naturalistic data these indices should be interpreted with some flexibility (Heene et al., 2011). To analyse internal consistency, McDonald’s ω, Cronbach’s α, the averaged variance extracted, and the composite reliability of all factors were estimated for each of the nine factors proposed (See table 1). Values ≥ .7 are indicators of good reliability (Gu et al., 2017). Messages were modelled with the matching motivation to learn (see figure 3 for an example). Separate models for engaging messages were run to keep models as parsimonious as possible (Hox & McNeish, 2020). Including all messages in a single model would add unnecessary complexity resulting in possible non-convergence and requiring a larger sample size and number of clusters (Lüdtke et al., 2008, 2009; Marsh et al., 2009). Moreover, factor loadings were also made constant across levels (Morin et al., 2014). L2 variables were built from the class aggregation of student responses and L1 variables were class-mean centred (Marsh et al., 2012; Morin et al., 2014). To test whether teacher’s engaging messages had a direct or indirect relation with student performance, fully and partially indirect ML-SEMs were tested and compared. For the fully indirect model, relations between variables followed the paths shown in Figure 2, whereas the partially indirect model included an additional direct path between teacher’s engaging messages and students’ academic performance. To estimate the standard errors of the indirect paths, the delta method was followed (MacKinnon et al., 2002). This method divides the difference between the simple and the partial correlation by the estimated standard errors and contrasts the result with the standard normal distribution to examine MOTIVATION MESSAGES 8 whether there is any interceding variable effect. 95% confidence intervals (CIs) were estimated around the point estimate of the standardised indirect path coefficient and CIs that do not cross zero at statistically significant at p < .05. Gain-framed intrinsic messages Intrinsic motivation to learn Academic performance Intrinsic motivation to learn Academic performance Teacher level (L2) Student level (L1) Gain-framed intrinsic messages Figure 3. Example of one of the nine ML-SEM. The weighted least square mean adjusted estimator (WLSM) was used as the estimation method due to the categorical nature of the variables and its higher accuracy over the maximum likelihood method especially in cases when categorical variables are not normally distributed (Schmitt, 2011; see Table 1). All data analysis was performed with Mplus 8.4 (Muthen & Muthén, 2021). Missing data were handled with the full information maximum likelihood approach. Results Descriptive Statistics Descriptive analyses, intra-class correlations, McDonald’s ω, Cronbach’s α, the averaged variance extracted, and the composite reliability are displayed in Table 1. ICC1 values show that a moderate proportion of the variability observed was attributed to the differences between classrooms (ICC1s .021 to .189). Table 1 Descriptive Statistics, Intraclass Correlations and Internal Consistency Indices for Teacher’s Engaging Messages, Motivation to Learn and Academic Performance. Composite Average variance M SD Skewness Kurtosis ICC1 ω α reliability extracted TEM: G-Intrinsic TEM: L-Intrinsic TEM: G-Identified TEM: L-Identified TEM: G-Introjected TEM: L-Introjected TEM: G-Extrinsic TEM: L-Extrinsic TEM: Amotivation MTL: Intrinsic MTL: Identified MTL: Introjected MTL: Extrinsic MTL: Amotivation Academic performance 4.03 3.54 4.96 2.75 4.14 2.33 4.32 2.43 1.34 4.80 6.02 4.76 5.61 1.85 5.24 2.21 1.52 1.52 1.58 1.57 1.67 1.70 1.57 1.50 .96 1.56 1.13 1.63 1.27 1.45 -.19 .16 -.79 .76 -.27 1.23 -.34 1.02 3.70 -.52 -1.55 -.50 -.90 1.88 -.01 -.67 -.78 -.08 -.47 -.93 .60 -.60 .18 14.79 -.46 2.47 -.62 .46 3.21 -.70 .18 .07 .10 .10 .12 .06 .14 .10 .07 .06 .02 .06 .07 .06 .19 .81 .81 .85 .89 .88 .92 .68 .83 .97 .90 .87 .85 .78 .91 - .81 .77 .84 .85 .86 .88 .69 .78 .92 .87 .78 .81 .67 .82 - .84 .82 .87 .90 .90 .92 .72 .85 .97 .90 .87 .86 .81 .91 - .56 .53 .62 .69 .68 .75 .40 .59 .90 .69 .62 .60 .55 .71 - Note. TEM = teacher’s engaging messages; MTL= Motivation to learn; ω = McDonald’s Omega; α = Cronbach’s alpha; G = Gain-framed; L = Loss-framed. MOTIVATION MESSAGES 9 Bivariate Correlations Bivariate correlations are displayed in Table 2. Gain and loss-framed messages were positively inter-correlated. Gain-framed messages showed negative correlations with amotivation messages and loss-framed messages positive correlations. Broadly, at L1, gainframed messages and loss-framed messages correlated positively with motivation. Gainframed intrinsic messages were positively correlated with grades, as well as intrinsic and identified motivation. Finally, at L1, amotivation messages and amotivation were negatively correlated with grades. Table 2 Bivariate Correlations Among Variables 1 2 3 1. TEM: G-Intrinsic 2. TEM: G-Identified 3. TEM: G-Introjected 4. TEM: G-Extrinsic 5. TEM: L-Intrinsic 6. TEM: L-Identified 7. TEM: L-Introjected 8. TEM: L-Extrinsic 9. TEM: Amotivation 10. MTL: Intrinsic 11. MTL: Identified 12. MTL: Introjected 13. MTL: Extrinsic 14. MTL: Amotivation 15. Academic performance -.58 .67 .59 .39 .20 .27 .15 -.04 .40 .27 .29 .14 -.09 .11 .86 -.62 .53 .35 .29 .24 .17 -.09 .28 .32 .26 .18 -.09 .05 .90 .81 -.68 .33 .27 .34 .24 -.02 .29 .24 .36 .17 -.02 -.01 4 5 6 7 8 9 10 11 12 13 14 15 .84 .73 .93 -.33 .26 .30 .25 -.04 .23 .22 .28 .19 -.03 -.01 .72 .82 .68 .61 -.54 .59 .49 .03 .23 .17 .19 .14 .03 .01 .25 .35 .49 .60 .20 -.78 .68 .15 .10 .12 .21 .18 .14 -.03 .39 .44 .63 .69 .25 .94 -.75 .16 .16 .12 .24 .13 .11 -.02 .25 .25 .47 .59 .17 .81 .88 -.12 .05 .05 .16 .14 .14 -.06 -.11 -.09 .16 .12 -.10 .66 .67 .62 --.06 -.15 .01 -.05 .29 -.08 .54 .41 .43 .43 .33 -.10 .11 -.06 -.16 -.52 .46 .17 -.20 .18 .20 .28 .19 .32 .25 .09 .06 -.20 -.30 .57 -.48 .54 -.38 .18 .40 .22 .65 .73 .12 .66 .76 .61 .53 .42 .27 -.40 -.05 .03 .23 .14 .43 .54 -.07 .82 .72 .52 .55 -.01 .14 .77 --.14 .02 .13 .06 .29 .28 -.03 .62 .59 .64 .76 -.28 -.57 .45 .64 --.19 .10 -.03 -.20 .03 .11 -.38 -.42 -.22 -.53 .37 .35 -.24 -.33 -.39 -- Note. N=1209 (below diagonal), N=63 (above diagonal); TEM =Teacher’s engaging messages; MTL=Motivation to learn; G=Gain-framed; L=Loss-framed. ML-SEM Fully indirect ML-SEMs showed model fit indices that were either comparable to, or superior to the partially indirect models (see Table 3). Given the greater parsimony of the fully indirect ML-SEMs and that, for the partially indirect ML-SEMs direct relations from teacher engaging messages and performance only reached statistical significance (p < .05) once (at L2 in the loss-framed identified model; p = .033), fully indirect models were retained (fit indices for the partially models can be found in the supplementary material). Table 3 Model Fit Indices for the ML-SEM Models Model χ² G-Intrinsic L-Intrinsic G-Identified L-Identified G-Introjected L-Introjected G-Extrinsic L-Extrinsic Amotivation 163.626 (1208, 62) 169.319 (1202, 62) 101.668 (1208,62) 83.510 (1202, 62) 406.851 (1208, 62) 697.683 (1208, 62) 193.288 (1208,62) 238.915 (1202, 62) 108.988 (1208, 62) RMSEA CFI TLI SRMR-w SRMR-b .037 .038 .023 .017 .068 .092 .042 .049 .025 .994 .993 .993 .998 .980 .950 .980 .979 .998 .993 .992 .992 .998 .977 .942 .977 976 .998 .034 .036 .039 .035 .049 .085 .048 .060 .040 .072 .114 .311 .471 .143 .205 .244 .218 .105 MOTIVATION MESSAGES 10 Note. G=Gain-framed; L=Loss-framed; χ² of all models was p <.05. Direct Relations Table 4 shows the direct relations in the ML-SEMs (Unstandardized parameters can be found in the supplementary material). Concerning path 1, mostly all engaging messages related significantly with their matching motivation to learn at both levels of analysis. Exceptions include gain and loss-framed identified; and loss-framed intrinsic messages at L2. When comparing the effects among the different teacher messages, it can be appreciated that among the messages that appealed to autonomous motivations (i.e., intrinsic and identified) stronger relations with motivation to learn where found among gain-framed messages. Regarding relations on path 2, overall, autonomous motivations to learn positively predicted academic performance at both levels of analysis, whereas controlled motivations to learn (i.e., introjected and extrinsic) negatively predicted academic performance at L2. At L1 extrinsic motivation to learn had a very small positive effect on performance. Finally, amotivation messages positively predicted amotivation to learn, and this in turn, negatively predicted academic performance at both levels of analysis. Table 4 Standardized Direct Effects from the ML-SEMs Model Level Path 1 TEM MTL ß SE G-Intrinsic L-Intrinsic G-Identified L-Identified G-Introjected L-Introjected G-Extrinsic L-Extrinsic Amotivation L2 L1 L2 L1 L2 L1 L2 L1 L2 L1 L2 L1 L2 L1 L2 L1 L2 L1 .54 .50 .20 .29 .98 .45 .96 .09 .66 .48 .98 .38 .55 .27 .64 .09 .86 .48 .10 .03 .17 .03 3.36 .02 3.13 .03 .13 .02 .12 .03 .17 .03 .22 .03 .09 .03 95% CI .37, .71 .45, .54 -.07, .48 .25, .34 -4.54, 6.50 .41, .49 -4.18, 6.11 .04, .15 .45, .87 .44, .51 .78, 1.17 .33, .42 .26, .83 .22, .32 .28, 1.00 .04, .15 .71, 1.01 .43, .53 Path 2 MTL Academic performance ß SE 95% CI .32 .21 .40 .18 -.17 .17 -.57 .18 -.32 .04 -.41 .04 -.30 .07 -.57 .07 -.70 -.23 .16 .03 .15 .03 .57 .04 1.89 .05 .22 .05 .21 .04 .20 .04 .23 .04 .13 .04 .05, .58 .15, .26 .15, .66 .12, .24 -1.10, .76 .11, .24 -3.68, 2.53 .10, .25 -.70, .04 -.03, .11 -.80, -.06 -.03, .11 -.64, .03 .02, .13 -.95, -.20 .02, .13 -.92, -.48 -.29, -.17 Note. TEM=Teachers engaging messages; MTL=Motivation to learn; G=Gain-framed; L=Loss-framed; L2=Teacher level; L1=Student level. Indirect Relations Table 5 shows the indirect relations in the ML-SEMs. Overall, the autonomous motivations predicted academic performance at both levels of analysis except for loss-framed identified messages, which negatively predicted performance at L2. Indirect relations between introjected messages and performance were never statistically significant at both levels of analysis (p>.05). At L2, extrinsic messages (gain and loss-framed) negatively predicted performance, whereas at L1 its relation with performance was positive, although MOTIVATION MESSAGES 11 this effect was small. Lastly, negative indirect relations at L1 and L2 were shown for amotivation messages and performance. Table 5 Indirect Effects from the ML-SEMs Model Level TEM  academic performance (via MTL) ß SE 95% CI G-Intrinsic L-Intrinsic G-Identified L-Identified G-Introjected L-Introjected G-Extrinsic L-Extrinsic Amotivation L2 L1 L2 L1 L2 L1 L2 L1 L2 L1 L2 L1 L2 L1 L2 L1 L2 L1 .14 .09 .13 .05 -.19 .06 -.64 .01 -.23 .02 -.55 .01 -.27 .03 -.43 .01 -.25 -.04 .09 .02 .11 .01 .24 .02 .25 .01 .17 .02 .34 .01 .20 .02 .22 .00 .07 .01 -.01, .28 .06, .11 -.05, .31 .04, .07 -.59, .20 .04, .09 -.1.05, -.23 .00, .02 -.51, .05 -.01, .04 -1.11, .00 -.01, .03 -.60, .06 .01, .05 -.72, -.06 .00, .01 -.37, -.13 -.05, -.03 Note. TEM=Teachers engaging messages; MTL=Motivation to learn; G=Gainframed; L=Loss-framed; L2=Teacher level; L1=Student level. Discussion Following a multilevel approach, the present study relies on the SDT and MFT to examine how engaging messages from teachers predict students’ motivation to learn and academic performance. Overall, teacher’s messages predict students’ motivation to learn, and this, in turn, predicts students’ performance. Major findings are discussed below. Regarding H1, as expected, gain-framed messages and autonomous motivational appeals are associated with students’ autonomous motivation to learn, whereas amotivation messages predict students’ amotivation to study. These findings are consistent with previous studies which have shown how teacher’s motivational approach is related to students’ motivation and engagement (Collie et al., 2019; Vansteenkiste et al., 2012). Moreover, they also add to this well-established relationship (Deci & Ryan, 2016; Jang et al., 2016; León et al., 2018) by not addressing teacher’s motivational approach as a mixture of many different teaching practices (Collie et al., 2019; Reeve & Cheon, 2016) but instead focuses on a specific one (i.e. teachers engaging messages) to precisely measure its unique effect on students. In such way, the present results strengthen the idea of the power teachers have to motivate students, and engage them in school tasks, but also the ability they have to demotivate them. In this sense, students whose teacher relies on gain-framed messages and autonomous motivational appeals might feel more supported, believing their teacher really wants the best for them. This might make students feel autonomous motivated, which would move them to engage in school-related tasks. An additional finding shows that, at a student level, when comparing both frames, gain-framed messages show stronger relations with student motivation (βs = .269 to .496) MOTIVATION MESSAGES 12 compared to those of loss-framed messages (βs = .091 to .377; see Table 5). This implies that highlighting the benefits of a certain activity stimulates students more than emphasizing and appealing to loss. As teachers’ engaging messages encompass both the frame and the motivation appeals, this finding suggests that self-determined motivational appeals are more effective when they are accompanied by a gain-frame. These results are the first to highlight the differences between the effect the message frame can have on students and complements the findings of previous works which have shown how loss-framed messages are associated with controlled motivations and lower engagement (Putwain et al., 2019; Putwain & Remedios, 2014). In this sense, results suggest that students might feel more motivated to focus on the positive outcomes they can obtain if they work hard than to focus on the threat or the possibility of losing something they might not even value or that they already have. Regarding H2, findings show that autonomous forms of motivation (i.e., intrinsic and identified) are positively associated with students’ academic performance, and that as expected, amotivation inversely predicts students’ academic performance. These results align with the assumptions of the SDT (Deci & Ryan, 2016; Ryan & Deci, 2000) and with previous studies that have identified the relation between autonomous motivation and positive academic outcomes (León et al., 2015; Ruiz-Alfonso & León, 2017). Students who are autonomous motivated will engage in school-related tasks because they enjoy and value them. Their engagement would in turn, influence positively their grades. Instead, amotivated students would have no reason to engage in a certain activity at all, resulting in poor performance (Cheon & Reeve, 2015). Finally, our results further confirm that teachers’ engaging messages are indirectly related to students’ academic performance (H3). This finding is key to understanding how teacher messages relate with students’ motivation and academic performance as fundamentally different interpretations can derive from paths being direct or indirect. If teacher’s engaging messages had a direct effect on performance, then these would be directly responsible for students’ performance. In contrast, results indicate that the messages relate indirectly with student performance via motivation to learn. This knowledge has practical implications for teachers as it articulates a new resource they can rely on to motivate their students and that result in a better academic performance. If teachers could simply rely more on gain-framed messages and those appealing to autonomous forms of motivation, it is likely for them to observe improvements among their students’ motivation and performance. Given the novelty of this result, this finding cannot be compared with others. Limitations and Future Directions Teachers’ engaging messages are addressed by self-reports. To overcome possible sources of unreliability future research should complement the data obtained with the scale with teacher self-reports and observational techniques. Second, our study is cross-sectional. Therefore, no casual relations can be drawn from the present study. Future research should endeavour to conduct longitudinal studies to establish directionality between the present study variables. Third, although teacher grades are better predictors than test scores (Galla et al., 2019) and despite their great relevance to predict several outcomes, such as standardized test scores (Duckworth et al., 2012); and lifetime educational attainment (French et al., 2015); these could seem subjective (Cross & Frary, 1999). Thus, future research could rely on test scores to obtain a more objective measure. Moreover, the present study conducted nine MLSEM models given their greater parsimony with the available sample. Future research should explore the relations on the present study conducting one ML-SEM. To do so, larger samples are required. Additionally, as previous research has highlighted the effect that the tone of voice might have on students’ motivation (Weinstein et al., 2018, 2019), future research MOTIVATION MESSAGES 13 could examine how the tone of voice influences the effect teacher engaging messages might have. Furthermore, future studies replicating the present one are needed to examine the reliability and factor loading of certain items and dimensions. To conclude, it could be interesting for future research to examine the predictive value that grades can have on students’ motivational experiences, as these could result from the actual fact of grading students (Krijgsman et al., 2017). Similar to previous studies (Liu et al., 2017), it would be of interest to further examine both positive (i.e., well-being) and negative (i.e., ill-being) student outcomes in regard with teachers’ engaging messages to further expand on how this teaching practice relate with student’s functioning. Practical Implications Considering the impact that teacher engaging messages can have on student’s outcomes, the above results may be of relevance for school staff, such as teachers and school psychologists, to tackle one of the main challenges they face: students lack of interest and engagement (Lazarides et al., 2019). As previous researchers have highlighted (Putwain & Remedios, 2014) most teachers are unconcerned about the type of messages they use during their lessons and, may be unaware of the effects they might trigger among students (Flitcroft et al., 2017). A way to tackle this problem could be setting up school-based interventions to instruct teachers about the different engaging messages and their effect. To start, the scale developed for the present study could be used to help teachers recognize their engaging messages and, if it proceeds, show them how they could improve it. Given the negative effects some kinds of messages might prompt (Putwain & Symes, 2011), it might be advantageous to advise teachers of what exact messages they could rely on. For example, based on the current study findings, a way math teachers can enhance autonomous forms of motivation and reduce controlled forms of motivations and amotivation among students, is relying on gain-framed messages such as “It's all about playing with algebra, if you play applying the logical rules, everything flows and works out fine”. This kind of intervention could be very easily implemented in schools as it is simple, inexpensive, and does not require much time. Conclusions The present study conceptualizes a new resource that teachers can rely on to face amotivation among students. A major conclusion can derive from the present results: teacher’s engaging messages predict students’ motivation to learn and this, in turn, predicts their academic performance. Specifically, gain-framed and autonomous motivational appeals messages predicted students’ autonomous motivation, and this, in turn, positively predicted performance. Contrastingly, amotivation messages predicted students’ amotivation to study, and these where negatively related to performance. Therefore, both the frame and the motivational appeals should be taken into account when trying to encourage students to participate in school-related activities. Given the ability teachers have to motivate students and the great influence they exert on them (Caldarella et al., 2020; Jang et al., 2016) these findings could help teachers find new ways to keep doing so. MOTIVATION MESSAGES 14 References Arens, K. A., & Morin, A. J. S. (2016). Relations between teacher’ emotional exhaustion and student’s educational outcomes. Journal of Educational Psychology, 108(6), 800–813. https://doi.org/10.1037/edu0000105.supp Behzadnia, B. (2020). The relations between students’ causality orientations and teachers’ interpersonal behaviors with students’ basic need satisfaction and frustration, intention to physical activity, and well-being. Physical Education and Sport Pedagogy, 0(0), 1– 20. https://doi.org/10.1080/17408989.2020.1849085 Behzadnia, B., Adachi, P. J. C., Deci, E. L., & Mohammadzadeh, H. (2018). Associations between students’ perceptions of physical education teachers’ interpersonal styles and students’ wellness, knowledge, performance, and intentions to persist at physical activity: A self-determination theory approach. Psychology of Sport and Exercise, 39, 10–19. https://doi.org/10.1016/j.psychsport.2018.07.003 Busemeyer, J. R. (2017). Introduction to special issue on theory integration. Decision, 4(3), 131–132. https://doi.org/10.1037/dec0000084 Caldarella, P., Larsen, R. A. A., Williams, L., Downs, K. R., Wills, H. P., & Wehby, J. H. (2020). Effects of teachers’ praise-to-reprimand ratios on elementary students’ on-task behaviour. Educational Psychology, 40(10), 1306–1322. https://doi.org/10.1080/01443410.2020.1711872 Cheon, S. H., & Reeve, J. (2015). A classroom-based intervention to help teachers decrease students’ amotivation. Contemporary Educational Psychology, 40, 99–111. https://doi.org/10.1016/j.cedpsych.2014.06.004 Collie, R. J., Granziera, H., & Martin, A. J. (2019). Teachers’ motivational approach: Links with students’ basic psychological need frustration, maladaptive engagement, and academic outcomes. Teaching and Teacher Education, 86, 102872. https://doi.org/10.1016/j.tate.2019.07.002 Cross, L. H., & Frary, R. B. (1999). Hodgepodge grading: Endorsed by students and teachers alike. Applied Measurement in Education, 12(1), 53–72. https://doi.org/10.1207/s15324818ame1201_4 Deci, E. L., & Ryan, R. M. (2008). Facilitating optimal motivation and psychological wellbeing across life’s domains. Canadian Psychology, 49(1), 14–23. https://doi.org/10.1037/0708-5591.49.1.14 Deci, E. L., & Ryan, R. M. (2016). Optimizing students’ motivation in the era of testing and pressure: A self-determination theory perspective. In W. C. Liu, J. C. K. Wang, & R. M. Ryan (Eds.), Building autonomous learners (pp. 9–29). Springer. Duckworth, A. L., Weir, D., Tsukayama, E., & Kwok, D. (2012). Who does well in life? Conscientious adults excel in both objective and subjective success. Frontiers in Psychology, 3, 1–8. https://doi.org/10.3389/fpsyg.2012.00356 Flitcroft, D., Woods, K., & Putwain, D. W. (2017). Developing school practice in preparing students for high-stake examinations in English and Mathematics. Educational and Child Psychology, 34, 7–19. French, M. T., Homer, J. F., Popovici, I., & Robins, P. K. (2015). What you do in high school matters: High School GPA, educational attainment, and labor market earnings as a young adult. Eastern Economic Journal, 41(3), 370–386. MOTIVATION MESSAGES 15 https://doi.org/10.1057/eej.2014.22 Froiland, J. M., & Worrell, F. C. (2016). Intrinsic motivation, learning goals, engagement, and achievement in a diverse high school. Psychology in the Schools, 53(3), 321–336. https://doi.org/10.1002/pits.21901 Galla, B. M., Shulman, E. P., Plummer, B. D., Gardner, M., Hutt, S. J., Goyer, J. P., D’Mello, S. K., Finn, A. S., & Duckworth, A. L. (2019). High school grades are better predictors of on-time college graduation than are admissions test scores: The roles of selfregulation and cognitive ability. American Educational Research Journal, 56(6), 2077– 2115. https://doi.org/10.3102/0002831219843292 Gigerenzer, G. (2017). A theory integration program. Decision, 4(3), 133–145. https://doi.org/10.1037/dec0000082 Gu, H., Wen, Z., & Fan, X. (2017). Structural validity of the Machiavellian Personality Scale: A bifactor exploratory structural equation modeling approach. Personality and Individual Differences, 105, 116–123. https://doi.org/10.1016/j.paid.2016.09.042 Haerens, L., Aelterman, N., Vansteenkiste, M., Soenens, B., & Van Petegem, S. (2015). Do perceived autonomy-supportive and controlling teaching relate to physical education students’ motivational experiences through unique pathways? Distinguishing between the bright and dark side of motivation. Psychology of Sport and Exercise, 16, 26–36. https://doi.org/10.1016/j.psychsport.2014.08.013 Heene, M., Hilbert, S., Draxler, C., Ziegler, M., & Bühner, M. (2011). Masking misfit in confirmatory factor analysis by increasing unique variances: A cautionary note on the usefulness of cutoff values of fit indices. Psychological Methods, 16(3), 319–336. https://doi.org/10.1037/a0024917 Howard, J. L., Bureau, J., Guay, F., Chong, J. X. Y., & Ryan, R. M. (2021). Student motivation and associated outcomes: A meta-analysis from self-determination theory. Perspectives on Psychological Science, 1–24. https://doi.org/10.1177/1745691620966789 Hox, J., & McNeish, D. (2020). Small samples in multilevel modeling. In R. Van de Schoot & M. Miocevi (Eds.), Small sample size solutions (pp. 215–225). Routledge. https://doi.org/10.4324/9780429273872-18 Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118 Jang, H., Kim, E. J., & Reeve, J. (2016). Why students become more engaged or more disengaged during the semester: A self-determination theory dual-process model. Learning and Instruction, 43, 27–38. https://doi.org/10.1016/j.learninstruc.2016.01.002 Krijgsman, C., Vansteenkiste, M., van Tartwijk, J., Maes, J., Borghouts, L., Cardon, G., Mainhard, T., & Haerens, L. (2017). Performance grading and motivational functioning and fear in physical education: A self-determination theory perspective. Learning and Individual Differences, 55, 202–211. https://doi.org/10.1016/j.lindif.2017.03.017 Lazarides, R., Gaspard, H., & Dicke, A. L. (2019). Dynamics of classroom motivation: Teacher enthusiasm and the development of math interest and teacher support. Learning and Instruction, 60, 126–137. https://doi.org/10.1016/j.learninstruc.2018.01.012 Legate, N., Nguyen, T., Weinstein, N., Moller, A., & Legault, L. (2021). A global experiment MOTIVATION MESSAGES 16 on motivating social distancing during the COVID-19 pandemic. PsyArXiv. https://doi.org/10.31234/osf.io/n3dyf Leo, F. M., Mouratidis, A., Pulido, J. J., López-Gajardo, M. A., & Sánchez-Oliva, D. (2020). Perceived teachers’ behavior and students’ engagement in physical education: The mediating role of basic psychological needs and self-determined motivation. Physical Education and Sport Pedagogy, 0(0), 1–18. https://doi.org/10.1080/17408989.2020.1850667 León, J., Medina-Garrido, E., & Núñez, J. L. (2017). Teaching quality in math class: The development of a scale and the analysis of its relationship with engagement and achievement. Frontiers in Psychology, 8, 1–14. https://doi.org/10.3389/fpsyg.2017.00895 León, J., Medina-Garrido, E., & Ortega, M. (2018). Teaching quality: High school students’ autonomy and competence. Psicothema, 30(2), 218–223. https://doi.org/10.7334/psicothema2017.23 León, J., Núñez, J. L., & Liew, J. (2015). Self-determination and STEM education: Effects of autonomy, motivation, and self-regulated learning on high school math achievement. Learning and Individual Differences, 43, 156–163. https://doi.org/10.1016/j.lindif.2015.08.017 Liu, J., Bartholomew, K. J., & Chung, P. K. (2017). Perceptions of teachers’ interpersonal styles and well-being and ill-being in secondary school physical education students: The role of need satisfaction and need frustration. School Mental Health, 9(4), 360–371. https://doi.org/10.1007/s12310-017-9223-6 Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B. (2008). The multilevel latent covariate model: A new, more reliable approach to grouplevel effects in contextual studies. Psychological Methods, 13(3), 203–229. https://doi.org/10.1037/a0012869 Lüdtke, O., Robitzsch, A., Trautwein, U., & Kunter, M. (2009). Assessing the impact of learning environments: How to use student ratings of classroom or school characteristics in multilevel modeling. Contemporary Educational Psychology, 34(2), 120–131. https://doi.org/10.1016/j.cedpsych.2008.12.001 MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7(1), 83–104. https://doi.org/10.1037/1082-989X.7.1.83 Marsh, H. W., Lüdtke, O., Nagengast, B., Trautwein, U., Morin, A. J. S., Abduljabbar, A. S., & Köller, O. (2012). Classroom climate and contextual effects: Conceptual and methodological issues in the evaluation of group-level effects. Educational Psychologist, 47(2), 106–124. https://doi.org/10.1080/00461520.2012.670488 Marsh, H. W., Lüdtke, O., Robitzsch, A., Trautwein, U., Asparouhov, T., Muthén, B., & Nagengast, B. (2009). Doubly-latent models of school contextual effects: Integrating multilevel and structural equation approaches to control measurement and sampling error. Multivariate Behavioral Research, 44(6), 764–802. https://doi.org/10.1080/00273170903333665 Marsh, H. W., Seaton, M., Trautwein, U., Lüdtke, O., Hau, K. T., O’Mara, A. J., & Craven, R. G. (2008). The big-fish-little-pond-effect stands up to critical scrutiny: Implications for theory, methodology, and future research. Educational Psychology Review, 20(3), MOTIVATION MESSAGES 17 319–350. https://doi.org/10.1007/s10648-008-9075-6 Mayer, K. J., & Sparrowe, R. T. (2013). Integrating theories in AMJ articles. Academy of Management Journal, 56(4), 917–922. https://doi.org/10.5465/amj.2013.4004 Morin, A. J. S., Marsh, H. W., Nagengast, B., & Scalas, L. F. (2014). Doubly latent multilevel analyses of classroom climate: An illustration. The Journal of Experimental Education, 82(2), 143–167. https://doi.org/10.1080/00220973.2013.769412 Muthen, L. K., & Muthén, B. O. (2021). Mplus user’s guide (8th ed) (Eighth). Muthén & Muthén. https://doi.org/10.1111/j.1600-0447.2011.01711.x Nicholson, L. J., Putwain, D. W., Nakhla, G., Porter, B., Liversidge, A., & Reece, M. (2019). A person-centered approach to students’ evaluations of perceived fear appeals and their association with engagement. Journal of Experimental Education, 87(1), 139–160. https://doi.org/10.1080/00220973.2018.1448745 Núñez, J. L., Martín-Albo, J., & Navarro, J. G. (2005). Validación de la versión española de la Échelle de Motivation en Éducation. Psicothema, 17(2), 344–349. Oostdam, R. J., Koerhuis, M. J. C., & Fukkink, R. G. (2019). Maladaptive behavior in relation to the basic psychological needs of students in secondary education. European Journal of Psychology of Education, 34(3), 601–619. https://doi.org/10.1007/s10212018-0397-6 Putwain, D. W., & Remedios, R. (2014). The scare tactic: Do fear appeals predict motivation and exam scores? School Psychology Quarterly, 29(4), 503–516. https://doi.org/10.1037/spq0000048 Putwain, D. W., & Symes, W. (2011). Teachers’ use of fear appeals in the Mathematics classroom: Worrying or motivating students? British Journal of Educational Psychology, 81(3), 456–474. https://doi.org/10.1348/2044-8279.002005 Putwain, D. W., & Symes, W. (2016). Expectancy of success, subjective task-value, and message frame in the appraisal of value-promoting messages made prior to a high-stakes examination. Social Psychology of Education, 19(2), 325–343. https://doi.org/10.1007/s11218-016-9337-y Putwain, D. W., Symes, W., & McCaldin, T. (2019). Teacher use of loss-focused, utility value messages, prior to high-stakes examinations, and their appraisal by students. Journal of Psychoeducational Assessment, 37(2), 169–180. https://doi.org/10.1177/0734282917724905 Putwain, D. W., Symes, W., & Wilkinson, H. M. (2017). Fear appeals, engagement, and examination performance: The role of challenge and threat appraisals. British Journal of Educational Psychology, 87(1), 16–31. https://doi.org/10.1111/bjep.12132 Reeve, J. (2009). Why teachers adopt a controlling motivating style toward students and how they can become more autonomy supportive. Educational Psychologist, 44(3), 159–175. https://doi.org/10.1080/00461520903028990 Reeve, J., & Cheon, S. H. (2016). Teachers become more autonomy supportive after they believe it is easy to do. Psychology of Sport and Exercise, 22, 178–189. https://doi.org/10.1016/j.psychsport.2015.08.001 Rothman, A. J., & Salovey, P. (1997). Shaping perceptions to motivate healthy behavior: The role of message framing. Psychological Bulletin, 121(1), 3–19. https://doi.org/10.1037/0033-2909.121.1.3 MOTIVATION MESSAGES 18 Ruiz-Alfonso, Z., & León, J. (2017). Passion for math: Relationships between teachers’ emphasis on class contents usefulness, motivation and grades. Contemporary Educational Psychology, 51, 284–292. https://doi.org/10.1016/j.cedpsych.2017.08.010 Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–78. https://doi.org/10.1037/0003-066X.55.1.68 Ryan, R. M., & Deci, E. L. (2020). Intrinsic and extrinsic motivation from a selfdetermination theory perspective: Definitions, theory, practices, and future directions. Contemporary Educational Psychology, 61, 101860. https://doi.org/10.1016/j.cedpsych.2020.101860 Santana-Monagas, E., Núñez, J. L., Loro, J. F., Huéscar, E., & León, J. (2022). Teachers’ engaging messages: The role of perceived autonomy, competence and relatedness. Teaching and Teacher Education, 109, 103556. https://doi.org/10.1016/j.tate.2021.103556 Scherrer, V., & Preckel, F. (2019). Development of motivational variables and self-esteem during the school career: A meta-analysis of longitudinal studies. Review of Educational Research, 89(2), 211–258. https://doi.org/10.3102/0034654318819127 Schmitt, T. A. (2011). Current methodological considerations in exploratory and confirmatory factor analysis. Journal of Psychoeducational Assessment, 29(4), 304–321. https://doi.org/10.1177/0734282911406653 Stapleton, L. M., Yang, J. S., & Hancock, G. R. (2016). Construct meaning in multilevel settings. Journal of Educational and Behavioral Statistics, 41(5), 481–520. https://doi.org/10.3102/1076998616646200 Symes, W., & Putwain, D. W. (2016). The role of attainment value, academic self-efficacy, and message frame in the appraisal of value-promoting messages. British Journal of Educational Psychology, 86(3), 446–460. https://doi.org/10.1111/bjep.12117 Taylor, G., Jungert, T., Mageau, G. A., Schattke, K., Dedic, H., Rosenfield, S., & Koestner, R. (2014). A self-determination theory approach to predicting school achievement over time: The unique role of intrinsic motivation. Contemporary Educational Psychology, 39(4), 342–358. https://doi.org/10.1016/j.cedpsych.2014.08.002 Vansteenkiste, M., Ryan, R. M., & Soenens, B. (2020). Basic psychological need theory: Advancements, critical themes, and future directions. Motivation and Emotion, 44(1), 1– 31. https://doi.org/10.1007/s11031-019-09818-1 Vansteenkiste, M., Sierens, E., Goossens, L., Soenens, B., Dochy, F., Mouratidis, A., Aelterman, N., Haerens, L., & Beyers, W. (2012). Identifying configurations of perceived teacher autonomy support and structure: Associations with self-regulated learning, motivation and problem behavior. Learning and Instruction, 22(6), 431–439. https://doi.org/10.1016/j.learninstruc.2012.04.002 Weinstein, N., Vansteenkiste, M., & Paulmann, S. (2019). Listen to your mother: Motivating tones of voice predict adolescents’ reactions to mothers. Developmental Psychology, 55(12), 898. Weinstein, N., Zougkou, K., & Paulmann, S. (2018). You “have” to hear this: Using tone of voice to motivate others. Journal of Experimental Psychology: Human Perception and Performance, 44(6), 898–913. https://doi.org/10.1037/xhp0000502 MOTIVATION MESSAGES 19 Supplementary material Supplementary Table 1 Model Fit Indices for the Multilevel CFAs of the Different Models Tested Model Hypothesised ninefactor model Unidimensional model Two-factor model Factors 1. 2. 3. 4. 5. 6. 7. 8. 9. 1. 2. Five-factor model (1) 1. 2. 3. 4. Five-factor model (2) 5. 1. 2. 3. 4. 5. G-Intrinsic L-Intrinsic G-Identified L-Identified G-Introjected L-Introjected G-Extrinsic L-Extrinsic Amotivation All variables Gain-framed messages Loss-framed messages G-Intrinsic and LIntrinsic G-Identified and L-Identified G-Introjected and L-Introjected G-Extrinsic and L-Extrinsic Amotivation G-Intrinsic and GIdentified G-Extrinsic and G-Introjected L-Intrinsic and LIdentified L-Extrinsic and LIntrojected Amotivation RMSEA CFI TLI SRMRw SRMRb 1873.427(1208, 1143) .028 .971 .968 .049 .138 67356.028 (1208, 1224) .211 .649 .638 .222 .502 45686.636 (1208, 1230) .173 .760 .754 .126 .341 45699.061 (1208, 1204) .175 .759 .748 .153 .325 9937.970 (1208, 1204) .077 .953 .951 .064 .258 χ² Note. χ² of all models was p <.001. G= Gain-framed; L= Loss-framed. MOTIVATION MESSAGES Supplementary Table 2 Factor Loadings for the Teachers’ Engaging Messages Scale Factor Item My teacher tells me that if I work hard… 1. I will enjoy this subject G-Intrinsic 2. I will appreciate new discoveries 3. I will learn interesting facts 4. I will have fun doing class work 5. I will be able to choose what to study 6. I will be prepared for high-qualified jobs G-Identified 7. I will be able to work on what I would like 8. I will be prepared for my future studies 9. I will feel important 10. I will feel proud of myself G-Introjected 11. I will feel satisfied 12. I will feel appreciated 13. I will have free time G-Extrinsic 14. I will receive a reward (sticker, star, etc.) 15. I will be able to do in class the activities I want 16. I will receive compliments My teacher tells me that unless I work hard … 17. I will miss the opportunity to understand interesting issues L-Intrinsic 18. I will miss the beauty of this subject 19. I will miss the joy of finishing exercises 20. I will miss the opportunity to increase my knowledge 21. I will not get anywhere in life L-Identified 22. I will only be able to get low paid jobs 23. I will have a tough life 24. I will have to study the less demanded degrees 25. I will feel like a failure 26. I will feel disappointed L-Introjected 27. I will feel sad 28. I will feel ashamed 29. I will get in trouble L-Extrinsic 30. I will be punished 31. I will miss my break 32. I will get my parents angry My teacher tells me that it does not matter if… 33. I work hard, I will fail anyway Amotivation 34. I come to class, I will fail anyway 35. I do the homework, I will fail anyway 36. I pay attention in class, I will fail anyway Note. G= Gain-framed; L= Loss-framed. 20 Factor loadings .691 .769 .807 .733 .756 .765 .813 .810 .762 .858 .846 .829 .592 .542 .583 .782 .651 .720 .783 .751 .735 .840 .887 .843 .856 .834 .897 .878 .842 .723 .681 .820 .929 .945 .957 .957 MOTIVATION MESSAGES Supplementary Table 3 Factor Loadings for the Échelle de Motivation en Éducation Scale Factor Item Intrinsic motivation Identified motivation Introjected motivation External motivation Amotivation Why do you study? 1. Because it is a pleasure and satisfaction for me to learn new things. 2. For the pleasure of discovering new things 21 Factor loadings .791 .830 3. For the pleasure of knowing more about the subjects I am attracted to. .797 4. .888 5. 6. Because studying allows me to continue learning many things that interest me Because I think that studying will help me in the future Because it will help me find a job I like. 7. Because it will help me to make a better career choice .760 8. Because studying will make me better at my job .800 .776 .805 9. To prove to myself that I am capable of finishing my studies 10. Because passing my studies will make me feel important .709 .683 11. To prove to myself that I am an intelligent person .792 12. Because I want to prove to myself that I am capable of succeeding in my studies. 13. Because without secondary I would not be able to find a well-paid job .895 14. To be able to get a well-paid job in the future. .864 15. Because in the future I want to have a "good life". .869 16. To have a better salary in the future .782 17. I honestly don't know, I think I'm wasting my time at school. .872 18. I used to have good reasons to study, but now I wonder if it is worth continuing. 19. I don't know why, honestly, I don't care. .711 20. I don't know, I don't understand what I do at school. .927 .275 .849 MOTIVATION MESSAGES 22 Supplementary Table 4 Fit Indices for the Multilevel CFA of the Partially and Fully Mediated ML-SEM Models Model Mediation χ² RMSE CFI TLI SRMR-w SRMR-b A 1 G-intrinsic Fully mediated 163.626 (1208, 62) .037 .994 .993 .034 .072 Partially 159.516 (1208, 60) .037 .994 .993 .033 .059 mediated 2 G-Identified Fully mediated 101.668 (1208,62) .023 .993 .992 .039 .311 Partially 111.363(1208, 60) .027 .991 .989 .039 .210 mediated 3 G-Introjected Fully mediated 406.851 (1208, 62) .068 .980 .977 .049 .143 Partially 429.246 (1208, 60) .071 .978 .974 .049 .144 mediated 4 G-Extrinsic Fully mediated 193.288 (1208,62) .042 .980 .977 .048 .244 Partially 198.626 (1208, 60) .044 .979 .975 .048 .213 mediated 5 L-Intrinsic Fully mediated 169.319 (1202, 62) .038 .993 .992 .036 .114 Partially 175.448 (1202, 60) .040 .993 .992 .033 .105 mediated 6 L-Identified Fully mediated 83.510 (1202, 62) .017 .998 .998 .035 .471 Partially 86.569 (1202, 60) .019 .998 .998 .032 .338 mediated 7 L-Introjected Fully mediated 697.683 (1208, 62) .092 .950 .942 .085 .205 Partially Not identified mediated 8 L-Extrinsic Fully mediated 238.915 (1202, 62) .049 .979 976 .060 .218 Partially 246.314 (1202, 60) .051 .978 .973 .058 .224 mediated 9 Amotivation Fully mediated 108.988 (1208, 62) .025 .998 .998 .040 .105 Partially 118.040 (1208, 60) .028 .998 .997 .039 .106 mediated Note: CFA = confirmatory factor analysis; χ² = Chi-square; RMSEA = root mean square error of approximation; CFI = comparative fit index; TLI = Tucker–Lewis index; SRMRw = standardized root mean square residual within level; SRMRb = standardized root mean square residual between level; G= Gain-framed; L= Loss-framed. MOTIVATION MESSAGES 23 Supplementary Table 5 Unstandardized Direct Effects from the ML-SEMs Path 1 Model Level TEM  MTL ß SE 95% CI G-Intrinsic L-Intrinsic G-Identified L-Identified G-Introjected L-Introjected G-Extrinsic L-Extrinsic Amotivation L2 L1 L2 L1 L2 L1 L2 L1 L2 L1 L2 L1 L2 L1 L2 L1 L2 L1 0.33 0.58 0.25 0.41 0.11 0.48 0.05 0.10 0.36 0.41 0.73 0.34 0.18 0.20 0.14 0.04 0.32 0.30 0.09 0.04 0.22 0.04 0.12 0.05 0.13 0.04 0.09 0.04 0.21 0.03 0.07 0.03 0.06 0.02 0.08 0.04 .18, .48 .51, .65 -.11, .61 .34, .48 -.08, .11 .40, .55 -.17, .26 .04, .15 .21, .51 .35,.48 .39, 1.07 .28, .39 .07, .29 .14, .26 .04, .24 .02, .06 .20, .45 .24, .36 Path 2 MTL  Academic performance ß SE 95% CI 0.42 0.15 0.54 0.13 -1.82 0.14 -13.93 0.13 -0.65 0.04 -0.76 0.03 -1.54 0.16 -3.18 0.15 -0.77 -0.13 0.25 0.03 0.26 0.03 3.42 0.03 40.21 0.04 0.49 0.04 0.48 0.03 1.18 0.08 1.98 0.08 0.24 0.02 .01, .83 .10, .19 .12, .96 .09, .17 -7.44, 3.80 .08, .19 -80.07, 52.21 .08, .19 -1.45, .16 -.03, .10 -1.55, .03 -.02, .09 -3.48, .41 .03, .28 -6.43, .07 .03, .28 -1.15, -.38 -.17, -.10 Note. TEM= Teachers’ engaging messages; MTL=Motivation to learn; G= Gain-framed; L= Loss-framed; L2=Teacher level; L1=Student level.