Applying multimodal learning analytics to examine the immediate and delayed effects of instructor scaffoldings on small groups’ collaborative programming

Background

Instructor scaffolding is proved to be an effective means to improve collaborative learning quality, but empirical research indicates discrepancies about the effect of instructor scaffoldings on collaborative programming. Few studies have used multimodal learning analytics (MMLA) to comprehensively analyze the collaborative programming processes from a process-oriented perspective. This research conducts a MMLA research to examine the immediate and delayed effects of instructor scaffoldings on small groups’ collaborative programming in K-12 education context with an aim to provide research, analytics, and pedagogical implications.

Results

The results indicated that the instructor provided five types of scaffoldings from the social, cognitive, and metacognitive dimensions, and groups had seven types of responses (i.e., immediate uptake and delayed use) to five instructor scaffoldings, ranging from the low-to-medium and high level of cognitive engagement. After the scaffolding was faded, groups used the content from the high-control cognitive scaffolding frequently to solve problems in a delayed way, but groups did not use the instructor’s scaffolding content from the social and low-control cognitive scaffoldings from the pedagogical perspective, instructors should consider scaffolding types, group states and characteristics, as well as the timing of scaffolding to better design and facilitate collaborative programming. From an analytical perspective, MMLA was proved to be conducive to understand collaborative learning from social, cognitive, behavioral, and micro-level dimensions, such that instructors can better understand and reflect on the process of collaborative learning, and use scaffoldings more skillfully to support collaborative learning.

Conclusions

Collaborative programming is encouraged to be integrated in STEM education to transform education from the instructor-directed lecturing to the learner-centered learning. Using MMLA methods, this research provided a deep understanding of the immediate and delayed effects of instructor scaffoldings on small groups’ collaborative programming in K-12 STEM education from a process-oriented perspective. The results showed that various instructor scaffoldings have been used to promote groups’ social and cognitive engagement. Instructor scaffoldings have delayed effects on promoting collaborative programming qualities. It is highly suggested that instructors should integrate scaffoldings to facilitate computer programming education and relevant research should apply MMLA to reveal details of the process of collaboration.

In computer-supported collaborative learning (CSCL), small groups of students engage in higher level cognitive activities (e.g., ill-structured problem-solving) to create knowledge and relevant artifacts through sustained interactions, communications, and actions (Dillenbourg, 1999; Goodyear et al., 2014; Roschelle & Teasley, 1995). Such attributes manifest during collaborative computer programming where learners interact and collaborate with peers to learn programming languages, solve programming problems, and improve computational thinking (Demir & Seferoglu, 2020a; Plonka et al., 2015; Umapathy & Ritzhaupt, 2017). However, it is difficult for novice programmers to achieve a high quality of collaborative programming without external assistance in K-12 education (Hwang et al., 2012). Empirical studies have indicated that instructors usually need to provide scaffoldings, use scripted roles, and create a socially supportive atmosphere to improve the quality of collaboration (Fanchamps et al., 2021; Sun & Hsu, 2019).

However, empirical research shows discrepancies about the effect of instructor scaffoldings on collaborative learning. Prior empirical studies have indicated that major types of scaffoldings instructors include social scaffolding (e.g., Grossen & Bachmann, 2000; Perret-Clermont, 1980), motivational scaffolding (e.g., Gresalfi et al., 2009; Roehler & Cantlon, 1997) and cognitive scaffolding (e.g., Nedić et al., 2015; Tartas & Perret-Clermont, 2008). However, the instructor scaffoldings lead to different effects regarding the immediate uptake (the group completes the ongoing task) and delayed use (the group solves problems after the scaffolding is faded) in CSCL (Barron, 2003; Wittwer & Renkl, 2008). After reviewing literature, we find a lack of mature standard to evaluate the immediate and delayed effect of instructor scaffoldings on collaborative programming (Grévisse et al., 2019; Zheng et al., 2022). Moreover, most previous research have collected and analyzed a single data source (e.g., discourses) to understand the process of collaboration (e.g., van de Pol et al., 2019). Compared to the prior approach, the emergence of multimodal learning analytics (MMLA) has potential to better take advantage of multi-source data and comprehensively analyze the collaborative programming processes (Damşa & Nerland, 2016; Socratous & Ioannou, 2021; Sun et al., 2021). Moreover, the analytics of immediate and delayed effects is beneficial to reveal details of how instructor scaffoldings influence groups’ collaborative programming from a process-oriented perspective (Brown & Renshaw, 2009). Therefore, it is necessary to further conduct a MMLA research to examine how the instructor scaffoldings influence the group’s immediate uptake and delayed use from a temporal perspective, with an aim to provide research, analytics, and pedagogical implications for collaborative programming in K-12 education.

This research aims to empirically examine the immediate and delayed effects of instructor scaffoldings on small groups’ collaborative programming in K-12 education. To achieve this goal, this research collects multimodal data from collaborative programming and uses MMLA (including click stream analysis, class video analysis, and quantitative content analysis) to analyze small groups’ collaborative programming in China’s elementary education. Based on empirical research results, this research proposes pedagogical and analytical implications for the practice and research of instructor scaffolding on collaborative programming education.

Computer programming has been widely studied in computer science education, which can be considered as a sub-branch of science, technology, engineering and mathematics (STEM) education (Fedorenko et al., 2019; Guzdial & Morrison, 2016). Collaborative programming, as a CSCL mode, supports two or more learners working together at one workstation or remotely online to solve the same programming problems (Zheng, 2021). Collaborative programming enables students to interact and communicate with their peers while learning programing languages and debugging codes (Serrano-Cámara et al., 2014). Empirical studies have indicated that collaborative programming is beneficial to improve students’ computer programming skills (Lin et al., 2018), to establish collective understanding of problems (Teague & Roe, 2008), and to develop computational thinking skills (Wei et al., 2020). However, collaborative programming requires students to use collaborative problem-solving strategies, higher-order programming logics and computational thinking skills, which usually bring up additional obstacles and difficulties, especially for novice students without sufficient programming knowledge and skills (Wang & Hwang, 2017). Major obstacles include establishing a positive collaborative relationship (Demir & Seferoglu, 2020b), sustaining a high-level knowledge construction (Papadakis, 2018), and making continuous knowledge innovation to solve problems (Veerasamy et al., 2019). Therefore, groups usually need external assistance or support from the instructor to achieve a high quality of collaborative programming and adjust the subsequent learning (e.g., Tohyama et al, 2018; Zheng, 2021).

From the theoretical and practical perspective, instructor scaffoldings are used to facilitate collaborative learning process and performance (Hmelo-Silver & DeSimone, 2013; Kirschner et al., 2006; Stahl, 2009), particularly for STEM education. Primary scaffoldings include cognitive, metacognitive, or social scaffoldings. Cognitive scaffolding aims to facilitate the groups’ knowledge-construction or problem-solving when the instructor provides additional explanations of the learning tasks, content or problems to be solved (Sandoval & Reiser, 2004; Zheng, 2021). Cognitive scaffolding can range from a low- to a high-level control, such as asking an open-ended question to directly providing answers. Second, metacognitive scaffolding aims to facilitate learners’ planning, monitoring, and reflection of peers’ and their own learning (Johnson, 2019; Kiemer et al., 2015; Kim et al., 2022). Research results show that the instructor’s metacognitive scaffolding is beneficial to help groups reflect on the collaborative learning, enhance group awareness, and lead groups to transition from instructor regulation to group regulation (Kwon et al., 2013; Ouyang et al., 2022; Rasku-Puttonen et al., 2003). Third, social scaffolding aims to facilitate social interactions and promote social emotions during collaborative learning (Belland et al., 2017; Ouyang & Xu, 2022). Instructors use temperate social scaffolding to resolve group interpersonal disputes, to create a positive collaborative environment, and to encourage students to share their thoughts. Collaborative programming requires students utilize their cognitive, affective and social competencies to facilitate the group’s social interactions, decision-making and problem-solving process in order to construct programming and assess the outcomes (Clark & Sengupta, 2020; Lai & Wong, 2022; Lavonen et al., 2002). Therefore, instructor scaffolding is a necessity during collaborative programming, that have been showed positive influences on groups’ collaborative learning (Socratous & Ioannou, 2022).

The instructors’ use of different types of scaffoldings has different effects on collaborative learning, usually reflected by the group’s immediate uptake and delayed use of scaffoldings during collaboration (van de Pol et al., 2019; Webb & Farivar, 1999; Webb et al., 2006). The immediate effect of the instructor scaffolding aims to help groups reduce cognitive load, ease interpersonal conflicts, and adjust the collaborative process, in order to complete the ongoing tasks (van de Pol et al., 2010). The delayed effect of the scaffolding is reflected by the group’s integration and application of the information into existing knowledge schemes to solve problems after the instructor’s scaffolding is faded (Webb & Farivar, 1999; Wittwer & Renkl, 2008). Empirical studies have indicated a positive relationship between groups’ immediate uptake to scaffoldings and the groups’ collaborative experiences or performances. For example, Grévisse et al. (2019) used a scaffolding platform to help students integrate relevant materials in a programming environment and examined the effect of the scaffolding. The results showed that compared to the traditional learning process, students actively used scaffolding to increase participation in collaborative programming and highly appraised the scaffolding. Lin et al. (2018) employed a cooperative programming editing tool as a scaffolding to teach programming. The results showed that compared with the control group without scaffolding support, the groups that received the scaffolding were more prone to participate in and reflect on collaborative programming. Moreover, related studies have focused on investigating how students integrate and apply the contents of the instructor scaffoldings in a delayed way and empirically examine its delayed effect on collaboration. For example, van de Pol et al. (2019) found that groups tended to formulate more accurate answers when they applied the instructor scaffoldings in subsequent group tasks, compared to the groups that ignored the instructor’s scaffolding. However, few studies have been located to explore how groups’ delayed responses to the instructor’s scaffolding influence their subsequent learning in collaborative programming. Therefore, it is necessary to further examine the immediate and delayed effects of instructor scaffoldings on collaborative programming and provide instructional strategies to improve collaborative programming quality based on empirical research results.

From an analytical perspective, with the development of MMLA, researchers can take advantage of audio, video, and other forms of multimodal data to understand the discourses, emotions, and behaviors of students in the group, which is conducive to gain a comprehensive understanding of collaborative programming. For example, Damşa and Nerland (2016) collected video-recordings of discussions, online messages, semi-structured interviews and final programming products, and analyzed students’ participation, characteristics of the inquiry tasks, and use of knowledge resources. Sun et. al. (2021) used the click stream analysis, classroom video analysis, and discourses analysis to identify pairs’ programming behaviors, discourses, and perceptions. Socratous and Ioannou (2021) collected and analyzed classroom video recordings, tablet screen-recordings and audio recordings and post-debugging tests data on collaborative programming, to compare the effect of a structured and an unstructured educational robotics curriculum. Compared to analytics driven by a single data source, MMLA can better enrich the data collections of students’ learning processes to help researchers understand the processes and environments of collaborative learning (Chatti et al., 2017; Di Mitri et al., 2021; Samuelsen et al., 2019). Moreover, multiple learning analytical methods are usually used in MMLA to analyze those data from multiple sources, including click stream analysis (e.g., Sun et al., 2021), video analysis (e.g., Khan, 2017), content analysis (e.g., Socratous & Ioannou, 2021), and discourse analysis (e.g., Gaul & Kim, 2020). In addition, collaborative programming is a dynamic process, consisting of a gradual changing process of problem-solving, meaning-making, and knowledge-construction (Lewis, 2012; Sun et al., 2021; Wu et al., 2019). Therefore, it is necessary to examine student groups’ discourses, in-class behaviors, and computer operations from a process-oriented perspective, to fully understand the details of collaborative programming. Previous studies have examined the temporal change of the groups’ verbal and embodied actions in STEM education (Riikonen et al., 2020), the student–student and student–instructor interactions during collaborative process (Kynigos & Diamantidis, 2022) and the groups’ gaze patterns during pairs programming processes (Villamor & Rodrigo, 2019). More importantly, the focus of the current research is to examine the immediate uptake and delayed use of instructor scaffoldings, which in particular focuses on the temporal attributes and requires a detailed examination of the change process (Gidalevich & Kramarski, 2019; Holmes et al., 2014; Jadallah et al., 2011). Therefore, the temporal analysis is beneficial to comprehensively analyze and understand the influence of instructor scaffoldings on collaborative learning (Bloome et al., 2009; Brown & Renshaw, 2009; Wiig et al., 2017). In summary, it is necessary to collect multimodal data and use multimodal learning analytics from a temporal perspective to examine the effects of instructor scaffoldings on collaborative programming.

To address those research and practice gaps, this research used MMLA to investigate the immediate and delayed effects of instructor scaffoldings on small groups’ collaborative programming. The overarching research purposes were twofold: (1) to improve the quality of collaborative programming through using instructor scaffoldings, and (2) to empirically examine the immediate and delayed effects of instructor scaffoldings on collaborative programming.

To achieve these purposes, this research used MMLA to collect multimodal data from students’ collaborative process and analyze small groups’ collaborative programming process before, during and after instructor scaffoldings. The research questions were: (1) How did the instructor use five types of scaffoldings during the groups’ collaborative programming process? (2) What were the immediate effects of instructor scaffoldings on collaborative programming? and (3) What were the delayed effects of instructor scaffoldings on collaborative programming? Specifically, multimodal process data were collected, including audio data, class video data and computer screen-recording data. Multiple learning analytics methods were used, including primary analysis (click stream analysis, video analysis, content analysis, and qualitative micro-level analysis) as well as secondary analysis (statistical analysis, social network analysis, epistemic network analysis and temporal analysis), to demonstrate the immediate and delayed effects of different instructor scaffoldings. According to these results, this research proposed pedagogical and analytical implications for future practices of instructor scaffoldings in collaborative programming.

Research context and participants

The research context was an extracurricular course titled “Artificial Intelligence in Maker Education” offered at an elementary school during Spring 2021 in the Eastern area of China. The subject of this course was one of the curricula in K-12 primary education in China (Chinese Ministry of Education, 2019). AI education helps K-12 students understand what the emerging AI technologies are and how they work. This course designed the computer programming practice to deepen students’ understandings of AI-related knowledge and applications (Chiu, 2021; Pedró et al., 2019). Kittenblock, a graphical programming environment based on Scratch, was used in this course to enhance novice programmers’ complex problem-solving and computational thinking skills (Tohyama et al., 2018). The course was taught by a female instructor, who was a doctoral student in the educational technology program at a top research-intensive university in China. She had 1 year of teaching experience in K-12 education and good understandings of different types of scaffoldings in collaborative learning. Twenty-eight 3rd to 5th graders (9 females, 19 males) enrolled in this course. All students were new to the course content, and new to computer programming. The instructor randomly arranged students into nine groups (three students/group) at the beginning of the course and the groups were fixed throughout the course.

The instructor designed two instruction and learning phases throughout 11 weeks (see Fig. 1), namely the instruction and learning phase (including instructor lectures and collaborative programming activities) (Week 1–8), and the project-based learning (PBL) phase (including collaborative mind mapping and project-based collaborative programming activities) (Week 9–11). During Week 1–8, all groups were required to complete eight collaborative programming tasks, such as text recognition, speech recognition, and image recognition, etc. During Week 9–11, all groups applied what they learned in Week 1–8 to design and produce their own innovative programming works, such as anti-fraud smartphones, smart glasses and smart crutches.

The instructional procedure

Full size image

In the instruction and learning phase (Week 1–8), the instructor used the traditional lecturing mode to deliver course content (see Fig. 2a). Then the instructor explained the collaborative tasks to students and asked the groups to discuss group plans to complete tasks (see Fig. 2b). Each group conducted the collaborative programming on one shared computer and used Kittenblock software, a block-based visual programming software to complete tasks (see Fig. 2d). During Week 1–8, each group completed eight tasks in total, including voice recognition, license plate recognition, color recognition, etc. In the PBL phase (Week 9–11), each group applied the knowledge learned in the previous lectures and used programming skills to design and make a product (such as anti-fraud smartphone, smart magnifying glass and smart crutches, etc.). In Week 9–10, the groups designed the product through drawing a mind map collectively; the purpose was to discuss and determine the user requirements, product functions, programming modules and required materials. In Week 11, the groups carried out collaborative programming based on the mind map and completed the programming task. Throughout the course (Week 1–11), when the groups discussed or conducted collaborative programming, the instructor walked around the classroom to observe and provided scaffoldings in terms of the groups’ current states (see Fig. 2c).

In-class photos and computer screenshots. a The instructor’s lectures; b collaborative programming; c the instructor provided scaffoldings; d a screenshot of the Kittenblock software; e collaborative drawing of mind maps; and f a completed mind map by a group

Full size image

Data collection and analysis approaches

This research collected and analyzed multimodal data through three ways (see Fig. 3). First, four cameras were placed in four different positions in the classroom to record the in-class behaviors of the instructor and all students. Then, the computer screens of nine groups were recorded to acquire students’ clickstreams of computer programming operations. To ensure the consistency of multimodal data sources, the current research used in-class video data, discourse audios, and computer screen-recording data during collaborative programming (Week 1–8 and Week 11). About 18 h of in-class video data (without audio) and 32 h of computer screening data (with audio) were collected and analyzed. Students were informed about the research purpose and data collection process; all of them agreed the data collection and signed consent forms before the course started with the assistance from the school teachers.

The proposed analytical framework

Full size image

Referring to the MMLA in CSCL as the rationale (Cress et al., 2021), primary analysis methods included click stream analysis, video analysis, and content analysis. There were three steps in the analysis process. The first step was to transcribe class video, computer screening and audio data into excel files in the chronological order. There were initially 81 files (nine groups * 9 weeks), of which 62 contained the instructor scaffoldings. Each file recorded the instructor and a group’s conversational discourses, computer programming operations, and in-class behaviors in each class. Moreover, 5-s time interval was used as the unit for transcription. The time-based segmentation gave a structured unit for analysis and also allowed a temporally unfolding overview of the group collaborative learning (Sinha et al., 2015). In most cases of our data, the 5-s interval was enough for the students to speak a complete sentence, produce a physical behavior, or complete a computer programming operation. Next, we divided all multimodal data into three phases with a 20-s segment (see Fig. 3): (1) Phase 1: 20 s before the instructor provided a scaffolding; (2) Phase 2: 20 s after the instructor provided a scaffolding, as the range of the immediate uptake; and (3) Phase 3: 20 s after Phase 2 until the end of a class, as the range of the delayed use of instructor scaffolding. The rationale for choosing 20-s as the interval depended on cases generated from our data.

In the second step, content analysis, video analysis, click stream analysis methods were used to code the transcribed data. Two coders first individually watched the computer screening and the in-class videos to proofread the excel file, and then determined the initial three coding frameworks, namely collaborative group states, instructor scaffoldings and groups’ immediate uptakes. Then two coders had discussions to achieve an agreement of the final coding frameworks of the collaborative group states (see Table 1), instructor scaffoldings (see Table 2) and groups’ immediate uptakes of instructor scaffoldings (see Table 3). There were two considerations in the coding process. First, for the instructor scaffoldings, if an instructor scaffolding was longer than 5 s, only the last instructor scaffolding which the group reacted to was coded. Second, immediate uptakes from students in the same group were all coded. For example, the same immediate uptakes from two students within the group were recorded twice. Then, two coders independently coded the data again in a chronological order based on the final coding frameworks, and reached an inter-rater reliability with the Cohen’s Kappa of 0.872.

Finally, based on the primary analysis of coding, the secondary analysis was used to demonstrate the immediate and delayed effects of instructor scaffoldings on collaborative programming, including statistical analysis (SA), social network analysis (SNA), epistemic network analysis (ENA), and temporal analysis (TA) (see Fig. 4). Regarding the effect of immediate uptake, SA (including analysis of variance, analysis of covariance, and Sankey visualization) was used to demonstrate the overall transitional frequencies from the collaborative group states to instructor scaffoldings and to groups’ immediate uptakes. SNA was used to examine the social networks of instructor–student interaction and student–student interaction after the instructor scaffoldings were provided based on groups’ in-class behaviors, computer operations and communication discourses. ENA was used to examine the overall structure of group’s immediate uptakes based on co-occurrence relationships of codes and was visualized in a network model to demonstrate the immediate effect (Shaffer & Ruis, 2017; Shaffer et al., 2009, 2016). Qualitative micro-level analysis was used to examine the delayed use of the groups to instructor scaffoldings based on groups’ in-class behaviors, computer operations and communication discourses, and TA was used to visualize the micro-level process in the form of time series diagrams.

Secondary analysis framework

Full size image

RQ1: How did the instructor use five types of scaffoldings during the groups’ collaborative programming process?

The Sankey diagram demonstrated the transitional frequencies from collaborative group states, to instructor scaffoldings, and to groups’ uptakes of scaffoldings (see Fig. 5). The instructor provided a total frequency of 919 scaffoldings. For the instructor scaffolding codes, there were 84 codes of social scaffolding (SS) (9.14%), 225 codes of metacognitive scaffolding (MS) (24.48%), 231 codes of low-control cognitive scaffolding (CS-L) (25.14%), 219 codes of medium-control cognitive scaffolding (CS-M) (23.83%), and 160 codes of high-control cognitive scaffolding (CS-H) (17.41%). For the group state codes, there were 144 codes of group discussion (GD) (40.11%), 141 codes of programming operation (PO) (39.28%), 62 codes of asking assistance from the instructor (AA) (17.27%), and 12 codes of group conflict (GC) (3.34%). When the group discussed collaborative tasks (GD), the instructor used metacognitive scaffolding (MS) (Freq. = 46) most frequently, followed by the medium-control (CS-M) (Freq. = 37) and low-control (CS-L) (Freq. = 36) cognitive scaffoldings. When the group was in a collaborative programming state (PO), the instructor used metacognitive scaffolding (MS) (Freq. = 56) most frequently, followed by low-control cognitive scaffolding (CS-L) (Freq. = 36). When the group asked assistance from the instructor (AA), the instructor used the low-control cognitive scaffolding (CS-L) (Freq. = 23) most frequently, followed by metacognitive scaffolding (MS) (Freq. = 14). When the groups had conflicts (GC), the instructor used social scaffolding (SS) (Freq. = 6), followed by metacognitive (MS) (Freq. = 4) and low-control cognitive (CS-L) (Freq. = 2) scaffoldings.

The Sankey diagram of transitional frequencies

Full size image

RQ2: What were the immediate effects of instructor scaffoldings on collaborative programming?

The immediate effects of instructor scaffoldings were examined from the perspectives of statistical significance, social interaction, and overall structure. First, from the statistical perspective, statistical analysis of the Shapiro–Wilk normality test, ANOVA and ANCOVA were conducted to examine the differences among five instructor scaffoldings (see Table 4). The normality of the distributions was first tested using the Shapiro–Wilk test and the result indicated that data were normally distributed (p > 0.05). Next, ANOVA results showed statistically significant differences among five instructor scaffoldings on all seven codes of immediate uptakes from groups, namely keeping silent to scaffolding (KPS) (F = 5.12, p < 0.001), ignoring scaffolding (IGN) (F = 6.28, p < 0.001), refusing scaffolding (REF) (F = 4.43, p < 0.001), repeating or copying scaffolding (REC) (F = 28.47, p < 0.001), asking questions about scaffolding (ASQ) (F = 2.62, p < 0.05), responding to scaffolding (RES) (F = 19.87, p < 0.001), and understanding or applying scaffolding (UAS) (F = 6.99, p < 0.001). The ANCOVA results showed that after taking collaborative group state as a covariate, there were statistically significant differences of five instructor scaffoldings on seven codes, namely KPS (F = 5.15, p < 0.001), IGN (F = 6.40, p < 0.001), REF (F = 4.45, p < 0.001), REC (F = 28.44, p < 0.001), ASQ (F = 2.64, p < 0.05), RES (F = 19.98, p < 0.001), and UAS (F = 7.02, p < 0.001).

After examining the average frequency of groups’ immediate uptakes, it was found that, the groups had the highest frequency of RES (mean = 0.54, SD = 2.03) and UAS (mean = 0.15, SD = 0.88), two codes that reflected the high level of cognitive engagement, after the scaffoldings of CS-L and CS-M. Groups had the highest frequency of REC (mean = 0.69, SD = 1.86) and ASQ (CS-L: mean = 0.12, SD = 0.56; CS-M: mean = 0.12, SD = 0.87), two codes that reflected the medium level of cognitive engagement, after the scaffoldings of CS-L and CS-M. Groups had the highest frequency of KPS (mean = 0.40, SD = 0.68), IGN (mean = 0.27, SD = 0.38) and REF (mean = 0.20, SD = 2.04), Three codes that reflected the low level of cognitive engagement, after the SS scaffolding. In summary, groups showed a medium or high level of cognitive engagement after the scaffoldings of CS-L and CS-M, and a low level of cognitive engagement after the SS scaffolding.

Second, from the social interaction perspective, groups had different levels of the instructor–student interaction and student–student interaction after the instructor provided five scaffoldings (see Fig. 6). On average, groups had the highest level of overall instructor–student interaction (mean = 49.44, SD = 17.33) after the instructor provided the low-control cognitive scaffolding (CS-L) (e.g., Group 2: Freq. = 82; Group 1: Freq. = 69; Group 4: Freq. = 57). Groups had the highest level of overall student–student interaction (mean = 23.33, SD = 10.22) after the instructor provided the medium-control cognitive scaffolding (CS-M) (e.g., Group 1: Freq. = 38; Group 6: Freq. = 37; Group 2: Freq. = 29). In addition, groups’ overall instructor–student interaction ranked at the second place after the instructor provided the medium-control cognitive scaffolding (CS-M) (mean = 43.89, SD = 25.13). The groups had a medium level of overall instructor–student interaction (mean = 42.44, SD = 12.07) and overall student–student interaction (mean = 20.78, SD = 11.36) after the instructor provided metacognitive scaffolding (MS). And groups had a relatively low level of overall instructor–student interaction (mean = 39.11, SD = 18.07) and overall student–student interaction (mean = 11.89, SD = 8.23) after the instructor provided high-control cognitive scaffolding (CS-H). Finally, groups had the lowest level of overall instructor–student interaction (mean = 18.56, SD = 10.88) as well as the lowest level of overall student–student interaction (mean = 6.89, SD = 5.80) after the instructor provided social scaffolding (SS). In summary, groups had the highest level of the instructor–student interaction after CS-L, and the lowest level of instructor–student interaction after SS; moreover, groups had the highest level of student–student interaction after CS-M, and the lowest level of student–student interaction after SS.

The social networks of the five instructor scaffoldings. Node sizes of instructor represented the frequencies of instructor scaffolding; the link widths between two student nodes represented the interaction frequencies

Full size image

Third, from the overall structure perspective, ENA plots of immediate uptake codes were used to demonstrate the structures after five instructor scaffoldings (see Fig. 7). Groups illustrated strong connections of KPS-REC (coefficient = 0.18) and REC-RES (coefficient = 0.16) after medium-control cognitive scaffolding (CS-M). Groups illustrated strong connections of KPS-REC (coefficient = 0.19) and REC-RES (coefficient = 0.19) after high-control cognitive scaffolding (CS-H). Therefore, similar connections occurred after the instructor’s cognitive scaffoldings of CS-M and CS-H, namely the strong connections between silence (KPS) and repeating/copying (REC) codes and between repeating/copying (REC) and responding (RES) codes. The results indicated that after the instructor provided the medium- and high-control cognitive scaffoldings, groups tended to keep silent and repeat the scaffolding content, and then simply respond to the scaffolding content. Compared with the CS-M and CS-H scaffoldings, groups had different connected structures after the CS-L scaffolding. Groups illustrated strong connections between the silence (KPS) and responding (RES) codes (coefficient = 0.23) and asking questions (ASQ) and responding (RES) codes (coefficient = 0.20) after low-control cognitive scaffolding (CS-L). That is, after the instructor provided the low-control cognitive scaffolding, the groups tended to respond to the scaffolding content after keeping silent or asking questions. In addition, groups illustrated a strong connection between the silence (KPS) and responding (RES) codes (coefficient = 0.18) after instructor’s social scaffolding (SS). Finally, after the MS scaffolding, groups had strong connections between repeating/copying (REC) and responding (RES) codes (coefficient = 0.29) and between silence (KPS) and repeating/copying (REC) codes (coefficient = 0.18). In summary, groups’ overall structures contained the highest frequent connections that involved higher level of cognitive engagement codes (e.g., ASQ, RES) after the MS and CS-L scaffoldings, but had the lowest frequent connections that involved higher level of cognitive engagement after the SS scaffolding.

The ENA plots for each instructor scaffolding

Full size image

RQ 3: What were the delayed effects of instructor scaffoldings on collaborative programming?

To understand the groups’ delayed use of the instructor scaffoldings, we finally conducted a qualitative micro-level analysis supported with temporal visualizations based on groups’ in-class behaviors, computer operations and communication discourses. The results showed that there were three types of instructor scaffoldings that were understood or used by groups in subsequent collaboration, including metacognitive scaffolding (MS) (Freq. = 2), medium-control cognitive scaffolding (CS-M) (Freq. = 4) and high-control cognitive scaffolding (CS-H) (Freq. = 5). Three exemplary fragments were selected here, namely Group 8 completed the color recognition task with the instructor’s MS scaffolding, Group 7 completed the text recognition task with the instructor’s CS-M scaffolding, and Group 6 completed the bank queue system with the instructor’s CS-H scaffolding (see Tables 5, 6, 7, Figs. 8, 9, 10). In Group 8, the instructor prompted the group to advance the collaborative task through exploratory activities, and in subsequent collaborative learning (about eight minutes after the instructor provided the scaffolding), the group actively engaged in exploratory activities on the task without external assistance. Specifically, S3 was the most active student after the instructor provided the metacognitive scaffolding (MS). After a quick response, she accepted and followed instructor's advice and conducted an exploratory programming (see Table 5: line 2, 4, 6, 8). In the subsequent learning (i.e., the delayed use of scaffolding), S3 initially showed silence to the new collaborative task and the suggestion presented by peers (see Table 5: line 11, 12), and then S3 verbally responded to the peer’s opinions and programming operations (see Table 5: line 14, 16). At the beginning, S2 stared at the computer screen and kept silent to the instructor’s scaffolding (see Table 5: line 3, 5, 7, 9). Then, he was the first to understand the instructor's metacognitive scaffolding and successfully programmed to complete the task (see Table 5: line 10, 13, 15). S1 consistently kept silent after the scaffolding and looked at peer’s computer operations in the subsequent learning (see Table 5: line 3, 5, 7, 9, 11, 12, 14, 16).

The temporal transition after the MS scaffolding

Full size image

The temporal transition after the CS-M scaffolding

Full size image

The temporal transition after the CS-H scaffolding

Full size image

In Group 7, the instructor prompted the group to initialize the programming before text recognition, and in subsequent collaborative learning (about five minutes after the instructor provided the scaffolding), the group initialized the programming after many debugging attempts without external assistance and finally succeeded in recognizing the idiom text. S1 was the most active student after the instructor provided the medium-control cognitive scaffolding (CS-M). He was the first to repeat the guidance from the instructor’s scaffolding, which pointed out the steps for color recognition (see Table 6: line 4, 6). In the subsequent learning (i.e., the delayed use of scaffolding), S1 was also the first student to understand the instructor’s scaffolding and solved problems with programming (see Table 6: line 4, 6, 8, 12). S2 ignored the scaffolding at the beginning and looked at the peer’s computer operations with silences (see Table 6: line 2, 3, 5, 7). In the subsequent learning, he nodded and verbally showed agreement to the peer’s opinion and silently looked at the computer screen (see Table 6: line 9, 11, 13). At the end, he brought up his understanding and pointed out the parameters that the group needed to modify (see Table 6: line 14). S3 consistently kept silent and looked at peer’s programming operations after the instructor offered medium-control cognitive scaffolding and in the subsequent learning (see Table 5: line 3, 5, 7, 9, 11, 12, 14, 16).

In Group 6, the instructor helped the group to set the parameters and complete the programming, and in subsequent collaborative learning (about five minutes after the instructor provided the scaffolding), without the instructor’s assistance, the group successfully set the control parameters after several times of debugging. S1, S2 and S3 all kept silent and looked at the instructor’s programming operations after instructor offered the high-control cognitive scaffolding (CS-H) about the parameter modification (see Table 7: line 2, 3, 4, 5). In the subsequent learning (i.e., the delayed use of scaffolding), S2 was the first to verbally show his understandings and use scaffolding to solve the problem through programming (see Table 7: line 6, 8). Then he nodded his head to show agreement to peer’s opinions (see Table 7: line 11, 14). In the subsequent learning, S3 nodded her head to show agreement to peer’s opinion and then verbally expressed her understanding of the scaffolding (see Table 7: line 7, 10). In the subsequent learning, S1 nodded, silently looked at peer’s programming operations and verbally responded to peer’s opinion (see Table 7: line 7, 9, 10, 12).

Addressing research questions

This research utilized the multimodal learning analytics to analyze small groups’ collaborative programming process, particularly examining the immediate and delayed effects of instructor scaffoldings on groups’ collaborative programming. The results indicated that the instructor provided five types of scaffoldings from the social, cognitive (the superficial, medium, and high level), and metacognitive dimensions. Groups had seven types of responses (i.e., immediate uptakes and delayed uses) to these scaffoldings, ranging from the low to medium and high level of cognitive engagement. Five instructor scaffoldings had statistically significant differences on all seven codes of groups’ responses, whether taking collaborative group state as a covariate or not. Groups in general had a high level of cognitive engagement after the instructor provided the low- and medium-control cognitive scaffoldings, while groups had a low level of cognitive engagement after the social scaffolding. Consistent with previous research (Barron, 2003; Maksic & Josic, 2021; van de Pol et al., 2019), the cognitive scaffoldings with low- and medium-level control were proved to be the most beneficial ones for improving students' cognitive engagement in collaborative programming.

Regarding the second research question, five types of instructor scaffoldings had different effects on the groups’ social interactions and cognitive structures. Regarding the social interaction, groups in general had the highest frequency of instructor–student interaction and student–student interaction after the low- and medium-control cognitive scaffoldings. On the contrary, after the social scaffolding, groups had the lowest frequency of instructor–student interaction and student–student interaction. Comparing to previous research results (e.g., Maksic & Josic, 2021), our results showed that compared to the social scaffolding, cognitive scaffolding not only had positive effects on improving group cognition and problem-solving, but also increased students’ social interaction and participation. Regarding the overall cognitive structure, groups in general had the most frequent connections with higher level of cognitive engagement after the metacognitive and low-control cognitive scaffoldings. On the contrary, after the social scaffolding, groups had the least connections regarding higher level of cognitive codes. Consistent with previous research results (Wittwer & Renkl, 2008; Wittwer et al., 2010), the instructor’s regulation of groups’ learning and low-level cognitive supports were beneficial for students to achieve a high-level engagement. Overall, the low/medium-control cognitive and metacognitive scaffoldings can improve students’ social and cognitive engagement during the period of immediate uptakes; in contrast, the social scaffolding had limited positive effect in prompting group engagement.

Third, after the scaffolding faded, groups used the content from the high-control cognitive scaffolding frequently to solve problems in a delayed way, but groups did not use the instructor’s scaffolding content from the social and low-control cognitive scaffoldings. Consistent with previous research results (Bulu & Pedersen, 2010; Tawfik et al., 2018), direct supports from the instructors’ definite, complete information (such as higher control) had more delayed effect than the low-controlled scaffoldings. That is, students tended to use the contents, operations, or answers provided by the instructor’s high level of cognitive supports in order to solve the programming problems after the scaffoldings faded (Margulieux & Catrambone, 2021; Tawfik et al., 2018). In addition, inconsistent with the findings of previous research (van de Pol et al., 2019), the groups’ ignorance of the scaffoldings during the period of immediate uptakes did not negatively influence their delayed use of scaffoldings in subsequent collaborative learning. For example, there were four times groups showed a low-level cognitive engagement (such as kept silent, ignored or refused the scaffolding) when the instructor provided the scaffolding, but later they used the scaffolding content in a delayed way to solve problems. Overall, the instructor’s high-control cognitive scaffolding facilitated the use of content or hint within small groups in a delayed way to solve problems and complete collaborative tasks.

Pedagogical implications

The empirical research results provide three pedagogical implications to help instructors design and utilize scaffoldings to improve small groups’ collaborative leaning. First, instructors should use the low- and medium-level of cognitive scaffoldings, along with metacognitive scaffoldings to initiate open-ended questions or provide hints to groups in order to improve collaborative programming quality. Consistent with previous research results (van de Pol et al., 2019; Wittwer et al., 2010), during collaborative programming, novice learners usually encounter complex tasks that cause high cognitive loads (Kunkle & Allen, 2016; Margulieux & Catrambone, 2021; Yeomans et al., 2019); in this case, the low- and medium-control cognitive scaffoldings can help students actively think about open-ended questions, process new information, and elaborate and reflect on ideas. With the cognitive supports, it is easier for groups to remove barriers caused by students’ low cognitive levels and integrate new information and knowledge into their self-perception system (Wittwer & Renkl, 2008; Wittwer et al., 2010). In addition, instructors should make a conscious effort to provide metacognitive scaffolding, since the metacognitive support from the instructors can monitor and guide groups’ learning pathways. This assistance can help students pay attention to knowledge-based programming tasks, and the cognitive support prompts students to think further on the existing learning materials or programming problems (Mattanah et al., 2005; van de Pol et al., 2019; Wittwer et al., 2010).

Second, strengthening the delayed effect of scaffoldings is conducive for groups to solve problems and construct knowledge by themselves in subsequent collaborative learning. The results showed that the high-control cognitive scaffolding helped groups understand and apply content or operation from the instructor and then solve collaborative problems in subsequent collaborative learning. Particularly, when students face problems of high difficult level, they need the instructor to provide direct assistance or offer answer to fill cognitive gaps in order to move on for problem-solving (Mattanah et al., 2005; Nedić et al., 2015). Consistent with previous research results (Bloome et al., 2009; Brown & Renshaw, 2009; Wiig et al., 2017), in some cases, although some student groups did not accept the scaffoldings immediately, but they understood and applied the content during the later stage and used it to solve programming problems after the scaffolding faded. Therefore, the instructors should consider the group collaborative states and characteristics and provide a high-control cognitive scaffolding when they face difficult problems, so as to facilitate groups’ understanding and application of knowledge and complete collaborative learning tasks.

Finally, social scaffolding needs to be carefully considered to enhance immediate and delayed effects on group collaboration. The results showed that among five instructor scaffoldings, social scaffolding had the weakest positive effect on collaborative group learning, either on social interaction or cognitive engagement, which was different from previous research results. One of the reasons is that when groups faced programming problems beyond their cognitive levels, it might be difficult for them to communicate their own opinions with peers, and only when the cognitive barriers were cleared, they would be more motivated to share, discuss and reflect on ideas or problems (Wittwer & Renkl, 2008; Wittwer et al., 2010). Previous research indicated that social supports from the instructor or peers were conducive to build a supportive, safe atmosphere, which may encourage students’ idea expressions and active participation in collaborative activities (Maksic & Josic, 2021; Ouyang et al., 2020). Taken together, we conclude that instructors should use social scaffolding carefully when students have a certain cognitive basis for the learning topics, in order to improve the scaffolding effect in collaborative learning.

Overall, the results suggest that instructors’ support and scaffoldings are needed, particularly when learners lack sufficient understandings of content and hands-on experiences. The instructors should grasp and understand groups’ difficulty levels as well as their cognitive load before providing scaffoldings, and then tailor the scaffolding to the groups’ situation in order to provide appropriate supports (Meloth & Deering, 1999; Ouyang et al., 2021; Webb, 2009). To avoid premature fading of scaffolding effect, the instructor should check the group’s learning state and make appropriate adjustments before leaving the group. Moreover, in eastern education system, such as China, instructors should also consider socio-cultural factors (Zhang, 2007) when providing the scaffoldings, in order to avoid students seeing the instructor as an authoritative source of knowledge and director of the learning process (Zhang, 2013). In summary, from the pedagogical perspective, instructors should consider scaffolding types, group states and characteristics, as well as the timing of scaffolding to better design and facilitate collaborative programming (Bliss et al., 1996; Ouyang et al., 2021; van de Pol et al., 2015).

Analytical implications

This research uses MMLA and multiple analytical methods to examine the immediate and delayed effects of instructor scaffoldings on the groups’ collaborative programming. MMLA is a critical and significant breakthrough for analyzing collaborative programming processes, and previous studies have called for using MMLA in collaborative programming (Twiner et al., 2021). The CSCL community has recently called for collaborative learning analytics, the intersection of collaborative learning and learning analytics, to focus on analytic and implementation of learning analytics on groups’ collaboration (Schneider et al., 2021). Because instructors and students communicate ideas and construct meanings through behaviors, discourses, gestures, and computer operations, it is necessary to collect and merge data from multiple sources in order to reflect the scattered activities of participants and alleviate the isolation of information perception caused by a single data source (Bezemer, 2008; Chatti et al., 2017). The multiple analytical methods are conducive to the analytics of collaborative learning from social, cognitive, behavioral, and micro-level dimensions, such that instructors can better understand and reflect on the process of collaborative learning, and use scaffoldings more skillfully to support collaborative learning (Maksic & Josic, 2021). Echoing this trend, this research collects and analyzes learners’ in-class behaviors, computer operations and their communicative discourses synchronously, so as to conduct an integrated microanalysis of the moment-to-moment details of how learners coordinate their communications, behaviors, and movements during collaborative programming. Future studies can further synchronize audio discourse data, video recording data, facial expressions and eye tracking movements to better reveal the collaborative programming learning patterns (e.g., Chevalier et al., 2020; Sun & Hsu, 2019; Zatarain Cabada et al., 2018). Overall, an integration of MMLA and collaborative analytics can provide feedback on similarities and differences of CSCL results at the group level to improve the understanding of groups’ collaborative work. An examination of instructor scaffoldings on collaborative learning can achieve a complete closed loop of analytics, feedback, and iterative application with a goal to develop collaborative learning quality and process.

Collaborative programming is encouraged to be integrated in STEM education to transform education from the instructor-directed lecturing to the learner-centered learning. Using MMLA methods, this research provided a deep understanding of the immediate and delayed effects of instructor scaffoldings on small groups’ collaborative programming in K-12 STEM education from a process-oriented perspective. The results indicate that various instructor scaffoldings have been used to promote groups’ social and cognitive engagement. In addition, instructor scaffoldings have delayed effects on promoting collaborative programming qualities. Based on these findings, this research offers implications for pedagogical support and learning analytics of collaborative programming. There are three major limitations of this research. First, the research data are originated from the natural classroom environment, which lacks the deliberate design and control of the instructor scaffoldings. Future research should consider a quasi-experimental research design to examine and verify the immediate and delayed effects of different instructor scaffoldings. Second, this research is conducted with a small sample size in one class taught by one instructor; future research should apply scaffoldings to multiple courses or disciplines, expand the sample size and balance student gender, in order to further verify the results and implications. Third, future work should expand the multimodal data collections to include psychological and physical data, which can complement discourse and behavioral data, and better reveal the collaborative programming learning patterns. Overall, this research takes a step forward to conduct a holistic analysis of instructor scaffoldings as well as their effects on collaborative processes in computer programming education in China’s K-12 education. Given the complexity of collaborative learning as well as the effects of instructor scaffoldings on collaborative learning, it is highly suggested that instructors should integrate scaffoldings to facilitate computer programming education and relevant research should combine multimodal data sources and multiple learning analytical methods to reveal details of the process of collaboration.

The data are available upon request from the corresponding author.

We appreciate the participation and support from the students, teachers and parents in this research.

This work is supported by NSFC funding (62177041), Zhejiang Province educational science and planning research project (2022SCG256), and Zhejiang University graduate education research project (20220310).

Authors and Affiliations

1. College of Education, Zhejiang University, #866, Yuhangtang Rd., Hangzhou, 310058, Zhejiang, China

   Fan Ouyang, Xinyu Dai & Si Chen

Contributions

FO designed the research, designed and facilitated data analysis and revised the manuscript; XD analyzed the data and wrote the first draft of the manuscript under FO’s supervision; and SC collected data and contributed to the preliminary data analysis. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Fan Ouyang.

Competing interests

The authors declare that they have no competing interests.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions