Leveraging Conceptual Change regarding Artificial Intelligence in Computer Science Education

The problem described by Duit [17] of guiding students’ (pre-) conceptions to (correct) scientific conceptions is referred to as conceptual change [38, 49]. Various theories on this have existed since the 1960s [49]. Students’ conceptual change takes place according to the approach described as the “classical” [49] approach by Posner et al. [38] at different levels. A distinction is made between two types: assimilation and accommodation [18, 38]. These represent patterns of learning similar to the acquisition of scientific knowledge. Assimilation is the phase of conceptual change in which known concepts are used to explain new phenomena. Accomodation is the stage at which the known concepts are no longer sufficient or inadequate to explain the new phenomenon and the learner has to replace or restructure these concepts [38]. According to Posner et al. [38], the latter in particular is the approach that needs to be considered when it comes to conceptual change, which is seen critical by many perspectives [49]. Accordingly, a student’s conceptual change can be triggered by cognitive conflicts based on dissatisfaction with their own conception compared to scientific reality/reasoning [18, 38, 49]. This theory is based on the fact that learners are presented with a substitute conception in class, whereby 1) there is dissatisfaction with existing conceptions, 2) the new conception is intelligible, 3) appears plausible to the learner and 4) suggests the possibility of a fruitful program [49, p. 11]. Vosniadou [49] discusses that this is a slow process, during which fragmentation and misconceptions can be created and which requires fundamental changes in student’s ontological, and epistemological commitments and in their representations. The framework theory [49] approach explains “the formation of misconceptions and of fragmentation as the result of the application of constructive types of mechanisms on incompatible knowledge structures [...], the product can either be an internally inconsistent –- fragmented –- conception or a misconception” [49, p. 15].

Student conceptions and the associated conceptual changes in computer science education are typically examined within the framework of educational reconstruction [13], drawing from phenomenological centering and its relation to technological clarification and social demands, in a similar way to how it is done in biology education [22]. Students’ conceptions – as the basis of the conceptual change theory – are therefore an essential component of the educational reconstruction of computer science and always interact with the selection of phenomena, the clarification of subject content and concepts, and teachers’ conceptions in equally important ways [13].

A relatively new field of research in computer science education is the educational reconstruction [13] of the subject area of artificial intelligence (AI). Kreinsen and Schulz [29, p. 1] argue that this is the case: “Due to technological developments and socially relevant issues in the last decade, the field of artificial intelligence offers a wide range of new possibilities to integrate it in computer science lessons”.

Kreinsen and Schulz [29] examined students’ conceptions using semi-structured interviews. The results show that the functionality of AI is described by the students as the storage and retrieval of pre-programmed data. Cookies in web browsers or voice assistants in cell phones were mentioned as familiar AI systems from everyday life. Furthermore, the students surveyed equated AI with robots, similar to the findings of Ellis et al. [19]. They also described AI as the brain of a robot. In addition, students assume that AI systems are pre-programmed and cannot learn independently [29].

Mertala, Fagerlund and Calderon [31] asked primary school students in an online survey and collected conceptions 1) of the kind of technology AI is, 2) of where AI is, 3) of why AI is used. Some results draw: AI is like a human, only trapped in a machine; there is no real-time input from humans; machines are programmed to solve a task on their own; AI only does what it is told; AI is capable of picking up information using various sensors, both auditory and visual [31].

Similar findings were uncovered analyzing primary school pupils’ conceptions of AI by Ottenbreit-Leftwich et al. [34]. In interviews, the students mainly associated AI with programming and were convinced that every single AI was programmed by humans. Thus, they argued, AI systems can only implement exactly what they were programmed to do by a human. Several other studies [26, 4, for example] exist on student conceptions of AI and are currently being researched.

Nevertheless, hardly any approaches in computer science education are known to date that go one step further towards a conceptual change. As already suggested by Schulz and Pinkwart [42], we were looking for a methodological approach in the long successfully established education of STEM subjects to clarify a question in computer science education. One methodological approach to achieve conceptual change is the usage of conceptual change texts [12].

Initial research results indicate that ordinary/traditional text structures commonly used in education may not be adequately effective for improving students’ understanding of subject-specific concepts in both the short and long term [24]. In studies examining effective teaching strategies within conceptual change frameworks, conceptual change texts (CCT) have emerged as a particularly beneficial tool, particularly in secondary science disciplines [12, 35, 36, 53].

According to Grospietsch and Mayer [23], a CCT consists of the following parts, based on the phases of conceptual change: 1.) Make pre-concepts explicit: The initial stage of a CCT involves students expressing their stance on a statement (student conception) through free text, with the objective of recalling their own conceptions. 2.) Create cognitive conflicts: The subsequent section comprises the textual content itself, which takes up the students’ conceptions, identifies and contrast disparities from the subject-specific concept, and offers impulses to challenge and refute the students’ initial conceptions. “The text itself had the function to first introduce the relevant misconception and then ask whether they could really be true” [21, p. 8]. As a result, there could be a cognitive conflict between the students’ initial conception and the scientifically correct conception. The linguistic design of the texts is crucial here. The text should address the students directly [37], be comprehensible (few technical terms, vivid examples, etc.) and plausible (convincing arguments supported by sources and further literature) as well as fruitful (reference to examples from the students’ lifeworld and stimulate critical questions) [38]. 3.) Offer a substitute conception: During the third part, students recall their conceptions using the statement provided in the first part, reflecting on them through free-text responses in light of the differences outlined in the text. Ultimately, the initial conception could either be expanded or discarded. Figure 1 illustrates the structure of conceptual change texts (the three parts are here structured in three tasks for the students).

In their quasi-experimental study, Çakir, Geban, and Yürük [8] explored the effectiveness of CCTs on students’ understanding of the cellular respiration concept in biology. The experimental group, consisting of 44 eleventh-grade students, received CCT-centered biology instruction, while the control group of 40 students at the same grade level was taught using traditional texts. Both groups underwent a pre-post test assessment, which included a Cellular Respiration Concepts Test (CRCT) and an Attitude Scale towards Biology as a School Subject (ASTB). Derived from educational literature on student conceptions of cellular respiration and teacher interviews, the CRCT contains items that students must select in a multiple-choice format based on whether they agree or disagree with them. The ASTB inquires about attitudes towards biology as a school subject by asking students to classify them on a 5-point Likert scale. The t-test carried out on the results of the pre-test showed no significant differences between the experimental and control groups. Following the post-test, a significant difference was observed between the experimental group and the control group in terms of the concepts (CRCT) related to cellular respiration, as indicated by a t-test. Thus, the research highlights the enhanced efficacy of CCTs compared to traditional texts in teaching biology, particularly regarding the topic of cellular respiration [8].

Beerenwinkel, Parchmann, and Gräsel [3] similarly explored the effectiveness of CCTs in the field of chemistry concerning students’ understanding of prevalent conceptions related to the particle model of matter. This research also adopts a quasi-experimental design with a pre-post test to assess students’ common conceptions. The study involved a sample of students, with one-third in seventh grade and two-thirds in eighth grade, who were at a comparable level and undergoing their initial year of chemistry education. Results indicated a significantly stronger impact of CCTs over traditional texts. Moreover, there was no indication that students with varying levels of prior knowledge benefited more or less from either teaching approach (with or without CCT).

These two studies are examples of the extent to which we transfer the approach used there to our study. Some other studies [52, 53] employing a comparable research design have demonstrated the effectiveness of CCTs. This study aims to transfer the methodological approach from science education to computer science education due to the absence of conceptual change approaches in this field. The following research question is the focus of this study:

The study was carried out at a German secondary school grade 10 and 11. A total of 76 students took part in the study, attending four different computer science courses. Out of these, the questionnaires of 69 students, 22 female and 47 male, were filed out completely and processed. Half of the four courses were from grade 10 and the other half from grade 11. All of the participating courses were elective, so the students voluntarily chose to participate in computer science classes. All participants in the study have received computer science lessons since grade 8, which take place once a week in all age groups. The teaching staff explained that the subject of AI had not yet been taught, but that ChatGPT was known.

The methodology follows a pre- and post-test design. In order to analyze the enhancement of conceptual understanding based on the agreement to a certain conception, data was collected by means of a questionnaire.

This pre-test was piloted and revised prior to the main intervention. On the one hand, the questionnaire was intended to record the student’s prior knowledge of AI and to identify a concept that had particularly strong approval. The CCT was then created based on this agreement. The designed pre- and post-test contains a four-point Likert scale, where 0 - strongly disagree, 1 - disagree, 2 - agree, 3 - strongly agree. The various multiple choice items are created from already existing research findings on students conceptions of AI. To analyze the effectiveness of the CCT, the participating students were assigned to either an experimental or control group.

First, all students got the pre-test one week before the intervention. One week after the intervention the post-test occurred. In total, the study lasted for three weeks. The experimental group (N = 34 students) received a teaching intervention focusing on the CCT, while the control group (N = 35 students) was taught without the CCT. The lessons lasted 90 minutes in both groups. The following table shows the different, yet comparable lessons (see Table 1).

In respect to the research question a CCT with the following structure [23] was developed:

Task 1: First, the students got the task to comment on the statement “Every output of artificial intelligence is pre-programmed” in a written form.

Task 2: Afterwards they read the following text: You’ve probably tried out the new chatbot ChatGPT recently and noticed the following: No matter what question you asked the chatbot, it had a suitable answer to almost every question. Whether your input is a simple question, such as “What is a byte?”, or something more complex, such as “Write me a book summary of the book ’Faust’ by Goethe”. You are probably also familiar with some voice assistants, such as Siri from Apple or Alexa from Amazon. If you ask these voice assistants a question, they will also answer appropriately. Even if you make a search query on Google, the most suitable websites will be suggested to you based on your query. This means that no matter which artificial intelligence (AI for short) system you and your classmates would use, the developers of the AI system would have to have pre-programmed their own suitable output for every potential question or query. But this is not correct from a scientific perspective! For the developers of an AI system, it is even impossible! An AI system, such as ChatGPT, generates its answers based on its training data set and the algorithms on which the system is based. [...].

This text continues with the content clarification and examples to create a cognitive conflict and to deliver a scientifically correct concept.

Task 3: After reading the CCT the students were requested to read their statement from Task 1 and comment again on the same statement like before reading the text. Thus they revised their statement to broaden their conception.

The results from the pre- and post-test were compared using the statistical software SPSS. Firstly, the mean values were compared with each other and significance test was carried out to assess the development of student’s conceptions with regard to the research question. In this case, the significance level was set at 5 %. If the significance is below 5 %, the test result is considered statistically significant. To carry out the significance test, a two-fold unpaired t-test was performed, which measures the difference in mean values between two groups in independent samples (in case of normal distribution), represented here by the experimental and control group. In this study the calculated mean values represent the student agreement to a common naïve conception. The greater the difference between the mean value of the two groups and the smaller the standard error, the less likely it is that the difference between the two groups is a coincidence.