Testing, Socializing, Exploring: Characterizing Middle Schoolers’ Approaches to and Conceptions of ChatGPT

Prior studies have examined user interactions with CAs in various contexts. For example, Sube et al. [83] studied factory workers’ interactions with a specialized AI system and delineated 13 patterns of interaction within three core themes: cognitive, emotional, and social. Wang et al. [89] focused on graduate students’ perceptions of a virtual teaching assistant across five features such as verbosity and readability. Importantly, this prior research has substantiated the feasibility of inferring user perceptions such as emotions [80], personality traits [63], conversational breakdowns [55], ascribed humanness [71], and politeness [13] from linguistic cues in interactions with CAs. However, existing literature often targets specific settings (e.g. customer service [48], education [89], or gaming [33]).

Though research in CAs has advanced their capabilities dramatically in recent years [10, 33, 70], the launch of generative AI made flexible and comprehensive interactions across a variety of use-cases possible, requiring new research into user experiences in using CAs [79]. Specifically, ChatGPT “can engage in fluent, multi-turn conversations out-of-the-box, substantially lowering the barriers to creating passable conversational user experiences” [94]. To understand users’ perception of LLMs, recent studies, such as Korkmaz et al. [50], have approached ChatGPT analysis through sentiment analysis of social media, finding generally positive user attitudes. Mogavi et al. [64] analyzed social media data to understand user perspectives of AI in various education sectors, and identified productivity, efficiency, and ethics as key discussion points. Skjuve et al. [79] surveyed users about their experiences with ChatGPT and suggest that ChatGPT was mainly understood from three perspectives: its (1) technical functionality, (2) uses and purpose, and (3) interaction capabilities. Other research looked into users’ prompting behaviors and barriers with large language models for creative writing [33] and collaborative programming [28, 95]. Users’ interaction with LLMs in creative writing highlighted the need for a deeper look into users’ mental models of such tools as participants exhibited varied notions of the AI system [33].

Our work distinguishes itself in two major ways: First, our study design allows for participants to guide their free exploration of ChatGPT, unbounded by specific tasks, capturing more authentic experiences and offering a broader understanding of user conceptions and approaches. Second, our analysis takes into account the role of participants’ group dialogue in addition to other contextual evidence. This holistic approach echoes Rapp et al.’s [71] view that perceptions of chatbots vary based on multiple factors including context, participant objectives and cues that the chatbot exhibits.

The field of child-computer interaction has investigated how children conceptualize AI. One strand of research has noted the age-dependent differences in how children understand and interact with AI technologies [19, 65]. For example, Nguyen [65] reported variances between different teenager groups in their perceptions of CAs such as more or less competent, trustworthy, sociable, and knowledgeable. Druga et al. [15] contrasted the perceptions of children (4-10 years) and parents in their interaction with smart devices through a maze-solving activity. Their findings indicated that children focus more on sensory and social-emotional aspects, whereas parents more often reference cognitive abilities. Druga et al. [19] argued that playful and interactive ways of probing children’s understanding of AI could be key to advancing our knowledge of their “cognitive and conceptual development.” They also call on future research to investigate “tasks without a clear goal, such as social interaction tasks” and to examine “the views of teenagers and young adults.”

Recent studies have also touched on the impact of social and contextual factors on children’s views and interactions with intelligent technologies [31]. For example, Rubegni et al. [76] examined the role of social interactions and settings in shaping middle school-aged children’s hopes and fears about social robots. Druga et al. [17] extended this line of inquiry by exploring the imagination and expectations of children across four countries (USA, Germany, Denmark, and Sweden), noting the influence of socio-economic and cultural backgrounds on their understanding of AI. Interestingly, children from lower socio-economic statuses displayed stronger collaborative abilities, while those from higher socio-economic backgrounds exhibited a deeper understanding of AI concepts, highlighting a need for AI literacy interventions catered towards children from lower socio-economic backgrounds.

Another avenue of research has concentrated on design characteristics that could influence a young user’s interaction with AI [18, 19, 88, 93]. Druga et al. [18] and Woodward et al. [93] proposed design recommendations focusing on elements like voice and prosody, and error detection and correction techniques, respectively. In summary, while existing studies offer valuable insights into younger children’s perceptions and conceptions of AI, there remains an unaddressed need for focused research on teenagers’ conceptions of these technologies [25, 65] and of CAs, more specifically. Our work aims to contribute to this area of research by informing the design of AI literacy interventions specifically tailored to middle school-aged group’s conceptions of and approaches to AI.

While teaching AI at the K-12 level is not yet widespread, researchers argue it is important for developing future societal readiness [12]. Most research on novice AI learning has focused on classrooms [2, 36, 88, 90, 96] with initiatives like AI4K12 [87], organizations like the Center for Integrative Research in the Computing and Learning Sciences [30], and commercial efforts such as AI4All, ReadyAI and Concord Consortium [1, 72, 73].

Although initiatives are emerging, the formal integration of AI literacy into K-12 curriculum remains limited, creating a significant public education gap [12]. Long & Magerko [58] define AI literacy as “a set of competencies that enables individuals to critically evaluate AI technologies; communicate and collaborate effectively with AI; and use AI as a tool online, at home, and in the workplace.” With nascent nationwide computing standards and elective AI courses, most students lack exposure, especially those from lower-resourced schools [5]. Much existing classroom AI content focuses on technical aspects rather than ethical/social implications [2, 29, 32, 38, 45, 92, 96]. Prior work shows that real-world, social, and cultural relevance motivate historically underrepresented groups, such as girls, in CS and AI [12, 23, 43]. We define cultural-relevance as “knowledge and practices in family and community life” [52] including beliefs, ritual practices, art forms (e.g., music, movies, video games), and academic subjects (e.g., science, history, literature), as well as informal cultural practices such as language, gossip, stories, and rituals of daily life [68, 85]. Culturally-relevant approaches in education have been shown to provide a low barrier of entry to computing subjects [11, 62] and empower "students intellectually, socially, emotionally, and politically" [51]. However, few AI curricula address learners’ lived experiences or cultural backgrounds, which research shows empowers diverse engagement [44, 62, 78, 82]. Thus, while progress in formal learning is being made, informal approaches are also crucial to build inclusive public AI literacy.

To inform educational initiatives, frameworks for AI literacy are emerging [20, 58, 87]. For example, Long and Magerko’s [58] AI literacy framework synthesizes interdisciplinary research into core competencies and design considerations, providing a conceptual framework to guide future research and development of learner-centered AI technologies. The framework defines five guiding axes: 1) what is AI?; 2) what can AI do?; 3) how does AI work?; 4) how should AI be used?; and 5) How do people perceive AI? Similarly, Druga et al. [20] have explored the perceptions of AI bias among children and proposed an AI literacy framework focusing on algorithmic justice. These frameworks support the growing trend towards utilizing informal learning environments, such as museums, after-school programs and at-home resources, to expand the reach of AI education [1, 15, 16, 22, 24, 53, 57].

While prior work has identified general design recommendations and direction to support informal AI learning activities, little work specifically targets designing museum exhibits. Informal learning spaces like museums have historically been integral to public science engagement, such as supporting science knowledge, interest development, and improved interdisciplinary connections [3, 4, 37, 84]. There have only been a few museum exhibits (e.g., Robot Revolution at Chicago’s Museum of Science and Industry [67], AI: More than Human at The Barbican [7], Exploring AI: Making the Invisible Visible at Boston’s Museum of Science [66]) focused on AI so far [53, 57], likely due to the novelty and opacity of most AI systems. For example, Lee et al. [53] designed an AI-related exhibition to cultivate critical thinking competencies and found that the exhibit supported youth in relating AI to their lives. Long et al. [57] have also explored the role of museum exhibits in fostering AI literacy among family audiences and suggest design considerations specific to this context. The conceptualization of museum spaces as free-choice, constructivist learning environments, as highlighted in Long et al.’s work [57], prioritizes learner agency and embodied social interaction in designing learning interventions in museums. To allow for this active process of meaning-making, we focus on a self-guided exploration in our focus group study. Based on this self-guided, free exploration interaction, we also use a holistic approach to understanding participants’ interaction with ChatGPT as the foundation for the design of informal learning interventions in museums.

We focus on middle school-aged children (i.e., 9-14 years old) as the target demographic for our current study and design decisions as prior work has shown that children younger than six struggle with understanding AI reasoning processes as they are still developing a theory-of-mind [91] and because introducing AI at the middle school level exposes children to the topic at an age when they are beginning to make decisions about their future [86]. Since a key focus of our larger project is broadening public AI literacy, we also emphasize underrepresented learner populations in AI and CS. More specifically, we focus on engaging middle school-aged learners without a CS background, middle school-aged girls, and students of Title 1 schools (i.e., schools that receive funding from the federal government to support low-income students). As we aim to inform designs for museum visitors with varying prior knowledge of AI, we did not explicitly require or inquire about participants’ prior interactions with AI.

Following Institutional Review Board (IRB) approval, participants were recruited through our museum partner’s mailing lists: a member mailing list with approximately 20,000 contacts, and a mailing list of people who have previously participated in user studies with approximately 3,000 contacts listed. Participants, with their parents, completed a screening questionnaire identifying their age, gender, race and ethnicity, computer science background, and current school they attend, as well as their legal guardian’s name and contact information to ensure that their age ranged from 9 to 14 and that they fit at least one of the following criteria: a) identify as female, b) attend a Title 1 school, and/or, c) have not previously taken a CS course. Eligible participants were then contacted to complete consent and assent forms, and choose a focus group timeslot. In total, we recruited 24 participants (P1-P24) from 17 different families (see Table 1). Some participants had familial ties (e.g., siblings or cousins) or were friends prior to the study. The participants identified mostly as female (n=17, n=7 male, n=0 non-binary / third gender), and their ages ranged from 10 to 14 years, with a majority (n=8) being 12 years old. About half of the participants (n=11) attend a school either listed as Title 1 or as eligible for Title 1 funding and the majority of participants (n=18) had not previously taken any CS course. Participants were compensated with free admission to the museum for the day for them and their immediate family (i.e., parents, legal guardians, and siblings), free parking, and $20 in cash.

This focus group study took place in April 2023, at a museum of science in a major Northwestern city in the U.S., as the first step in a larger effort to design interactive museum exhibit for AI literacy. Overall, we conducted nine focus group sessions (FG1-FG9) (see Table 1). Each focus group lasted approximately 30 minutes and consisted of two separate, consecutive activities. In the first activity, participants were asked to name several examples of their interests (not related to AI). Though this activity may have primed students to be thinking specifically of these topics during the second activity, it is not included in the analysis presented here as it does not directly relate to children’s interactions with AI. We only describe it as it may have primed participants’ prompt topics during the second activity. The second activity required participants, in small groups of sizes ranging from two to four, to interact with ChatGPT¹ (version 3.5) for 10-15 minutes (equalling 99 minutes and 12 seconds of recording overall). We deliberately structured the second activity to have a duration of no more than 15 minutes, mirroring the typically brief duration of interactions that visitors have with museum exhibits [42]. Participants were told that ChatGPT is “a chatbot that can hold conversations just like humans, it can talk to you about anything you want, and you can ask it any questions you want.” They were encouraged to discuss their ideas for conversation and prompts out loud as a group, type them in the text box, and send the message by hitting the enter key. Following the interaction, we asked each group their thoughts about ChatGPT.

All focus groups were audio and video recorded, and their chat log with ChatGPT saved from each computer with participants’ consent. Authors 1 and 2 generated manual transcriptions for each focus group’s second activity and included prompts sent to ChatGPT in the transcriptions at the appropriate moments.

The final dataset consisted of 1) the prompts participants sent to ChatGPT during the study, 2) the responses generated by ChatGPT, and 3) transcripts of participant discussions from audio and video recordings to capture certain movements pertinent to the interaction (e.g., pointing to the computer, direction of speech).

We conducted a multifaceted qualitative analysis of the study data, as shown in Table 2, to elucidate students’ topics of interest, interaction patterns, and AI conceptions and misconceptions when engaging with ChatGPT. The first two authors met regularly to discuss emerging observations and refine the codes to capture nuances. Coding was iterative, with regular discussion among all authors to reach consensus and consolidate codes. Resolution strategies for disagreements included: revisiting definitions of codes, expanding the discussion to other authors, and examining additional data points.

To address our first research question, we categorized each prompt by its overall topic to understand the topics participants chose to explore with ChatGPT. To address our second research question, we used an inductive thematic analysis approach [9] to examine three key data dimensions simultaneously: 1) the transcripts of discussions between participants where they explicitly stated their goals such as “I would like to see how much it knows and question it” (P1) or reactions such as “It feels good to talk to someone who’s actually as smart as me” (P9), 2) the types of prompts participants sent to ChatGPT, and 3) the types of responses ChatGPT provided. We examined all three aspects of the data simultaneously to provide a more holistic and richer account of each FG’s interaction. More specifically, participants’ goal-oriented thoughts and reactions surface their motivations in interacting with ChatGPT and their intent behind each of their prompts. Participants’ prompts demonstrate how they implemented their motivations, and ChatGPT’s answers provide additional context for their reactions and subsequent prompt choices.

We identified three themes in participants’ goal-oriented thoughts and reactions: (1) testing AI, (2) socializing with AI, and (3) exploring AI-generated content (see Table 3). For prompts participants sent to ChatGPT, we categorized each prompt based on its nature and intent. Through this process, we identified seven general types of prompts from our participants: 1) anthropomorphized entity (embodied), 2) anthropomorphized entity (not embodied), 3) clarification, 4) creative, 5) fact question, 6) operationalizable opinion question, 7) speculative (see Table 4). ChatGPT’s responses were categorized into the following: (1) AI unable/refused to answer because answer does not exist, (2) correct answer, (3) answered with caveat, (4) AI explaining itself or its capabilities, (5) AI produced something, (6) incorrect answer (see Table 5).

During the analysis of participants’ dialogues and interactions, we observed patterns of AI-related conceptions and misconceptions. To further investigate these patterns, we conducted a round of targeted coding of the dialogues to extract participants’ AI-related conceptions and misconceptions. For instance, the statement “It has to search through the web and the web is not always the most reliable thing” (P17) suggests a misconception that AI searches the web to answer any question asked of it or “It’ll say I don’t have any feelings”reflects an understanding that AI does not have human-like feelings or emotions. In total, we extracted 57 excerpts exhibiting AI conceptions or misconceptions. We used Long & Magerko’s AI literacy framework [58] to deductively categorize the extracted conceptions and misconceptions into five overarching questions: "what is AI?", "what can AI do?", "how does AI work?", "how should AI be used?" and "how do people perceive AI?" Then, we inductively surfaced recurring themes of conceptions and misconceptions under each overarching question, such as attributing human qualities to ChatGPT or comparing ChatGPT to other tools.

In total, participants explored 10 broad topic categories in their prompts to ChatGPT consisting of school and academics, hobbies and interests, pop culture, technology and gaming, AI, relationship to self or others, current world events and issues, food, life and death, and other. Additionally, 8 prompts were nonsensical words or emojis, such as “som[e]thing” and were coded as gibberish. School and academic subjects were the most common, with 17 prompts on topics like history (“how much do you know about history”), math (“what is 1”), and science (“will space keep expanding”). Participants’ hobbies and interests were also prevalent, with 16 prompts on poetry (“make a poem about Chicago”), jokes (“Can you tell me a joke”), sports, and more. Pop culture was another popular topic, encompassing 15 prompts about music (“when[‘]s yeat rele[a]sing a[n] album”), celebrities (“who is drake”), and movies. Other notable topics included technology/gaming (n = 6) like Minecraft or Roblox; AI characteristics (n = 6) regarding ChatGPT’s gender, age, or preferences; relationship to self or others (n = 5) focused on body image and intelligence; current world events/issues (n = 3) such as politics, population, or pollution; and food (n = 3) on topics like cheese and milk. Less frequent topics were existential questions (n = 2) about death and the afterlife and other conversation statements without clear topics (n = 4) such as greetings.

While participants’ prompts highlight their interests, some of them also voiced additional preferences for their future interactions with CAs such as nine participants discussing their struggles with spelling words in text-based interactions (“I wish I knew how to spell more elements. I really know a lot of elements but don’t know how to spell a lot of them” (P9)), and four suggesting “mak[ing] answers short because it’s a lot to read I was just skipping” (P12). While this analysis reveals the breadth of subjects participants explored with ChatGPT, understanding the rationale behind their choice of prompt requires examining the sequence of their interaction holistically.

From the 24 participants, only three participants from FGs 3, 6, and 8 offered minimal prior familiarity with ChatGPT, stating that “Oh, ChatGPT, you can use this for essays” (P21) and "it’s blocked over school computer though, so annoying" (P15).

We examine three key dimensions of each FG’s interaction sequence including a) each focus group’s dialogue, b) their prompts to ChatGPT along with the prompt’s type, and c) ChatGPT’s corresponding answer in order to examine participants’ motivations in interacting with ChatGPT, how their motivations were implemented through their prompts, and the context for their subsequent reactions. To clarify our approach, we provide a walkthrough of FG4’s interaction sequence (see Figure 1). FG4 consisted of three participants where P9 asked the most questions, while P8 and P10 mostly spectated. As shown in Figure 1, FG4 prominently exhibited dialogue that represents socializing with AI with P9 actively comparing his own knowledge to ChatGPT’s, stating that “it feels good to talk to someone who’s actually as smart as me” and “it’s making me look like an idiot” when ChatGPT answered with more specificity than anticipated or introduced information previously unknown to P9. In some instances, participants in FG4 also discussed testing AI, stating that “I already know it, I’m just gonna see if it knows” (P9) and exploring AI-generated content, saying “will space keep expanding? Ask it, I really want to know” (P9). FG4 asked six questions in total. Four of the six questions were categorized as fact questions such as “will space keep expanding?”, and “how long ago was the big bang?” One was a creative request: “make a poem about Chicago”, and one was an operationalized opinion question: “how many years till the next major ice age?” ChatGPT replied with correct answers to all fact questions, produced a poem for the creative request, and replied with a caveat to the operationalized opinion question (see Figure 1). Overall, FG4’s favored interaction approach appears to consist in socializing with AI, which motivated P9’s factual questions about science, a “shared interest” between P9 and ChatGPT. However, not all fact-based questions were given in such a social manner. Some groups were explicitly trying to trick the AI with fact questions they thought it would not have an answer to, such as FG1’s “what day was the Normandy invasion of 1944 planned to be on?” which is a different day than when it actually happened because of a heavy storm off the coast of France.

Therefore our inductive categorization of groups took into account not only the prompts but also their dialogue and stated intentions. While FGs may have exhibited multiple kinds of behaviors, we categorize them here based on the most prominent approach they took during their interaction. Overall, FG1, FG2, FG7, and FG9 mostly focused on testing AI’s knowledge and capabilities – we refer to these groups as AI testing-oriented. FG4, FG5, and FG8, prominently approached ChatGPT as a social entity – we refer to these groups as AI socializing-oriented. FG3 and FG6 mostly exhibited interest in the content ChatGPT was producing rather than ChatGPT itself – we refer to these groups as content exploring-oriented. We present each approach with examples from FGs where that approach is most prominent.

Separate the analysis on interaction approaches, we also examined the entire corpus of transcripts for conceptions and misconceptions shared by the participants during their interaction with ChatGPT. We extracted a total of 57 AI-related conceptions and misconceptions from the data (see Figure 2). Of these, 15 referred to "What is AI?", 33 to "What can AI do?", and 9 to "How does AI work?" Our analysis of the extracted conceptions and misconceptions surfaced themes such as assigning human attributes to AI, comparing AI to other tools or entities, and sharing assumptions about AI’s intelligence, capabilities, and modality of interaction (see Figure 3). We note that the categories are soft boundaries and the extracted conceptions and misconceptions might overlap between guiding questions. For example, assuming AI uses voice commands [M-15] refers to how AI works, but can also relate to recognizing what AI is, if participants are assuming that modality of interaction because they are conflating ChatGPT with Siri or Alexa.

Our findings point to a diverse range of topics that hold personal and/or cultural relevance for children such as school and academics, hobbies and interests, pop culture, and technology and gaming. In light of this, educational interventions might benefit from incorporating a wide range of topics or open-ended activities, enabling students to delve into their own interests through AI. While interests are varied, common threads like science or creativity emerge as potential focal points for group engagement. Incorporating design elements familiar to youth culture, such as Minecraft’s or Roblox’s design features (e.g., pixelated visuals), could serve as a shared platform for groups of children to discuss AI topics around. This observation aligns well with the insights from Ellis et al. [23], which suggest that embedding technical AI content in socially relevant contexts can engage a broader spectrum of learners. Therefore, a nuanced approach that blends individual interests with broadly appealing elements could enhance the learning experience.

Based on our open-ended study design, participants explored ChatGPT in their own ways; this led to a number of missed opportunities in their explorations. More specifically, our analysis indicates that AI testing-oriented groups primarily focused on evaluating ChatGPT’s knowledge and capabilities through fact-based questions they already knew the answers to. We suggest that this approach may restrict a full exploration of ChatGPT’s abilities (e.g., its creative potential) as it only surfaced ChatGPT’s ability to generate what one participant referred to as “informative essays” (P23, FG9, AI testing-oriented), giving the illusion of competency. Further, this focus on fact-based questions could actively perpetuate the misconception that ChatGPT is a fact repository that delivers accurate information. Additionally, fact-oriented prompts rarely triggered ChatGPT to explain itself or its capabilities, which we consider to be a learning opportunity for participants. Another example of a missed opportunity for exploration is that, unprompted, participants did not ask or discuss any ethics-related questions about what AI *should* be used for, signaling a gap and potential design opportunity. These observations resonate with prior work by Markelle et al., which states that participants did not accurately estimate the strengths and weaknesses of AI agents [48].

Even so, specific interests and prompts did occasionally surface new areas of exploration by highlighting important limitations of the system. For example, P4 in FG2 was surprised and curious about ChatGPT not having access to real-time information, after asking a seemingly simple, factual question (i.e., “how many people are on earth?”). This aligns with insights from Ellis et al. [23], suggesting the challenge for educators is to help students “address their misconceptions and develop an increasingly sophisticated understanding” of AI. We observe that participants’ types of questions (e.g., fact-based) and topics of interest are important to highlight because participants made sense of the agent through questions/prompts about those topics. We suggest a constructivist approach where these types of questions and topics of interest can be leveraged to scaffold participants towards an increasingly sophisticated understanding resulting in them accurately assessing the system’s strengths and weaknesses.

While ChatGPT’s disclaimers about its capabilities (e.g. knowledge cutoff date) captured participants’ attention, more detailed explanations seem necessary to render this information meaningful for them. For example P133 in FG5 mentioned “it says it’s an AI model [...] but I don’t know what it is.” We also propose making AI’s explanations of itself more accessible for the target age group, considering that verbose text that does not match the learners’ reading level might be counterproductive, as expressed by four of our participants. For example, this suggestion can also broadly benefit the design of educational AI systems aimed for middle-school classrooms. The quality of explanations are crucial as highlighted in Wang et al.’s work “recognizing user perception of CAs and providing appropriate feedback to help users revise their perceptions is thus critical in building smooth human-CA interactions” that also allow users to revise their mental model [89].

In informal learning settings, such as museums, it is widely acknowledged that learning arises from the conversations visitors have around the exhibit and oftentimes, revisions to individuals’ mental models happen through this productive talk [14, 27, 74]. We noticed that AI socializing-oriented groups shared a higher number of statements related to AI than the other two group types – this may be due to the social context they created or their interest in the topics discussed with ChatGPT. This dialogue can provide them with more opportunities to adjust their mental models. While we do not argue that the AI socializing-oriented approach is superior to the other two, we highlight the importance of approaches that promote conversations between all visitors and encourage future work to examine why this approach leads to more conversation.

Overall, given the difficulty in crafting effective prompts, as underscored by Liu et al. [56] and Zamfirescu-Pereira [94], we suggest the design and use of pre-designed prompt guides or templates that cover important aspects of the AI agent. Alternatively, for learners who seem more interested in AI’s answers rather than AI itself, the suggested prompts could incorporate their topics of interest to encourage AI exploration. For AI testing-oriented participants, designing short challenges to assess AI’s capabilities and knowledge can help them uncover more dimensions of AI (e.g., ethics).

Anthropomorphism emerged as a notable trend in our study, especially among participants in the AI testing-oriented and socializing-oriented groups. AI testing-oriented groups ascribe diminished humanness to ChatGPT (e.g., stupidity, lying, smart) as opposed to AI socializing-oriented ones who ascribe high humanness (e.g., feelings and experiences) [71]. This anthropomorphic approach aligns with previous research about personification as an inherent strategy for young learners to grasp the concept of programmability [19]. Further, the use of personification has been shown as valuable in introducing or explaining scientific issues to young learners (e.g., steam is escaping through a valve) [21, 34]; however, caution is advised to avoid inaccurate mappings and to prevent false inferences [46]. Additionally, differences in the attribution of humanness, among similar topics such as the presence of emotion in AI, often evoke strong opinions in students making them “a potential hook for engagement” [23].

Similarly, incorporating embodied interactions in museum exhibits can not only render concepts more understandable to those with limited prior knowledge but also create a more engaging visitor experience [39, 40, 41, 74]. A potential design direction might involve allowing children to "step into the agent’s shoes" [17] providing them an opportunity for perspective-taking that could deepen their understanding of what it is, how it works, and what it can do. For example, this perspective-taking could support a deeply needed understanding of how AI works, as highlighted by the prevalence of misconceptions around ChatGPT’s operational mechanics in our findings. Additionally, exhibits could consider assigning personalities to AI agents – be it a “spy”, “writer”, or “scientist” – to emphasize particular features or capabilities. For example, the ‘spy’ agent could highlight the trust dimension whereas the ‘writer’ agent could emphasize the creative capabilities of AI. Previous work has shown this approach has the potential to change user’s perceptions of AI’s capabilities [49].

Prior research highlights that open-ended creative exhibits promote prolonged engagement and facilitate visitor-led learning experiences that can lead to more personally relevant meaning-making [8, 26, 35]. Our findings show that all three approach groups, AI testing-oriented, AI socializing-oriented and content exploring-oriented groups, noticed and were interested in chatGPT’s creative abilities, a major capability of recent generative AI platforms [33]. For example, P8 and P10 who were initially disinterested in interacting with ChatGPT, regarding it as too STEM-focused, used poem generation as an entry point to re-engage with it. Additionally, P8 asked if they could draw with ChatGPT, suggesting that offering multiple modalities of interaction (e.g., voice, text, image) can foster more authentic personal expression. Highlighting different modalities of interaction would also allow participants to recognize the existence of different types of AI [60].

Furthermore, participants often expressed misconceptions about how AI functions, likely stemming from the lack of transparency in ChatGPT’s interface. Allowing participants to program or customize an AI agent as a way of co-creating with it can provide a personally meaningful learning experience [17, 61, 62]. For example, in music co-creation, providing sliders that allowed users to control different parameters of the AI agent “not only increased users’ trust, control, comprehension, and sense of collaboration with the AI, but also contributed to a greater sense of self-efficacy and ownership of the composition relative to the AI.” [61]. Additionally, learner-centered explanations of AI could be leveraged to achieve learning objectives as users interact with the generative AI systems [47].