Relating Narrative, Inquiry, and Inscriptions: Supporting Consequential Play

Our work contributes to an extended body of research that responds to calls for socio-scientific inquiry in classrooms (e.g., Yager, 1996; Kolstø, 2000; Linn et al., 1999; Patronis et al., 1999; Pedretti, 1999; Zohar and Nemet, 2002). It also contributes to research on using networked computing technology to connect learners with rich and diverse resources (including expert mentors, instructional videos, books, and journals). Additionally, it sheds light on the scaffolding learners need as they engage in collecting evidence, testing hypotheses, and formulating arguments. Some examples of NSF-funded K-12 projects that have effectively leveraged technology include GLOBE (Global Learning and Observations to Benefit the Environment), KGS (Kids as Global Scientists), WISE (Web-based Interactive Science Environment), Model-It, and The National Geographic Kids Network (see Barab and Luehmann, 2003, for an overview of these projects). The present project will build on these projects, but will also leverage multi-user virtual environment technologies and game-based pedagogies.

Continuing calls for meaningful scientific inquiry in classrooms increasingly conflict with the pressures teachers face to prepare students for high-stakes tests that emphasize factual recall. Given educators’ public accountability and their time and resource constraints, shifts in pedagogy require new artifacts (curricula, instructional tools, assessments, professional development) designed for teachers, as well as methods of accountability that justify their usefulness. Thus one of our interests was whether or not we could design a multi-user virtual environment that engaged students in socio-scientific inquiry, while also ensuring that students come to know relevant content in ways that would be revealed on measures of scientific achievement.

Our Socio-Scientific Inquiry Framework

In terms of supporting meaningful learning, our interest is not in aiding students in “acquiring” facts related to a discipline. Instead, we are interested in students becoming what Lave and Wenger (1991) described as knowledgably skillful. Someone who is knowledgeably skillful does not only know about a domain, but can also use domain-related practices and resources to address important problems. In our thinking, practices include those that are material-related (e.g., using tools, objects), discursive (e.g., engaging in language games), and conceptual (e.g., using concepts to develop inferences) (see Roth, 1996a, b). Resources refer to the content of a domain when it is enlisted by a practice towards the satisfaction of a particular domain problem. The types of problems of most interest to us are those that evade simplistic explanations and that require the balancing of a host of issues in advancing plausible hypotheses and solutions. In fact, a core focus in our work has been to engage children in ‘socio-scientific inquiry,’ the process of using scientific methods to interrogate rich narratives about societal issues that have a scientific basis, yet whose solution requires balancing scientific claims with political, economic, and ethical concerns. For us, socio-scientific inquiry initially included three core components: narrative engagement (“context”), inscription construction/deconstruction (core “resource”), and scientific inquiry (core “practice”) (see Figure 1).

Framework for supporting socio-scientific inquiry.

Full size image

Focusing on these core components, the first challenge was to develop a narrative that would be compelling to students and whose solution required using scientific inquiry to use scientific resources (graphs, diagrams, models, concepts etc.) in the service of identifying underlying cause(s) of the core problem introduced by the narrative. Narrative entails a setting through which characters engage in a plot or series of actions (Murray, 1997). With respect to socio-scientific inquiry, narrative has the additional role of contextualizing particular content, thereby transforming it from facts or concepts to be memorized into useful tools to address significant issues. Bruner (2002) suggests that it is through narratives that meanings are made. The content of narrative is self-evidently appealing: we lose interest in a world without story (Mateas and Sengers, 2003). Leveraging 3D technologies and game-based methodologies, we were able to move beyond the presentation of a static or linear narrative, instead establishing an interactive narrative in which the ‘‘reader’’ has agency in co-determining how the story unfolds (Traenkner, 1998). An important aspect of socio-scientific inquiry that influenced our interactive narrative is that the science necessarily underdetermines the solution, in that students must consider political, economic, and ethical implications of any solution even if it has scientific merit. In fact, in our work the best ‘‘scientific’’ solution has the most problematic implications in terms of the key stakeholders.

The second core component concerns inscriptions. While there are many scientific resources that can support inquiry (scientific concepts like erosion, facts like amount of dissolved oxygen in healthy stream water, tools like water analyzing equipment), of particular interest are inscriptions written or printed objects (e.g., graphs, charts, tables, equations, schematics, timelines, diagrams, etc.) that represent and crystallize knowledge or information and that use a standard convention, thereby requiring specialized understandings to interpret them in meaningful ways (Roth and McGinn, 1998). Inscriptions are powerful because of their potential to focus in on and simplify particular data from a larger narrative: for example, using a diagram to represent the process of erosion, a graph to show the changes in cost-revenue over an extended time period, or an equation to demonstrate the relations among time, distance, and speed. An essential component of learning science is the ability to use simplified representations (inscriptions) to understand and even make decisions about the larger narrative from which the inscription was derived. In this way, an inscription, when properly situated, can be thought of as a conceptual tool for making sense of the world.

The third component in our model of socio-scientific inquiry concerns the process of scientific inquiry, solving problems in a particular discipline using particular knowledgeable practices. These practices can be conceptual or tool-based and can take on many forms, but always involve someone using resources to carry out an activity. We view inquiry as a dynamic approach to learning that involves exploring the world, asking questions, making discoveries, rigorously testing those discoveries in the search for new understanding, communicating findings, and considering solutions in terms of their societal impacts (AAAS, 1993; NRC, 1996). The framework of socio-scientific inquiry advanced here positions inscriptions and scientific inquiry more generally in the context of a larger societal narrative; or, reciprocally, positions narrative and inscriptions in the context of inquiry. Using this initial framework, and with the broader goal of understanding how to leverage game spaces to support academic content learning, it was our interest to examine fourth graders using a persistent virtual world that was troubled with a number of socio-scientific problems children could investigate.

The Quest Atlantis Environment

This study took place in a 3D, multi-user virtual environment known as Quest Atlantis. Building on strategies from online role-playing games, Quest Atlantis combines strategies used in the commercial gaming environment with lessons from educational research on learning and motivation (Barab, Arici, & Jackson, 2005; Barab et al., 2005). It allows participants to use an avatar to travel to virtual places to perform educational activities (known as Quests), talk with other users and mentors, and build virtual personae. A Quest is an engaging curricular task designed to be entertaining, yet educational. In completing “Quests,” students are required to participate in both real-world and simulated, socially and academically meaningful activities, such as environmental studies, researching other cultures, interviewing community members, and developing action plans. All of these academic activities are embedded in a secure online gaming context where children explore the 3D virtual environment, “chat” online with other students and teachers using Quest Atlantis, and take part in the story of Atlantis – a complex civilization on a far away planet, similar to our own and in need of help.

Quest Atlantis includes a number of persistent worlds where users can congregate and engage in particular activities. Some of these are related to science, and others focus on academic content areas associated with social studies or language arts. For this study, a storyline was written and a virtual world called “Taiga Park” was created. Through participation in the unit, students: (a) learned concepts including erosion, eutrophication, water quality, and system dynamics; (b) built skills including graph (de)construction, hypothesis generation, water quality analysis, socio-scientific reasoning, and scientific inquiry; and (c) developed a richer commitment to environmental awareness. Additionally, and consistent with the socio-scientific framework being advanced here, students were expected to develop an appreciation for the complexities involved in scientific decision making, having to balance ethical, economic, political, and scientific factors (e.g., the best solution from a scientific perspective is not sustainable from an economic or political perspective and results in high unemployment).

The storyline suggested that the park was facing a problem in that there had recently been a decline in fish numbers such that the fishing company generating important park revenue was threatening to leave. This was introduced in a letter, a fictional plea for help in determining the problem from Ranger Bartle who managed the park. There were three groups: an indigenous population, a logging company, and a fishing company all of which were blaming each other for the problem and all of which contributed to part of the problem. For examples, the loggers were not leaving an adequate buffer zone and consequently, erosion was occurring; the indigenous group was using gill nets; and the anglers were over-fishing.

The underlying narrative then gets further fleshed out in greater detail as students navigate the virtual environment and click on non-player characters who share their diverse perspectives. The virtual world included two rivers (one starting in the Northeast and the other in the Northwest) that ran together in the center of the park (Figure 2 is a screenshot of the 3D park and Figure 3 is a 2D map). Different viewpoints are learned by navigating through the environment and clicking on any of the 13 non-player characters (computer bots). Clicking on one of the characters triggers a dialog tree in which the student selects statements to which the virtual character would reply. While selecting different choices revealed different specifics, the dialog trees and the content pages to which they were connected were designed such that, by the end of talking to the character, all students had access to similar information, although phrased somewhat differently. More than a static presentation, students actually had to explore the Taiga context to find each of the different virtual characters if they wished to gain a rich appreciation into the happenings of Taiga.

Screenshot from the Taiga Virtual Environment.

Full size image

A 2D map of the Virtual Taiga National Park.

Full size image

The story was designed to engage students in the role of expert helper to assist Ranger Bartle. In that role, students interviewed people with different perspectives on the problem (Assignment One), collected and analyzed data to develop a hypothesis about the problem (Assignment Two), with the intent of proposing informed solutions (Assignment Three). Students engaged a socio-scientific inquiry process that involved three central interrelated activities and continual reflection and revision of one’s understandings. This cycle began with immersion into a narrative with the task of identifying the problem. This was followed by a more substantial investigation, involving gathering and analyzing data, recognizing stakeholders, and formulating a hypothesis.

An important aspect of socio-scientific inquiry is that the narrative “unfolds”; that is, the narrative is revealed over time based on student choices. In the case of the Taiga, students began with the printed letter and then moved their avatar through the virtual space and clicked on virtual characters who shared information. For example, in the beginning of the simulation after clicking on Lim, the Logging Mill Manager, the students respond to a dialog tree, and Lim’s follow-up response is based on choices made by the student in the dialog tree:

Manager Lim: “Welcome to our little piece of the Kongakut. I am Lim. We’re just a small operation now, but we are growing. Did you know that we added twenty new jobs to this area last year? I’m very proud of our logging efforts.”

• I thought this was a national forest, why do you call it “our” little piece?

• You have a very impressive operation here. Could you give me any more information about this area?

If a student chooses the first question, then Lim responds defensively and provides less information than if students ask the more polite second question. After completing the first Quest, Lim provides students with different information. Also, whereas Lim tends to ascribe the water quality problem to Acid Rain, Norbe (the leader of the indigenous group) and Maria (a park visitor) blame the loggers. In contrast, Sara, the girlfriend of a logger, defends the loggers:

From what I understand, everyone is anxious to hear your report. I know the loggers and the people here at K-Fly really want to make sure the park remains healthy. Financially, the loggers invest more than any other group to keep the park healthy. My boyfriend tells me that Ranger Bartle might have had to shut down the park if the loggers didn’t pay as much as they do to log here.

As a result, student inquiry involves gathering both social and, eventually, scientific information.

In particular, students eventually collected chemical indicators of water quality, including dissolved oxygen levels, turbidity, phosphates, nitrates, and pH. They would collect samples at different sites of the river (see Figure 3), and brought them to a laboratory where they received values for the different indicators. Note that the space was designed such that students could see each other in the space, and that most could only collect two samples so they were forced to interact with each other. Once they got their samples, they were coached by the computer character representing the technician to interpret the findings and eventually develop claims about the impacts of the various land use activities along the river. Students used the data as evidence about the problem. For example, the high turbidity values of the water near where the loggers were cutting down trees indicated an erosion problem.

Using this data, students were to pose a solution that balances the scientific evidence with a political and economic awareness; for example, knowing that the loggers provide the largest source of revenue to the park or that the indigenous people are not legally required to follow park protocols on their own land. In addition to information directly relevant to the Taiga problem, students were also exposed to irrelevant information and red herrings, such as Manager Lim suggesting that acid rain was a causal factor and even presenting graphs to support his argument.

For this study, naturalistic inquiry methodologies involving both quantitative and qualitative data were used to gain a holistic vision of the 10-day long intervention. Rather than segmenting out various aspects and examining them independently, we have attempted to develop a holistic view of our intervention. More specifically, our research agenda has been consistent with Brown’s (1992) notion of “design experiments,” in which naturalistic interventions are designed (as opposed to constrained laboratory contexts), and then the impact of innovations on the learning process is examined and data are used to inform subsequent designs. An important aspect of design-based research is that the intervention is theoretically inspired such that the goal is to demonstrate local impact, but in the service of advancing more generalizable theory (Barab and Squire, 2004). Because design experiments develop theory in practice, they lead fairly directly to “useful” theory. It is important to note, however, that the basic methodological concern for the independence of the researcher and the learning environment is not operative in design-based research, which fundamentally challenges the credibility of assertions generated through design research. In the present study, this issue is partially addressed by the use of carefully developed assessments that allow us to make claims about the transfer of ways of knowing supported in the learning environment to successful participation in subsequent knowledgeable activity.

Participants

This study took place in a fourth grade gifted class located in a Midwestern town. There were 28 students, with 16 girls. The teacher was an exceptional educator with whom the first author had been involved in another project and who was very comfortable with university educators coming to conduct research in her classroom. The students had all been labeled as gifted and had done other project-based, collaborative, and inquiry-based work. However, they had not used the computer laboratory much to this point, and had rarely used the Internet as a meaningful part of their curriculum. The particular unit being observed lasted 2 weeks, including a pretest and posttest examination described below. We videotaped class segments throughout the 2-week intervention and conducted numerous mini interviews with students. We selected this particular class because we were confident that the teacher had enough experience using Quest Atlantis and working with research studies that we would be able to obtain a great deal of input for further improving the environment. Clearly, the use of gifted students constrains the generalizability of our findings as does the fact that a number of researchers whom were present in the classroom during implementation. The reader should note that the Taiga curriculum was developed to target topics normally taught in grade five (and has been implemented in over two dozen non-gifted classrooms around the country), and that the distal level test items were aligned to fifth-grade standards.

Materials

The Taiga curricular unit was designed to engage students in scientific practices (including graph (de)construction, hypothesis generation, water quality analysis, socio-scientific reasoning, and scientific inquiry) and support them in learning concepts (including erosion, eutrophication, water quality, and system dynamics), and to care about saving the environment. Central to these understandings was an appreciation for the nature of complex systems and how real-world problems have complex causes and solutions whose properties-as-a-whole do not derive from the simple combination of constituent parts. These systems-level dynamics function across disciplines and involve balancing ethical, economic, political, and scientific factors (e.g., the best solution from a scientific perspective is not sustainable from an economic or political perspective and results in high unemployment). In addition to the Taiga virtual world described above, students also received a field notebook to record notes and focus their particular task. Also, teachers were given a unit overview with key information about water quality to help them facilitate the implementation of the unit.

Data Collection

Learning in the Taiga curriculum was examined with a range of measures that we organize in terms of four levels. Each of the levels represents an increased “distance” from the specific routines that defined the 10-h Taiga curriculum, with each level employing increasingly formal characterizations of the targeted ways of knowing (Hickey and Pellegrino, 2005; Hickey and Zuiker, 2003). Adapting the labels established by Ruiz-Primo et al. (2002), we distinguished between (1) immediate-level analysis of the discourse that defined the enactment of the curriculum, (2) close-level examination of artifacts from the enactment of Quest Atlantis, (3) proximal-level assessments that targeted the specific concepts and skills that the students were expected to learn in the Taiga context, but in a somewhat more formal and new context, and a (4) distal-level test covering that the targeted state science standards, in entirely new contexts, and not necessarily even the same specific content. Following is a description of the methods and instruments used at each level.

The immediate-level observational data were gathered via direct observation and field notes, as well as the use of multiple video cameras directed at individual learning groups in a particular classroom, interviews with students and teacher, document and artifact analysis, and retrospective recall analysis. Consistent with the work of Roth (1996a) and Barab and Hay (2001), these efforts collected data that: (a) documented practices (e.g., tool use, problem solving, student inquiry) and resource use (e.g., concepts implemented, tools); (b) captured the discussions among students and between students and teacher; (c) documented the progress of student work; (d) traced the same students, artifacts, actions, and procedures over time; and (e) supported and refuted emerging hypotheses about how practices, resources, task constraints, task manifestations, and student understandings evolved over time. The issues were redefined during fieldwork, group meetings, and increasingly focused data collection and analyses. In constructing and triangulating interpretations (Guba and Lincoln, 1983), we used field notes, interviews, document analysis, previously developed case studies, and the evolving codes stored as part of the In-Vivo database.

These data collection efforts resulted in a large corpus of the data, including over 20 video recordings of student-student and student-teacher interactions, as well as unstructured interviews about their interactions in the space, approximately 50 pages of field notes and transcribed video interactions, numerous student-produced resources and artifacts, online databases containing student work and interactions in the MUVE context, and pretest/posttest performance. Lincoln and Guba (1985) recommended triangulation as one means of increasing the credibility interpretations derived from naturalistic interpretations. Interpretations were triangulated using these multiple data sources, which were coded and sorted according to the three different socio-scientific inquiry components using the qualitative research In-Vivo software.

The close-level learning outcomes were the contents of the submissions that each student submitted in each of the three Quests. The first was completed during the fifth class period and had students write a letter explaining the data they had collected and what this told them about the observed environmental problems. Quest Two had students collect and analyze water quality samples at three different locations and explain what they found. Quest Three was a culminating activity where students wrote another letter proposing a solution, based on their inquiries, for the environmental problem. The content of the submission (especially for Quest Three) was used as a data source to warrant claims about the reasoning processes used by students as they negotiated complexities of the virtual socio-scientific issue. We employed inductive strategies to interrogate these data with the intent of identifying emergent themes characteristic of socio-scientific reasoning. In particular, we looked for evidence of how students attempted to balance economic and ecological concerns, considered scientific data and other information, appreciated the under-determination of data in socio-scientific contexts, and considered the complexity of the situation. We also worked to maintain sensitivity to patterns reflective of socio-scientific reasoning, which had not been identified a priori.

The proximal-level outcome measures were akin to conventional classroom assessments. They were “curriculum-oriented” in that they were aligned directly to the specific concepts and skills the Taiga curriculum was designed to address. But as more formal measures that assessed those concepts in new and more general contexts, the proximal measures provided useful feedback for comparing how much different students learned in that curriculum, and provided a useful target and valid evidence for increased learning across subsequent refinements of the curriculum. The measures consisted of two challenging open-ended performance assessment items that asked students to solve new water quality problems and provide a rationale for the solutions that they generated. Doing so supports claims about ways that the Taiga experience increased a student’s ability to engage in socio-scientific reasoning and to understand science content knowledge. Proximal questions reflected these goals, but not necessarily in terms of the curricular topic. For example, the curriculum considered erosion along the bank of a river, but a proximal question might consider erosion along the shoulder of a road. The scientific concept is the same in both, but the assessment topic represents it at a greater distance from the curriculum, as suggested by the different contexts in which erosion is featured. An identical version of the proximal-level assessment was administered several days prior to the implementation and several days after.

The distal-level test was designed to let us make claims about the impact of the Taiga curriculum on student’s achievement on high-stakes externally developed tests. The items assessed knowledge of the range of content that made up the targeted learning standards without regard for the alignment of the content or context to the Taiga curriculum. For example, one of the four targeted standards was evaluate claims based on the amount and quality of evidence provided. While Taiga addressed this standard in terms of water quality, the pool of released items included claims concerning air quality, as well as claims entirely unrelated to environmental indicators. Tests consisting of items sampled at random from such pools provide unbiased comparisons of achievement impact with other curricula that target those same standards. We built such a test by first identifying 4–10 released items from existing state science achievement tests that were aligned to each of the 10 targeted standards. We then randomly selected 1 or 2 items from each of these pools, creating an 18-item test. The test was securely administered (i.e., the teacher did not see any of the items), before and after the curriculum, along with the proximal-level assessment. As such, score gains could be used warrant claims that the 10-day curriculum was very likely to have impact on externally developed science achievement measures that the study participants encountered as part of mandated criterion-referenced achievement testing. It is important to note that raising scores on such a “far-transfer” measure in relatively short-term interventions is a challenging goal, particularly in an initial implementation.

Data Analysis

Field Notes, Observations and Student Interviews

In examining this data, we had the joint goals of crafting a rich account of the experience, while at the same time using our observations to evolve the presented theoretical framework. Toward this end, two researchers analyzed the 10 days of transcribed video in terms of the three core components (narrative, inquiry practices, and inscriptions (de)constructed). In presenting this data, we looked for examples that were illuminative of local happenings and at the same time would prove useful to others (Geertz, 1983). While we tried not to overly impose structure on the data, we were particularly interested in the three theorized components. Therefore, the majority of the data was coded in terms of one of these components, although the particular valence and interpretations of how, for example, a particular instance related to narrative was data driven. Based on the data, the two coders developed from 15 to 28 subcategories (e.g., narrative-alternative conception) for each of the core categories: narrative, inscription, inquiry.

Once we began to generate a list of interesting happenings and resultant insights, we asked of the data whether these insights were representative of the experience (Stake, 1995). To accomplish this task, we grouped existing data under various headings and then used the data to build the core descriptions, again ensuring that the characterizations had both local meaning and experience-distant significance for others. In writing the characterizations presented in the results section, we examined all the data coded in terms of the core category and negotiated what particular interpretations and data selections should be included in this paper (Strauss and Corbin, 1998). This resulted in a richer accounting for what the three core components looked like in practice, leading to the enabling structures described below. We shared our final claims with the teacher and, where appropriate, students in the class to confirm that we captured the experience from their perspective.

The student performance on the close-level classroom assessment and the distal-level test were analyzed using standard quantitative methodology. Three raters used a rubric to score the open-ended items. Inter-rater reliability was then calculated before the differences in the codes were reconciled. Scores on each assessment were summed and the resulting scores were analyzed using standard repeated measures analysis of variance.

Following is a review of the evidence of learning in the Taiga curriculum across the various levels of analysis. First we discuss the immediate-level evidence that was derived from analyzing the classroom discourse that comprised the enactment of the Taiga curriculum. We then describe the close-level analysis of students’ submissions to Quest Three, a culminating activity that required students to apply the insights gained in the previous 10 class periods. We then consider evidence of increased individual knowledge on the proximal-level classroom assessment and the distal-level achievement test.

Immediate-Level Evidence in Classroom Discourse

Narrative Engagement

As described above, we examined the video recordings of student discourse in the computer lab where they completed the activities for evidence that students were actually engaged in the Taiga narrative. There was substantial evidence that this was the case. For example, in the classroom discussion below, a student who successfully completed the first Quest tells his classmates that people are now saying different things:

Teacher: How did you find that information?

Girl 1: You just talk to Anna and Tom.

Girl 2: Yesterday and earlier this morning, I talked to them and—well, I didn’t really talk to them, we were just having a discussion

Teacher: Are you getting new information by talking to the people again?

Boy: Yeah.

Teacher: Oh, good.

Girl 1: They’re saying many different things and they all know that you’ve submitted the quest.

Also of note in the above exchange is the girl’s reflection on the interaction as involving a discussion with the virtual character—illuminating the richness of the experience students were having.

On many occasions, students referred to the virtual characters as if they were real. By the end of their experience, students even began revealing their frustration with the behavior of virtual characters. One student in a fairly passionate class discussion stated:

Boy: I have a question: Why aren’t the people just coming clean? Why aren’t they saying ‘I think I did this because we’ve been doing this’ or something? ‘Cause they aren’t really saying what they’re doing or coming to the park or—they’re not coming to the Rangers saying ‘Can we have a meeting so everyone can say what they’ve done?’. I mean, it seems like if you wanted to help the park and you wanted to still use it, the rangers would be more happy if you told them what you were doing now and try and change your ways instead of keeping going ‘cause you’d probably get kicked out of the park if you don’t tell them now. You’re probably gonna ruin the fish population. And it’s gonna take them at least probably two years to at least get it back to normal.

Appreciating the socio-emotional dynamics of people, another student replied that, “it’s kind of hard to just say that ‘I’m the problem, please blame me’ because usually people have a very hard time doing that, but it would be better for the park.” What is important to highlight in this dialog is that the school science lesson involved much more than examining dissolved oxygen or interpreting graphs; it involved sorting through complex issues that were as much human as they were scientific.

The narrative in the Taiga simulation was not simply about virtual characters, but about engaging in unfolding rules that altered due to one’s behavior. In this way, students were not simply observers, but also, as evident by their language, they were participants in the unfolding story. Also of import, the narrative was not located in any one place, but was distributed across multiple interactions, some of which involved reading about a character, and some of which involved acting on affordances such as collecting water quality samples.

Boy 1: Oh, so you can get the water samples but you can’t analyze them until you have powercells.

Girl 1: Actually you have to do your quest then you can go to the Mission Terminal next to Ranger Bartle to get the powercells.

Boy 1: So you go to the mission terminal—I already have one from Ella so, go to the mission terminal and then it says ‘Click here’ to get the power cells, and then you get your power cells and you click on that, then I go to the bottles, to the other side of the room and then I click on—

Boy 2: You click on the ‘Analyze here’ sign.

Girl 1: But have you actually collected anything yet? Any water?

Boy 2: You know how there’s three docks? There’s bottles on each dock, and you click on the bottles and then it will say what you got.

The challenge was to integrate the context such that being immersed in the narrative meant being immersed in science. To be clear, the narrative was not simply a story to be read, but a situation to be interrogated.

Teacher: Okay. Does anybody else know about fly fishing?

Boy 1: I know something about K-fly. They do it [fly fishing] during the fish spawning season. They hold the tournament, and they’re only catching adults.

Teacher: What’s spawning season?

Girl 1: It’s when the females are carrying eggs.

Teacher: Explain that a little bit more.

Boy 2: So, I got this from the Mulu, from Norbe: they hold the tournament during the spawning season, and he says that they claim they catch and release, but he says that the trauma from being caught and being thrown back into the water is probably too much for the fish. Especially since they’re carrying eggs.

Girl 2: Yeah, that’s dangerous.

Teacher: Do you understand what spawning is?

Boy 1: Yeah.

Teacher: And if you catch a fish who’s carrying eggs, what are you doing?

Girl 3: You’re hurting the fish.

In fact, to understand the narrative meant to understand the science, in that students had to appreciate the importance of different levels of dissolved oxygen, turbidity, phosphates, etc. in different parts of the river. Based on these understandings, as well as discussions with different people about the activities that took place along different parts of the river, students could develop hypotheses about the overall socio-scientific narrative of the river.

While understanding the science was central to understanding the Taiga narrative, it underspecified the solution in that a quality proposal for fixing the problem involved acknowledging economic, political, ethical, and scientific factors. For example, kicking out the loggers meant removing some of the key funding sources for the park. Most of the students seemed to appreciate this tension as evident in the following submission:

Dear Ranger Bartle,

We have come up with a plan to balance the needs of hopefully everyone. In our plan, no one needs to be moved and it will not cost much.

For the first part, the loggers log a little bit more inland, not by the river. Then they use a boat to float the logs down the river to the logging mill. The boat must not pollute the water though.

The pros of this first part of the plan are the erosion won’t keep going into the river. There will be just a little cost for a boat. Only where the loggers log is moved. All of these are good.

The cons of this are that if the loggers choose a boat that pollutes the river, there could be a whole other Taiga fiasco. Another con is that the boat does cost something. Also, the river might take longer to naturally clean itself out. But cleaning naturally is free and easier.

The challenge was not in getting students to appreciate the socio-scientific issues, but in bringing together evidence with complexity such that their responses were not simply relativistic (acknowledging that there were different views, but not using evidence to evaluate one view over another), but instead were probabilistic (drawing on the science to advance suggestions that were potentially more useful than others). As evidenced above and further discussed in the Outcomes Assessment section, all too often student responses separated out the science from the social dynamics, an issue that we hope to address in future iterations by engaging students in more activities involving analysis of different solutions—that is, offering multiple narratives.

A final, related observation involved the amount of narrative. When we chose to situate the disciplinary content and activities in one narrative, we viewed the challenge as getting students to “buy in” to the narrative. Our observations suggest that this was not an issue. Possibly due to the immersive simulation context (perceptual immersion) or to the compelling nature of the story (narrative immersion), literally all students appeared to be engaged, with actual shouts of glee occurring when the teacher announced that it was time to work on the Taiga problem. Students also deeply interrogated the narrative, and used their understanding of scientific concepts to interpret the problem. In terms of the predominance of the narrative as part of their Taiga experience, when the two coders analyzed the data, one stated “perhaps it might be more interesting to look for places where the narrative is not present.” While a seemingly successful experience from the perspective of someone who is trying to deeply situate the content, we now question whether such deep and exclusive immersitivity is the most useful pedagogical strategy. Our data reported below suggests that many students may have not developed a rich appreciation for the underlying science as a generalizable skill.

Scientific Inquiry

We have already discussed inquiry as a process of moving from some indeterminate situation to a determinate or, as we are now suggesting, a probabilistic (best given the data) solution. Developing an informed, albeit probabilistic, response involves making hypotheses, collecting evidence, formulating explanations, challenging one’s understandings, and communicating them to others with a continual cycling back and forth among all these activities. Central to this process is that students begin with an indeterminate situation around which they have a reasonable doubt (Dewey, 1963). In the Taiga curriculum, student inquiry was primarily situated around understanding the cause of the fish decline. The following is an excerpt from the letter from Ranger Bartle that students read before entering the 3D virtual environment.

A few years ago we started to notice a decline in the number of almost all species of fish in the Manongahela River. While fishing is a past-time many people enjoy—and take quite seriously—the decline in fish is a sign of something very wrong with our waterway. I am sure you are aware that the water, the bugs, the fish, the birds, and even us, are all connected. That’s why you have been hired: your job is to determine what is causing the fish decline, how we can work to stop it, and how to balance the needs of everyone who uses the forest.

To answer this question, students first interview the people in the park. As indicated above, different non-player characters provide different insights into the Taiga problem. For example, the Mulu suggest it might be the loggers, while the logging mill manager suggests it might be acid rain. On day two, we interviewed a number of students to determine their appreciation of the problem:

Boy 1: I actually think it’s the loggers. It seems to all start with them. I don’t really think it’s the Mulu ‘cause they’ve been fishing the same for years and years.

Girl 1: Tons of people. The Mulu say erosion. And I just talked to Lim and he says acid rain.

Girl 2: I’m not quite sure yet. I need to be gathering more information.

Boy 2: There’s a lot of people that said the Mulu were kind of bad. ... Well, Ranger Bartle didn’t really blame them, but he said that they went spiking trees and then Markeda said that she needed evidence to not win the Mulu. And then Tom and Anna were suggesting to go talk to the Mulu because they thought something was suspicious about them. And then there’s like—

Boy 3: I think that the Mulu are over-fishing.

Girl 3 & 4: There’s a big trench. We think that the fish are staying in there so that the fishing lines can’t get them because they’re not long enough.

Interviewer: So you think the fisherman should get longer lines?

Girl 3: No, we don’t want them to get longer lines. We don’t want them to find out.

What is apparent in the diversity of responses is that the problem was complex and did require in-depth exploration to truly understand. This was important in that we observed many interactions in which students passionately argued one side or the other, with students even changing their opinions over time.

One could loosely argue that the first Quest focused on the social aspects, with students being asked to submit an essay to Ranger Bartle that overviewed the various perspectives. Quest Two then focused on using scientific data to narrow down the possibilities. This process involved collecting water quality samples at the different locations on the river (see locations “A, B, C” as defined on the map in Figure 3) and submitting them for chemical and biological analyses. This process of understanding what the important chemical indicators were was facilitated by the teacher:

Teacher: We’re going to figure out a few things together then we’ll hopefully go back to the lab, if not today then definitely tomorrow. How can we tell if the stream is healthy or not healthy? Talk to your neighbor real quick. Okay, how can you tell if the river is healthy or not healthy? What are the scientific factors that will help the scientists?

Girl 1: Temperature.

Teacher: The temperature, yes.

Girl 2: The pH.

Teacher: So, temperature, pH, and we know that pH is—

Boy 1: How acidic the water is.

Teacher: Okay, what else?

Girl 3: The dissolved oxygen is the amount of oxygen gas in the water.

Teacher: Why is that important?

Girl 3: To see how much the fish can breathe.

Teacher: Okay. Who else wants to add to that?

Boy 2: Well, if there’s not much oxygen in the water then maybe the fish have some sort of trouble breathing. Maybe something causes the lack of air.

Boy 3: Fish don’t need oxygen.

Girl 4: They need the oxygen inside the water.

Boy 3: Yeah, ‘cause they breathe through their gills.

Girl 4: The bubbles in the water.

....

Teacher: So by comparing the dissolved oxygen amounts in sites A, B and C, we’ll be able to see if it’s similar or if it’s different. What else do we have? Stream needs, or, determine the health of a stream: Ph, temperature, dissolved oxygen, what else do we have?

Girl 6: It needs to have clear water.

Teacher: Clear water. There’s a fancy science word for it.

Girl 5: Turbidity.

We witnessed numerous instances like this where, following students’ use of computers in the laboratory to collect data and develop tentative hypotheses, the teacher would have debriefing sessions with just-in-time, Socratic questioning. Through these debriefing sessions, the teacher was able to ensure that all students were exposed to overlapping information and that they understood the underlying science.

Another essential part of student inquiry was social negotiation, as students worked in small groups, participated in whole class discussions, or even had informal conversations in the hallways. At times these were just occasions for summary, and at others they became heated discussions as students argued for their particular perspective.

Girl 1: Well, the lack of trees doesn’t really have much to do with the fish decline and the soil.

Girl 2: I disagree with that because they said that when they had cut down the trees the silt was running in the water, so the trees couldn’t stop it.

Sometimes these discussions were around the science of the problem, and sometimes around the surrounding socio-economic issues.

Boy 1: Cause I think I heard once that the Mulu need food, but they are kind of fishing more than they actually need. And then they sell them and get more stuff.

Girl: I kind of disagree with him because the Mulu people are actually fishing to live

Girl 2: Well, I don’t see why they would over fish ‘cause if they’re fishing to live, what do they buy at the market? Cause if they’re fishing to live, they could buy other things at the market in exchange for the fish. They could keep some of the fish, but they don’t necessarily have to fish to live. They could go to the market and get other kinds of food.

The important point was that the Taiga narrative afforded numerous opportunities for rich interrogations of which students developed individual alignments.

As students completed their scientific data collection, they were expected to use this information to develop hypotheses about the problem. More specifically, students used the obtained information to develop graphs that revealed certain biological indicators were out of reasonable ranges for Location B and C, for example, but not so for A. Based on this scientific data, they could then develop hypotheses about the source of the problem. An example hypothesis extracted from a student’s second Quest is below:

I think there is one main thing that is causing the decline in fish. The particles in the water absorb sunlight. Since there is SO MUCH sunlight coming into the water that makes the temperature of the water rise. Then, the Dissolved Oxygen goes down. Then, since the D.O. goes down, there might not be enough oxygen to keep the fish alive. So, then the fish start to die off and that can cause a fish decline.

All students were able to submit a Quest 2 response that acknowledged the differences among the three locations and were able to advance the claim that the area by the loggers seemed to be where much of the problem was originating. Much of their understanding was scaffolded by class discussion, as the teacher developed overheads and worked with students to collectively plot the differences and begin the interpretation process. While some of the responses like the above student example seemed to appreciate the complexity of the interactions among the different variables, over half created more simplistic linear characterizations.

It was clear in observing students that the inquiry component of the experience was not simply a scientific process, but one that was very social and involved understanding people. We would argue that this framing, in part, turned the experience from a purely scientific one to a lived experience that deeply engaged the students and led to rich social negotiations. In fact, as the 2-week unit progressed, students began defending people as if they were friends, using first names and even imbuing them with social values (“Lisa wants to help the environment,” “Norbe really cares for the land”). Situating the science as bound up in the “lived” (yet virtual) world also contributed to richer scientific understandings as students debated people, using science as one means of substantiating their arguments.

As a final note on student inquiry, an important component from the perspective of the teacher was the embedded data design. In an interview, the teacher indicated “one of the most useful parts of the Taiga was that all the data was in there. While it made connections to other activities, it did not require external information to make sense of the experience.” Later, she stated that, “I think it might have been a stronger activity if I made more links to other situations ... maybe using our textbook or even current articles and things we had discussed in class.” It would seem that there might be a tension to provide all the information at the same time, not wanting students to have understandings that were too embedded in the one narrative. We think that curricula based on embedded data, such as in the case of the Taiga, one needs to be careful not to create a world unto themselves, but instead should provide a stepping off point, providing both the skills and interest to get involved in other related inquiries.

Inscription (de)Construction

For this paper, the primary resources we chose to highlight in our curricular development and in our data analysis were inscriptions. At times, this involved students reading graphs, other times it involved them creating graphs, and still others involved examining representations of scientific process such as erosion. In terms of the deconstruction of graphs, as students worked on the second Quest about what the scientific data reveals as the problem, they were required to first obtain “power cells” that they would use to run water quality tests. Obtaining one of the samples required that they correctly interpret graphs. While this task did require that they engage in graph interpretation, there was no debriefing about the experience, and some students simply asked their neighbor which multiple-choice letter was correct. The activities were very similar to school-based tests and were not very meaningfully woven into the narrative. More problematic was that students took little time in selecting answers, sometimes under a minute, so that the actual process of working with these graphs was somewhat trivial. The feedback they received for the actions was usually simply a correct/wrong evaluation, as opposed to meaningful debriefing in a way that could leverage their selections to foster deeper understanding.

In contrast to these simplistic interactions, we found that inscriptions were most engaging and useful for promoting socio-scientific inquiry when they contained some ambiguity and/or required interpretation. One of the illustrative examples of a problematic resource was the introduction of a hand-written diagram, in the form of a note (represented in the 3D space as a scroll on the ground outside the laboratory), illuminating what might be happening in the ecosystem of the Taiga. This led to an interesting class devoted to students working in small groups to interpret the meaning of the diagram (see Figure 4). Even before the teacher held a class-long activity in which the students worked in groups to decode this artifact with their peers, discussions emerged surrounding the diagram, its meaning, and the implications of what it illustrated in terms of the Taiga problem. On the other hand, communication around chemical measures, such as pH and dissolved oxygen, were limited mostly to teacher-led discussions.

Diagram of erosion found as a scroll in Taiga.

Full size image

In one class period, when the students were gathered in a circle talking about what new information they had gathered, a girl mentioned that she had learned about bacteria consuming oxygen from the scroll.

Boy: The scroll’s kind of weird but it kind of helps.

Girl 1: That’s right! That (scroll) said there was bacteria in the water.

Girl 2: Did you all find a scroll, a piece of information?

Boy 2: Yeah.

Girl 2: It was near the lab?

Leveraging the enthusiasm around this diagram, the teacher decided to dedicate an entire class period to its interpretation. The students began trying to interpret the meaning of some of the less explicit notations.

Girl 1: Look at the top. There’s a little hint. It says ‘No trees cut down. Bad’, so it must be the loggers that wrote that—

Girl 2: No, no, no. It doesn’t say ‘No trees cut down’. I just figured it out ‘cause I just changed it and I thought it said that too. Then I found that ‘No trees bad’ means there are not trees, so it probably isn’t the loggers and the loggers would think it was good that there was no trees ‘cause then they’d have logs.

Girl 1: But they wouldn’t have anything more to cut down.

Boy: But still, they do replant trees once in a while.

Girl 2: But the thing up there says ‘Trees by the bank, shade, sunlight, water, temp, more dissolved oxygen’.

The teacher assumed a strong classroom presence during the diagram interpretation, using Socratic questions and facilitating interpretive explanations.

Teacher: Okay. To the left of it, it has no trees and an arrow. What does that mean? There’s an arrow right here and another arrow. What do you think that’s showing?

Girl 1: Well, ‘no trees’ is pointing to the water, and in the water there are a bunch of dots. So I think it’s showing that no trees means dirt in the water.

Boy: It says there’s no trees in that spot because they’ve been cut down and then the dots under it, that’s like the stuff that came off the trees when they were being cut down, the silt and dirt and mud that came from under the trees and soil. And then that just goes right into the water once it’s been cut down. ...

Teacher: What’s the next piece of the theory we have?

Girl 1: That the little pieces of soil and stuff absorb sunlight which makes the river hotter, which makes ...

As they uncovered more pieces, especially those that they saw were relevant (such as mention of cutting down trees), the intensity of the conversation increased.

One of the affordances of the socio-scientific framework being advanced is that inscriptions are embodied and not presented as some decontextualized concept. Students consistently made connections back to the Taiga that they were investigating.

Boy: There are things about the particles absorbing sunlight; it sort of proves that the Mulu are not guilty because their temperatures were actually really low, and so that means they had trees too. And the loggers probably are able to cut in the place where the K-fly fish. So I think it’s the loggers and sort of the K-fly fishers combined and not the Mulu, because the Mulus, first of all, they have trees because they’re off the park and not in it, and also, they don’t really have much stuff; their turbidity was really good, so...

While above we critique traditional curriculum as leaving inscriptions disembodied and potentially meaningless, our analysis of the data leads us to question if we went too far on this respect. In other words, inscriptions became overly literal as highlighted in the cautionary statement from one classmate.

Girl: But, this is just a diagram, it’s not the real river. And if you see there was a line between the tree and the other stuff, and so it’s just a diagram and not the real river. So, it doesn’t matter where all the soil is. It’s just a diagram showing what the things do.

Teacher: How many of you agree, this is just a diagram and not an actual picture of the river.

Class: (most raise their hands)

There were occasions in which the teacher made connections to other disciplines and narratives, drawing links to the invariant concepts being discussed and how they related to other narratives.

Teacher: What is silt? Did we talk about this in geology?

Boy 1: It’s like this real sticky gross stuff, kind of like moss.

Teacher: Like moss? What happens when rocks wear down? What is that process called?

Boy 2: Erosion.

Teacher: And then what process takes place next? After erosion, all the sediment’s settled and what does it become? If you walk out into a middle of a river or a lake, and you step in that mucky, muddy, soily kind of stuff—

Girl: That’s silt.

However, these types of discussions and links to other curriculum were few and far between, with most conversations being integrally tied to the Taiga narrative.

All but two students thought the scroll was an important piece of information and worthy of informing Ranger Bartle in Quest 2. We believe that the importance the students gave this diagram can partially be attributed to its appearance as an indeterminate (but potentially important) piece of data; other graphs and diagrams that were integrated into the narrative did not receive the amount of attention that this one did, even though it was not referred to by any other part of the story. We view this difference in part to the scroll’s relatively ambiguous placement in the space, creating a buzz (e.g., “did you find the note?”, “it is really important”, “who do you think left it?”) with the students, as well as to the fact that its meaning was not unambiguously revealed on its surface, but instead required deeper inquiry. The interpretations the students made from the diagram greatly influenced their final assertions of what was causing the problems in the Taiga. In fact, while nearly all of the participants included data about the chemical levels at different parts of the river and used other pieces of data to narrow their hypotheses prior to the report, almost all focused their explanation on what they learned from the scroll, with some failing even to tie in the water quality measurements that were designed to be the core data.

As a final point, when the students came across resources in the virtual world, the non-player characters usually provided some built-in scaffolding to help students access those resources. However, student understanding did not generally go very deep until there was a discussion with peers or the teacher about it. For instance, when talking about the chemical data they would get from analyzing the water samples provided for them in the virtual world, the teacher kept probing until more complex responses emerged:

Teacher: When we get these numbers, what are we going to look for?

Girl 1: Change.

Teacher: Change. Why are we looking for change?

Girl 1: To see where change in your result and where it’s too high and where it’s too low and where it’s just about right.

Teacher: What if there is no change?

Girl 1: Well, then you know that it can’t be the problem of the nitrates.

Teacher: What if the nitrate numbers and the phosphate numbers are kind of the same?

Boy 1: I think if there’s no change then that might not be the problem in the Taiga.

Boy 2: Actually, it kind of depends on what level is bad and what level is good because if it’s bad and it doesn’t change, that might be the problem. But if it doesn’t change, that’s good. That means it might not be the problem.

More generally, the role of the teacher was critical to the learning experience of the students. Much of the deep insight students received from the inscriptions was facilitated by the teacher. More specifically, students in the classroom received strong scaffolding in the form of assignment clarification and refocusing, probing questions, restating student comments, leading class activities and discussions on the issues students were encountering, and providing resources. Many of these resources were introduced, brought to the students’ attention, and/or made relevant by the teacher; just some were highlighted by their classmates.

While interpreting and creating inscriptions are an important part of doing science, rich inscription interpretation and creation requires an appreciation of related science concepts. For example, interpreting a graph that reveals dissolved oxygen levels as being problematic at a specific location on a river requires understanding what levels of dissolved oxygen are problematic. Even more essential is to recognize that getting data on dissolved oxygen is an important step in evaluating the health of a particular body of water. In the Taiga unit, it appears from examining the dialog that we over-emphasized narrative, inquiry, and inscription (de)construction with less focus on having students understand some core science concepts. So, for example, while student interrogation of the erosion diagram did stimulate an understanding of the concept of erosion, this was not a persistent finding. In fact, there was almost no occasion in which students had to interact with (not simply read about) the underlying scientific concept; that is, investigating what are dissolved oxygen or phosphates as scientific concepts. This is clearly a priority for future development.

Close-Level Evidence in Quest Submissions

As described above, we examined student products created through their participation in QA as a means of accessing the reasoning in which they engaged. With these data, we sought to qualitatively describe patterns of socio-scientific reasoning evidenced by the students. In this section, we discuss general themes for the class rather than specific assessments of individual students with a pre-defined rubric. We opted for this analytic strategy for a variety of reasons: (1) Students decided how to group themselves for the final task; therefore, some students worked in small groups of two or three learners while others worked independently. This grouping strategy, chosen by the classroom teacher, prohibited the analysis of individual student responses. (2) Throughout the QA experience, students were encouraged to collaborate with their peers, and the teacher provided classroom forums for the exchange and critique of ideas and strategies. The analyzed products reflected ideas, strategies, and solutions, which had been socially constructed. Therefore, it would have been inappropriate to treat the products as independent assessments of student reasoning patterns. (3) Even if possible, individual scores would not have been overly useful because we had nothing against which to compare them. The very nature of the embedded assessment precluded pre and posttest or control group comparisons. Given these issues, we deemed qualitative analysis of reasoning patterns across the class as the most appropriate analytic strategy.

For the presentation of these results, we first describe attributes of student reasoning that we deemed as strengths. In other words, these were patterns indicative of high quality socio-scientific reasoning. The discussion of desirable patterns is followed by a description of flawed reasoning patterns. In both cases, the description of patterns is supported by examples excerpted from student products; these excerpts are presented in the appendix. Specifically, the embedded assessment reasoning patterns are: balances ecological and economic concerns, presents strengths & weaknesses, considers scientific data, considers multiple lines of evidence, displays inconsistency between conclusion and solution, makes inaccurate scientific assumptions, and underestimates social impacts.

In proposing their solutions, all student groups attempted to balance economic and ecological concerns. This tendency reflected a general appreciation for the complexity and dynamic nature of the problem under investigation. By balancing potentially conflicting interests, the students acknowledged multiple perspectives contributing to the debate. All student groups also explicitly considered both potential benefits and weaknesses of their plans. Whereas it is common for individuals to ignore conflicting evidence and potential weaknesses of a position they support (Chinn and Brewer, 1993), students working in the QA context consistently attended to potential problems with their solutions. It was quite likely that the design of the assignments and teacher supports contributed to the prevalence of this reasoning pattern. The teacher did encourage students to carefully examine their solutions from multiple vantage points, and their products indicated that the students did so. Work from all groups also suggested that students appreciated the significance of inquiry and evidence for socio-scientific issues.

All groups collected data and information from within the QA environment and used this evidence to help explain the observed problem (i.e., the fish kill). In addition, the students demonstrated a willingness to consider multiple lines of evidence, further revealing their appreciation for the complexity of the issue. For example, the students collected water quality data at different points along the waterway in which the problem was observed. Based on these findings, they were able to rule out one group as likely causes of the problem, but reported that the water quality data alone could not be used to assign blame to the two remaining groups. The students talked about how they had to collect other kinds of information, such as analyses of the region’s geography and group activities, to make decisions about the most likely causes of the problem. Socio-science, by definition, involves the assessment of multiple parameters; and the products indicated that students understood this complex dynamic, at least with respect to the issue explored in the QA environment.

In addition to these patterns indicative of high quality socio-scientific reasoning, students also demonstrated episodes of flawed reasoning. The most prevalent problem was a logical inconsistency between evidence-based conclusions and proposed solutions. While it is true that solutions to socio-scientific issues are necessarily under-determined by scientific data, proposed solutions should be consistent with data-based conclusions. Students in all groups appropriately used scientific data to make an evidence-based conclusion about likely causes of the fish kill; however, most of these groups posed solutions that were not necessarily consistent with their conclusions. The students used water quality data, geographic information, and information collected through inscriptions to conclude that one group, the loggers, were the most likely causes of the fish kill. However, in crafting their solutions, all but two groups implicated other parties along with the loggers. In other words, proposed solutions were not necessarily related to the particular data they presented in the argument regarding the causes of the problem. Only sharing evidence implicating the loggers, but then suggesting that the solution is to limit the farming practices of indigenous people is an example of this complication. Most often, the students tended to develop solutions that covered all possibilities, rather than offering a more focused solution based directly on their conclusion. This latter focusing was important because Bartle stated that he cannot simply require all groups to change their practices, for park relations are already strained. Instead, he says that the different groups will only change if there is data implicating them as part of the problem.

The other patterns of flawed reasoning were only expressed in about half of the student groups. Some groups presented creative solutions, but these solutions were based on inaccurate scientific assumptions. For instance, a few groups discussed the construction of a lake to be stocked with fish from the river as a means of controlling the problem of over-fishing. While this represents an innovative solution, it is based on the naïve assumption that riparian species would survive and breed in a lake environment. Given the fact that the students ranged in age from 9 to 11, it was not surprising that they possessed some ecological misconceptions. Several groups also posed solutions that tended to underestimate social impacts. In these cases, students made unlikely assumptions regarding the interactions of groups with very different interests. Here again, these naïve assumptions were not all that surprising given the learners’ age range.

Proximal-Level Evidence of Individual Understanding

Proximal assessment items required individual students to employ invariant properties of the conceptual resources supported by the intervention, but the wordings and graphic representations that invoked these resources varied by degrees from their representations in the learning environment. Because both questions required open-ended essay responses that were typically 1–2 paragraphs, they were scored by three raters. Inter-rater reliability (Kappa, which accounts for chance agreement) across the three comparisons were all greater than 0.8 before the scorers reconciled discordant scores through discussion. Scores ranged from 0 to 3 on each of the questions, so that student’s total score across the two assessments ranged from 0 to 6.

The class’ mean score increased from 1.7 on the pretest to 4.0 on the posttest. Given a pooled standard deviation of 1.5 across the pre–post proximal measure, this gain represented an increase of approximately 1.6 standard deviations. Repeated measures ANOVA indicated the gains on the exam (preM = 1.67, PostM = 3.96) were very unlikely to have occurred by chance, F (1, 23) = 39.73, p < 0.001. This provides evidence that the students learned about the concepts detailed in the intervention in a way that transferred to typical classroom assessments targeting that same topic, but in a more general context. As illuminative of the types of changes we observed, below are the pretest and posttest responses of one student describing the impact that cutting down the trees near a river might have on and around the river:

Pretest: Everyone would die, everything would die as everything depends on oxygen the gas trees give off. Trees give off oxygen something everything needs.

Posttest: Well, cutting down all the trees would make it difficult for people and animals to live and if there was a river or stream near by the fish population would decrease. Trees hold soil still and without them soil will erode into the river absorbing sunlight makeing [making] the temp go up, and the DO go down. Bacteria uses up more DO to decay plants and this environment is unhealthy for fish and they will die. Also trees provide oxygen for people.

Clearly, while she repeats her statement about trees providing oxygen, there is an additional understanding about the possible connection to water quality.

Distal-Level Evidence of Individual Achievement

To reiterate, distal items aligned to targeted learning standards without regard for curricular design and served a broader research goal of learning how to refine inquiry-oriented science environments in ways that impact performance on external achievement tests. Pretest performance left little room for gains; students averaged 13.5 items correct out of the 18 possible. Pointing to one of the many challenges in constructing and impacting valid proxy measures for external achievement tests, our design precluded enough distal items to adequately capture the range of competencies for both this gifted class and the other classes against which the measure serves as a comparison. Mean scores increased by 0.67 items nonetheless. Repeated measures ANOVA indicated that this gain had a roughly 1:8 chance of occurring by chance [F (1, 23) = 2.57, p = 0.122]; with a pooled pre–post standard deviation of 1.9. This represents a gain of 0.35 standard deviations.

We reiterate that the relationship between specific interventions and student performance on valid distal-level assessments is quite indirect, and a challenging goal. These findings highlight the challenge in supporting socio-scientific inquiry in ways that also impact achievement measures. A multi-level assessment strategy considers socio-scientific inquiry across levels of instructional sensitivity in order to explicate ways that and degrees to which curricular experiences transfer to the grainsize of various assessments (cf. Mislevy et al., 2003). By using the same achievement measures in subsequent future refinements of Taiga, we will refine both our curriculum and our model of practice for ensuring further transfer to these increasingly important forms of evidence.

We believe that our socio-scientific inquiry framework and this particular curricular instantiation have much value in that we had learning gains in terms of this particular implementation, and have worked to align our intervention with more generalizable theory. Overall, students showed statistically significant gains, demonstrated rich insights in terms of their submitted work, were clearly engaged, and participated in rich scientific discourse. Further, through participation in this narrative, students developed a rich perceptual, conceptual, and ethical understanding of science. This understanding did not simply involve factual acquisition, but instead resulted from students’ participation with a narrative that involved a real-world problem and the socio-economic complications that such problems involve. Students’ final submissions further indicated that through their work they gained such an appreciation. However, our research goals were not simply to say whether one could design an academic play space, but to illuminate some of the challenges in the design and implementation of such spaces in supporting science education. We close here by highlighting three lessons learned.

Design Framework for Supporting Science Learning

Central to this work was the adoption of a theoretical framework that included three design foci: narrative, inscription, inquiry. Beginning with narrative, the first challenge was to ensure that the narrative actually immersed students within an issue that was meaningful. In this case, we used a virtual world to establish a sense of immersion. This state of immersion, however, engages more than just the senses: it engages the imagination, the fancy, and does not depend on the richness of the media (Ryan, 2001). The educational promise of virtual reality lies not in engagement with the media, but with the narrative; not only through sensory immersion, but also through narrative immersion. A narrative context that supports true immersion is very different than traditional word problems or what the CTGV (1993) referred to as micro-contexts. These contexts become foil through which children simply have to identify the more relevant numbers to be used in solving the problem. In contrast, our “solution” was distributed across understanding the narrative, conducting scientific inquiry, and interpreting the relevant inscriptions.

In fact, and related to supporting meaningful inquiry and inscription usage, the data suggest that we might have over-emphasized narrative. As a context designed to support socio-scientific inquiry, just as important as establishing a meaningful context was that the narrative led students to engage the socio-scientific process, a part of which involved interacting with scientific inscriptions. The idea is that we would use the narrative to establish a problematic situation that required rich inscription interpretation and carrying out meaningful inquiry. Dewey (1963) discussed inquiry as beginning with a reasonable doubt; for it is in response to a reasonable doubt that inquiry becomes a meaningful tool for students. However, it was not our interest to support inquiry in a vacuum, but instead to use the inquiry frame as a means of focusing children on the underlying science. We found the most involved and meaningful interactions occurred when students were required to interrogate the narrative and the relations of discovered inscriptions to the narrative.

The rich dialog and debate, which emerged as students worked to understand the diagram, involved enlisting what are all too often facts memorized in science class as useful tools for solving a problem. In fact, as students interpreted this diagram or tables of water quality data they had collected, we observed rich discursive inquiry practices. It is exactly these types of experiences that the socio-scientific inquiry framework was designed to support. Students not only developed an appreciation for the underlying science, they also gained insight into the socio-scientific issue and the role of politics and economics in scientific decision-making. However, while their first submission captured the socio-political complexities and their second submission captured the underlying science, their third submission frequently fell short of sophisticated probabilistic reasoning. Students reverted to overly relativistic solutions, and did not necessarily use science to make “productive” recommendations. With that said, students’ pre–post curricular gains did suggest they started to develop an appreciation for such sophisticated reasoning strategies. An important finding, which underpins the next lesson, is that there frequently is a gap between the intended (designed) curriculum and the implemented curriculum.

Implementing Socio-Scientific Inquiry Curriculum

In this paper we began with a theoretical framework for what constitutes the components of socio-scientific inquiry. Using this framework, we then examined an implementation instance designed to support learning, while at the same time further evolving the framework. In this way, the study allowed us to situate the framework in terms of an actualized implementation instead of an idealized conceptualization. When examining the framework in practice, other necessary elements beyond the three core components (narrative, inscriptions, inquiry) were illuminated. These other elements became a necessary part of characterizing the framework in practice with respect to the water quality implementation examined in this study. Rather than see the difference in our conception and the factors illuminated through practice as a mutation of the model or even the curriculum, we have worked to integrate these additional “enabling elements” into our framework. While one can design a narrative and supporting inscriptions to bring about scientific inquiry and engage complex socio-economic issues, each learner will need different supports to successfully take on the process. In our data we identified four possible elements that can impact the implementation: external resources, teacher facilitation, social negotiation, and prior experience (see Figure 5).

A revised framework for socio-scientific inquiry.

Full size image

These four components can be thought of as scaffolds that hold up the core elements in a way that makes them structurally coherent and, in terms of a learning curriculum, useful. For example, one might imagine a student initially failing to consider the economic implications of a particular solution, but—through negotiations with another student—may come to appreciate more of the economic or even ethical implications of the proposed solution. This same appreciation could be facilitated through teacher questioning or teacher led, just-in-time lectures occurring in response to student questions. Dependent on the particular classroom, each implementation will involve different amounts of the four components, with some having a lot of teacher facilitation and social negotiation while others employ more external resources and prior experience. However, if the sum total of these components does not equal some optimal value, then the core aspects of socio-scientific inquiry will not be engaged. Consistent with Vygotsky’s (1978) notion of one’s zone of proximal development and Csikszentmihalyi (1990) concept of flow, the trick is to balance the overall amount of these four components so that the designed narrative space is not too easy so as to be boring, or too difficult so as to prove frustrating.

Vygotsky’s (1978) zone of proximal development refers to the range of accomplishment that can be reached with material and social support as compared to acting alone. Csikszentmihalyi’s (1993) flow theory predicts that “experience will be most positive when a person perceives that the environment contains high enough opportunities for action (or challenges), which are matched with the person’s own capacities to act (or skills)” (Csikszentmihalyi and LeFevre, 1989, p. 816). Combining these two theoretical bases, the challenge becomes two-fold: (1) for designers it involves developing a compelling narrative context that supports meaningful inquiry on significant socio-scientific issues, and (2) for teachers it involves determining the appropriate scaffolding necessary to engage the learners in the process such that it is not too easy or difficult. In our framework, we are suggesting that this latter scaffolding can come from teachers through supporting the learning process, other students through social negotiation, external resources that are available to support interpretations, and prior experiences that make learners ready for such complex forms of inquiry and inscription (de)construction.

Facilitating Transfer of Underlying Concepts

While situativity theorists have compelled us to acknowledge the integral relations of content and context, we are still in our infancy with respect to understanding what is too much or too little (narrative) context. One can critique views that separate content from context; but, at the same time, we need to acknowledge the power of using disciplinary knowledge to meaningfully impact real-world problems. In this first implementation of our design, we appeared to have over-emphasized our particular narrative; and, in the case of the standard focusing on graph interpretation we did not support students learning the skill in a way that translated to performance on standardized tests. The decline in statistical significance of related items—coupled with the dialog in which one student reminded the class that a particular diagram was not a representation of this narrative, but an abstract representation of the scientific process of erosion—lead us to speculate one can overly situate a learning environment.

The fact that the erosion diagram became part of the narrative, rather than being viewed as a representation for understanding the narrative, suggests that too much narrative might make it difficult for students to discriminate between underlying invariant structures (e.g., transferable scientific formalism representing the erosion process) and those that are variant (e.g., different narratives centered on erosion)—the latter being used pedagogically to situate the former. The challenge becomes how to leverage richer narratives and surrounding contexts without complicating the learning situation such that the students get so wrapped up in the narrative that they fail to appreciate the underlying invariant formalisms; that is, not being able to transfer the formal concepts, principles, and methodologies. In other words, helping students to recognize the complexities of the eutrophication problem in a particular watershed should also involve ensuring they can apply this understanding in identifying eutrophication when it occurs in other watersheds.

By emphasizing context over content, learners may fail to identify the nested invariant and potentially transferable domain formalisms, instead developing highly contextualized understandings that have little generalizable value. As expressed by Bereiter (1997, p. 286), “the main weakness of situated cognition is, it seems, precisely its situatedness. In traditional language, the limitations of situatedness are referred to as problems of transfer.” In our work, we do not want children to simply recognize that scientific content has real-world meaning, but we are trying to ensure that students participate in and learn authentic socio-scientific inquiries in ways that transfer to other contexts and even to standardized achievement items on externally developed science achievement tests that are aligned to the corresponding science education standards.

Clearly, this work has limitations. It was conducted in a gifted classroom, and the experience for students was altered given that our research team was asking questions of some of them. However, we believe we have tied this curricular instantiation to a more generalizable pedagogy focused on supporting socio-scientific inquiry, and that this work does shed light on the pedagogical framework. In contrast to our initial overly simplistic theory (as presented in Figure 1), this investigation allowed us to further develop the theory, elevating it from an abstract proposal to an embodied framework that we expect will be of use to others (see Figure 5). It is this latter connection between theory and local practice that makes design-based research such a useful methodology to science educators and learning scientists more generally.

We have suggested that this implementation helped us gain further insight into our framework. We began with an impoverished notion of designing to support socio-scientific inquiry. While one can design a narrative and supporting inscriptions to bring about scientific inquiry and engage complex socio-economic issues, each learner will need different supports to successfully take on the process. Although many variables can account for this disparity, in the evolved framework we discussed four possible enabling elements that appeared to impact the implementation experience. We need to further evolve this framework, using it to design multiple curricular units and examine the resultant learning that occurs. Beyond science learning, we also view the model as having value for defining the ontological aspects of any richly contextualized curriculum: narrative context (i.e., virtual park narrative), core practices (i.e., scientific inquiry), and available resources (i.e., particular scientific inscriptions such as the erosion diagram). We will continue to refine this model both in terms of supporting socio-scientific inquiry in particular and embodied curricula more generally.

While some might find the contamination of science with the socio-political/economical as problematic, our observations suggest that this context appeared to stimulate rich, scientific debate, with the challenge being to help students integrate both into their model of doing science so that doing science is not treated as an activity that is cleaved of the socio-political implications. This is quite a challenge given the school emphasis on separating out disciplines, so that doing science is different than doing social studies. It is our conviction that a pedagogy that situates learning domain formalisms in their broader context is constructive for a variety of reasons: Motivationally, it creates a more engaging learning context than simply teaching the formalisms as coldly cognitive material; practically, it is more consistent with how the content is experienced in the real world in which even “scientific” problems have a socio-economic agenda that must be considered in determining effective solutions; and conceptually, it is more consistent with what we are learning about what it means to know something.

Finally, this study speaks to the power of gaming methodologies and technologies for supporting academic learning. In particular, we leveraged the power of multiplayer online role playing games (Gee, 2003, Steinkuehler, 2006) to establish an interactive narrative and play context for learning about water quality. Gee has argued that well-designed game play immerses the player in a rich network of interactions and unfolding storylines where the player must learn about the underlying game grammar to solve real and imagined problems. In terms of using these spaces for academic ends, designers can, for example, establish a persistent world that immerses the user into a simulated habitat where they research the quality of a virtual river. While not the real thing, the virtual world has the advantage of having readily manipulable chemical levels and other complex dynamics such that they afford rich learning potential. It is our belief that educators have only scratched the surface in terms of the potential of game spaces to engage learners in meaningful learning. Of significance in these spaces is how they position the learner with agency in determining relevant information, making decisions of what to do next, and situations where their choices have consequence.

Further, we believe that there would be value in the future in providing students opportunities to build more connections between the disciplinary content and other narratives—thereby beginning to understand what is invariant (the underlying science strategies) and what is variant (the contexts in which these are applied). It would seem that such an understanding might be necessary in supporting transferable understandings. In the next iteration of the curricular experience we will facilitate interactions in which students will be engaged in dialog explicating the relations of more formal data representations and the contexts in which they are representing—appreciating what is being left out of these representations and why they are useful. We also intend to provide interactions that engage students in comparing local data to that from other contexts so that they can gain a richer appreciation for the representations. With that said, this study suggests that videogames can be effectively leveraged to support academic content learning.