Physical Science Connected Classrooms: Case Studies

TECHNOLOGY-FACILITATED FORMATIVE ASSESSMENT IN PHYSICAL SCIENCE CONNECTED CLASSROOMS: CASE STUDIES Karen E. Irving1, The Ohio State University Vehbi A. Sanalan2, The Ohio State University Melissa L. Shirley, The Ohio State University Abstract Detailed case-study descriptions of secondary and middle school classrooms in a variety of school contexts provide explicit examples of how teachers implement connected classroom technology to facilitate formative assessment in physical science instruction. Connected classroom technology refers to a networked system of handheld devices combined with software specifically designed for use in the classroom. Archer’s social theory provides a framework to interpret progress in technology integration for physical science teachers who have completed their first year of a multi-year study of connected physical science classrooms. From the social theory perspective teachers initially faced challenges as the connected classroom required changes to existing classroom structures and procedures. However, teachers found the system compatible with traditional assessment, formative assessment and also hands-on data collection lessons. As their proficiency developed, teachers used the connected classroom to facilitate both interactive and preplanned classroom formative assessment. Teachers were surprised at the knowledge they gathered about student learning, recognized that their students were also better informed about their own learning, and believed that use of the technology increased student engagement in on-task behavior. Connected classroom technology reduced some barriers to the practice of formative assessment and informed teachers about their students as well as informed students about their own and other’s learning. The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305K050045 to The Ohio State University. The opinions expressed are those of the authors and do not represent views of the U.S. Department of Education. 1 2 Contact information: irving.8@osu.edu Visiting scholar at Ohio State University; Assistant Professor, Erzincan University, Erzincan, Turkey The full paper may be accessed at: http://ww.ccms.osu.edu/ASTE2008/PSCaseStudy.pdf 1 Draft 2008.01.03 Technology-facilitated formative assessment… Introduction Recent research on connected classroom technology with a large scale field trial using a random assignment control group cross-over design indicates that student achievement in algebra 1 connected classrooms is significantly higher than a control group of students in classrooms without the technology (Pape, Irving, Owens, et al. in progress). In addition, moderate correlation between student achievement and teacher technology implementation lends further support to the idea that appropriate use of pedagogy and connected classroom technology improves student learning in mathematics (Pape, Irving, Owens, et al. in progress). Prior work in connected classrooms revealed that this innovative technology aided teachers in determining student’s prior understandings. Researchers reported that connected classrooms fostered an atmosphere that helped students think through issues, establish and commit to positions, and that the resulting timely feedback provided opportunities to improve the quality of both teaching and students’ thinking and learning. Additionally previous study of connected classrooms showed that students felt greater cognizance of class position, better understanding of their own and others misconceptions and a changed sense of classroom community (Abrahamson, Davidian, & Lippai, 2002; Dufresne, Gerace, Leonard, Mestre, & Wenk, 1996; Owens, Demana, Abrahamson, Meagher, & Herman, 2004). Connected classroom technology refers to a networked system of handheld devices combined with software specifically designed for use in the classroom. Networked systems with these characteristics include response systems and classroom communication systems (Roschelle, Penuel, & Abrahamson, 2004). While a variety of connected classroom systems exist (see Fies & Marshall, 2006), this project utilizes the Navigator™ technology produced by Texas Instruments which allows teachers to wirelessly communicate with students’ handheld graphing calculators. This system offers the teacher a variety of components to probe student understanding through student calculator displays, responses to preplanned or on-the-spot opened-ended or multiple choice questions, and the sharing and public display of student-generated individual or aggregated data. However, studies in technology integration efforts also suggest that reforms requiring teacher change in classrooms have been relatively unsuccessful (Cuban, 1998, 2001; Cuban, Kirkpatrick & Peck, 2001; Priestley, 2005). Despite infusion of educational technology resources in school classroom environments, Cuban and his colleagues report little use of these technologies in classroom instruction. Changing teacher practice involves more than simply providing new tools for teacher use. Priestley and Sime (2005) apply Archer’s (1988) social theory as a framework to understand the process of change in schools. Archer uses the terms ‘morphogenesis’ and ‘morphostasis’ to describe change and lack of change respectively in a cultural setting. In some cases, the ideas and knowledge of the change system are complementary to the former cultural situation and are utilized to promote change. In other cases, the new situation exists in contradiction to the existing cultural system and change becomes more problematic. Three outcomes are suggested by Archer: 1) morphostasis, when the new ideas are modified to fit with existing ideas – and little change occurs; 2) morphogenesis, when existing ideas, norms and values are modified to fit the new ideas; and 3) both reform and traditional ideas are adapted to reduce contradictions. This paper views the inclusion of connected classroom technology as a cultural change in schools and explores the process of teacher change in physical science classrooms in diverse school settings as teachers work with connected classroom technology in their classrooms. The research question addressed is: How do physical science teachers in diverse school cultures integrate connected classroom technology in middle/secondary school physical science classrooms? 2 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… Theoretical framework Archer’s social theory provides a framework to interpret changes in teacher classroom practice for physical science teachers who have completed their first year of a multi-year study of connected classrooms. The success of an educational innovation depends on the distinctive ecology of the classroom with the inherently challenging tasks of both managing and instructing large numbers of non-volunteer students. In order for school change to succeed, the new ideas must accommodate both the attitudes and values of the teachers implementing the change as well as the structural realities of school, including staffing considerations, scheduling issues, classroom structure and systems for testing and evaluation (Priestley & Sime, 2005). Reforms may be consistent with teacher views, but may not be congruent with the underlying school structures. Three dimensions describe the practicality of a particular change and may influence the success of a reform effort (Doyle & Ponder, 1977): Congruence with teacher’s values and practice; Instrumentality, or compatibility with the existing school structures; and Cost, whether the reward is worth the effort. This practicality construct, combined with the ecological framework of the classroom, serves as an interpretive tool for understanding how teachers integrated connected classroom technology in their teaching. While student achievement in science and mathematics remains a national priority, US high school students score poorly in comparison to their peers on international tests (TIMSS, 1995 & 1999). Almost every state employs some accountability measure to determine student performance (Council of Chief State School Officers, 2000). However, these high-stakes tests often fail to provide teachers with useful information to improve student learning. Unlike summative assessment, formative assessment occurs every day in the classroom. According to Cheung (2006), formative assessment is “the process used by teachers and students to recognize and respond to student learning in order to enhance that learning during the learning” (p 61). When questioning and feedback are frequent and involve students actively in reflecting on what they know and how they learn, and when assessment data are used to inform and adjust the course of instruction, formative assessment has been reported to achieve large gains in student achievement (Bell & Cowie, 2001; Cowie & Bell, 2001; Black & Wiliam, 1998a, 1998b; Fuchs & Fuchs, 1986). Researchers distinguish different kinds of formative assessment practices. Bell and Cowie (2001) describe both: a) interactive formative assessments – characterized by interactions between students and teacher during the lesson in response to specific classroom ecology; as well as b) planned formative assessment – pre- or mid-lesson assessments that the teacher prepared before the lesson began for specific purposes such as revealing student prior knowledge or midpoint checking of student progress. Ruiz-Primo and Furtak (2007) describe informal formative assessment practices as teacher information gathering about students’ developing understanding as the lesson proceeds. These researchers focused on assessment conversations that take place during a series of inquiry lessons . Studies show, however, that formative assessment use by classroom teachers is one of the weakest aspects of teacher practice (Assessment Reform Group, 1999; Daws & Singh, 1996, 1999). Connected classroom technology supports formative assessment by facilitating the task of gathering information about individual students and rapidly aggregating data for interpretation. Teachers may then use these data to guide classroom decision making (Roschelle, Penul & Abrahamson, 2004). Connected Classrooms Connected classrooms provide teachers and students with a technology system designed to facilitate student-teacher communication. The Navigator™ utilized in this project is a multi-faceted technology that includes student handheld graphing calculators attached by wires to a hub; wireless communication from the hub to an access point wired to the teacher’s computer; and a sophisticated software management package (Figure 1). The Navigator features four components: 1) Activity Center – allows students to contribute points, equations, or lists to a shared workspace that can be projected 3 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… for class viewing; 2) Quick Poll - an audience response system that allows teachers to pose on the spot questions with a variety of student response choices (e.g. dichotomous choices such as true/false; Likert-style responses such as strongly agree, strongly disagree; multiple choice forced answers; and open-ended questions); 3) Screen capture – allows teachers to view and/or display each student’s calculator screen; 4) Learning Check & Class Analysis – allow the teacher to create assessment instruments; distributes the assessments to individual students; collects, aggregates and scores forced answer choice questions; produces a slide show depicting class responses. In addition, probe ware devices such as motion detectors, temperature or pressure probes, can be used to gather data with the graphing calculator. The data can be sent to the teacher’s computer and aggregated to produce whole class data sets for analysis. Figure 1. Connected classrooms Design/Procedure The results presented here are based on data gathered as part of a larger research project (Classroom Connectivity in Promoting Mathematics and Science Achievement [CCMS] - Institute of Education Sciences, U.S. Department of Education, Grant R305K050045). The overall CCMS project design is a randomized cross-over trial where the control group is exposed to the intervention sequentially. The larger study employs a mixed-methods approach using quantitative data for statistical analysis as well as qualitative data for in-depth analysis of connected classroom conditions. The main focus of the CCMS study is to examine the impact of connected classroom technology with interactive pedagogy and professional development on mathematics and science achievement, teacher professional growth, student self-regulated learning and students’ dispositions toward mathematics and science. This paper presents case studies from three purposively selected physical science teachers who have completed their first year of the CCMS study as members of the experimental group. Intervention The intervention consists of four parts: 1) provision of connected classroom technology; 2) professional development consisting of a weeklong Summer Institute held at ___ University; 3) online web-based training and discussion forum for the teacher community; and 4) follow-up professional development at the annual Teachers Teaching with Technology International Conference for two years 4 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… following the Summer Institute training. For a more detailed description of the connected classroom technology and the professional development program, see Sanalan, Irving, Pape & Owens (2008). Data sources & analysis A brief description of the data sources that support the case studies presented in this paper follows. Technology Use and Professional Development Survey This instrument was administered to science teacher participants during the week long summer professional institute on connected classroom technology. The survey included 85 Likert-style questions (4 or 5 point scales) targeted at these constructs: Ability to use computers (Qs 1-10); technology use in science education (Qs 11-39); technology skills (Qs 40-51); professional development opportunities completed in the recent past (Qs 52-62); and science content knowledge expertise (Qs 63-85). Telephone interviews (Autumn & Spring) Participants in the experimental groups were interviewed by telephone in the autumn and spring of the academic year. A telephone interview protocol was established and reviewed by 5 members of the research team. Each question probed a particular construct of interest and included requests for participants to provide explicit examples to document their statements. The Year 1 Autumn interview included 7 questions and was intended to illuminate initial startup issues with teachers in the experimental group. Interviews were conducted by one member of the team (first author), lasted from 20 to 40 minutes, and were audio-recorded. Participants were asked to describe the set-up and installation of their connected classroom systems and their initial successes and challenges. In addition, participants responded to questions about individual component use, studentliking, teaching planning, and descriptions of lessons incorporating connected classroom technology. Telephone interviews in the autumn were conducted mostly during late October and November. Spring telephone interviews included 17 questions and probed teacher practice more deeply than the autumn interviews. Spring telephone interviews began in May and continued through early summer. Interviews lasted about 30-45 minutes and were audio-recorded. One member of the team (first author) conducted all science teacher telephone interviews. Questions probed availability of connected classroom technology and supporting devices, frequency of use, perceived technology implementation successes and challenges, specific descriptions of component use, comfort level, student-liking, lesson planning and implementation, interactions with students, classroom atmosphere, perceived pedagogy implementation, and professional development plans and presentations. Classroom observations Each teacher in this study was observed teaching two sections of science on two successive days, for a total of 4 class periods per teacher. Each lesson was videotaped. Audio portions of the observations were transcribed as faithfully as possible. Each transcript was reviewed by at least two researchers for accuracy. In addition, transcripts were annotated with information regarding nonverbal teacher and student actions such as teacher gestures, teacher physical movements, and board displays that were evident from viewing the video. Screen shots of quick poll and learning check questions and responses allowed researchers to determine exact questions and student responses during videotaped classes as well as class participation levels. Artifacts such as student handouts were collected during observation trips and catalogued. 5 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… Post observation interviews Observers conducted post observation interviews with teacher participants. Post observation interviews lasted about 30-40 minutes and followed an established protocol. Questions were designed to probe teacher planning and implementation of the observed lessons. In particular the teacher was asked to describe lesson objectives, to detail pre-planned formative assessment intentions, to recall interactive formative assessment occurrences, and to reflect on the progress of the lessons observed. Since each teacher was observed teaching the same lesson twice, interviewers also asked about changes that occurred between the first and second teaching of the same lesson. Student focus groups A student focus group (4 to 10 students) was interviewed by the researchers during each observation sequence. Due to student scheduling issues and travel restrictions, student focus group interviews sometimes occurred after all observations had been completed and sometimes occurred during free periods or lunch breaks in the middle of the observation sequence. Researchers followed an interview protocol that included questions about the observed lesson, about other uses of connected classroom technology in their science lessons, and about student attitudes and opinions regarding the technology. Focus group interviews lasted about 20 minutes, were audio-taped and transcribed verbatim as faithfully as possible for subsequent analysis. Physical Science Achievement Test A 46-item physical science achievement test was created by the research team for the larger CCMS project. The process of instrument creation started with a thorough review of the National Science Education Standards, the TIMSS standards, McREL standards and NEAP standards for middle grade physical science concepts. After identifying overlap and differences between these standards documents, the research team examined the state standards for the 12 most populous states. Topics included on 80% of the standards documents for middle school science were included in the final instrument. Released TIMSS, NAEP, and state standardized test items were carefully selected to align with the topics identified from the standards analysis. In addition, items were selected representing both higher and lower order Bloom’s Taxonomy categories. The instrument was administered in a pilot trial to a total of 269 ninth grade students enrolled in physical science classes from two different high schools, one urban and one suburban/rural. The test was administered in May/June after a full year of physical science instruction. Completed tests were machine scored and subjected to item-response theory (IRT) analysis. All IRT analyses were conducted using BILOG MG, using a 3PL model. For each test, the IRT analyses produced estimates of item discrimination (α), item difficulty (β), and guessing (c). On the basis of these results, four items were removed from this instrument as they had low correlation with other items on the test and their removal would result in a higher reliability. These four items also demonstrated other undesirable item characteristics, such as a low discrimination index (1 item), very high difficulty (2 items), and both a high difficulty level and a high index of guessing (1 item). The final instrument contains items that perform in a broad range of difficulty levels, allowing it to more precisely measure differences in student abilities. The overall reliability (Cronbach’s alpha) of the instrument is 0.88. The final instrument contains 42 items: 16 chemistry items, 16 physics items and 10 earth/environmental science related items. Participants While the algebra 1 CCMS study included over 100 teachers in experimental and control groups, the science part of the larger project was envisioned as a pilot study to explore use of 6 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… connected classroom technology in science classrooms (Pape, Irving, Owens, et al., in progress). Nine science teachers and their students participated in the experimental group. From this group, three physical science teachers were purposively selected for this study to explore teacher practice in significantly different teaching contexts. Table 1 summarizes differences in grade levels, school diversity and socio-economic conditions for students at the three selected school sites. The data show that Ms. S worked in a community with high socio-economic levels that was predominantly White. Ms. C and Ms D worked in schools with more diversity in their student populations with large percentages of students from economically disadvantaged families. In addition to different school contexts, the three case study participants experienced different preservice preparation programs. Although Ms.C was the most experienced teacher and the only one with a graduate degree, she completed a teacher preparation program for elementary classrooms. Ms. S completed an undergraduate teacher preparation program for secondary school science teaching and Ms. D’s undergraduate training was in animal science (Table 2). Table 1. Case study participant school demographic information* # Students Grade levels School Diversity Economically disadvantaged Name** Location Ms. C Ohio 662 Grades 6-8 51% White 36% Black 11% Multiracial 2% Unspecified 72% Ms. D Texas 856 Grades 7-8 57% Hispanic 35% Black 5% White 3% Asian/Pacific Islander <1% Native American 82% Ms. S Ohio 897 Grades 9-12 98% White 8% *www.Greatschools.net ** Pseudonyms Table 2. Teacher preparation. Name* Undergraduate major Ms. C Elementary Education Ms. D Animal Science Ms. S Secondary Education *Pseudonyms Year graduated 1987 2000 2003 Graduate degree 1995, Curriculum & Instruction None None Years teaching 18 5 3 The case study participants also self-reported differences in initial technology experience/skills and science content knowledge. The data in Table 3 were collected from the Technology Use and Professional Development Survey administered to participants during the week long Summer Institute on connected classroom technology. The values reported in Table 3 represent percentages calculated using individual teacher scores divided by maximum possible positive scores for each set of items. Higher values represent greater confidence in their technology experiences and skills, participation in professional development opportunities and confidence in their knowledge of physics and chemistry content. All three teachers reported participation in a variety of professional development opportunities offered by their districts, professional organizations and national conferences. Ms. C reported much lower initial comfort level with educational technology and lower confidence in her knowledge of chemistry and physics than the other two teachers. Her elementary science preparation 7 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… program may have contributed to her self-reported lower confidence level in teaching these science content areas (Table 3). An important characteristic of all three case study participants is their willingness to participate in a national field study of connected classroom technology. Table 3. Participant self report of technology skills, professional development activity, expertise in teaching physics and chemistry. Technology Professional Expertise in Expertise in Teacher Experience & Skills Development teaching Physics teaching Chemistry (Qs 1-51, %) (Qs 52-62, %) (Qs 63-72, %) (Qs 73-82, %) Ms.C 51 40 49 38 Ms.D 78 49 72 84 Ms.S 75 40 90 93 Results: Case studies The descriptive multiple case study approach was selected for this research project as the optimal design to compare the individual experiences of participants in the CCMS project (Yin, 2003). The researchers chose the teacher as the focus of analysis and his/her experiences the informative data set. The researchers used interview, focus group and observation transcripts, observation video, and surveys to determine the school context, instrumentality, congruence and cost/benefit views for each participant. All data sources were converted to electronic text and coded. NVivo 7™ software aided in the coding process. Provisional starter codes for the constructs of congruence, resource gathering, student liking, mentoring, school content, cost/benefit, and student engagement provided an a priori coding framework (Miles & Huberman, 1994). In addition, researchers used the formative assessment typology created by Torrance and Pryor (2001) as a guide in establishing coding categories for the four primary steps in the formative assessment process: task definition and criteria; gathering data about students; evaluation of data; and teacher planning. As researchers worked with the coding process, discussions of coding category distinctions were negotiated and refined. A coding book was created to describe each coding category and to document the discussion process. The collected qualitative data were analyzed both individually and together to generate empirical assertions (Patton, 1990). To establish the evidentiary warrant of these assertions, the researchers repeatedly searched the evidence for both confirming and disconfirming examples. Assertions were verified, modified, or eliminated though an iterative process. The assertions from these phases of the study provided a window into the experiences of the three case student participants in different school contexts as they experienced their first year of connected classroom use. Multiple data sources offered evidence for triangulating the data to strengthen the credibility of the assertions. For example, classroom observations, student focus groups, and teacher interviews provided three perspectives on teacher experiences using the connected classroom technology. Because the small sample size did not warrant the use of inferential statistical analysis, descriptive statistics were used to report the results of the physical science pre and post tests. Because the three teaching contexts varied considerably (seventh grade, eighth grade and ninth grade) and did not share a common curriculum, results of the student achievement tests provide only a glimpse into the science achievement levels of the students in these three teachers’ classrooms. In the case studies that follow, the context of each participant’s teaching environment is presented first. Next, we explore the structural support (instrumentality) afforded each participant by their school communities. Gathering resources and social support represent important aspects in the instrumentality description. The third section of the case study describes each teacher’s views of the congruence between prior practice and current practice in a connected classroom. Lastly, for each case 8 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… study, we present results regarding costs and benefits of using connected classroom technology in physical science classrooms. Case study 1: Ms. C Ms. C - Context Ms. C was a veteran (18 years) middle school science teacher teaching eighth grade science for the first time. Previously, Ms. C. taught seventh grade science for 15 years. She completed an undergraduate program in elementary education and held a graduate degree in curriculum and instruction. Of the three teachers involved in this study, Ms. C reported the least prior experience and comfort level with educational technology. Her first introduction to the graphing calculator occurred during the week long Professional Development workshop provided by our research team for all teachers in the CCMS study. For me technology is just overwhelming. It scares me. And in the past, when it comes to the computer, I tend to just go, oh, that’s not for me. (3050.TISp07.P17) Ms. C also reported the lowest confidence in her science knowledge of the three case study participants, not surprising perhaps given her elementary education preparation program (see Tables 2 & 3). Ms. C taught in an inner city school in a mid-size city in Ohio. Students at her middle school were tracked according to their mathematics ability. According to Ms. C, the students in the two class periods in our study represented the lowest achieving students in the eighth grade. Period 1 had an interventionist teacher assigned to help with the many students with individualized educational plans (IEPs). Ms. C. reported that while period 3 seemed higher functioning than period 1 and had no additional aide assigned, many of the students in period 3 also had IEPs. One student was hearing impaired and had a sign language interpreter assigned during science. Around 20 students were present in each of Ms. Cs classes during the days we observed her teaching. Ms. C - Instrumentality Curriculum. Ms. C was teaching eighth grade science for the first time during her participation in the CCMS project. The science faculty at Ms. C’s school adopted the FOSS (Full Option Science System, developed by the Lawrence Hall of Science) curriculum. Ms. C. received training in implementation of this hands-on science curriculum just prior to joining the study. Her school district sponsored a science van that delivered the specialized equipment needed to implement the curriculum to various schools in the county when it was needed. Ms. C utilized this service to provide motion detectors and other specialized science tools for her students. Technology resources. Ms. C’s classroom was well equipped with modern educational technology. Her classroom had an interactive white board, a laptop for her personal classroom use, a computer projector, an overhead projector, her own audio speakers, and a device to project her calculator screen for student viewing. Her students enjoyed access to a class set of graphing calculators. Ms. C’s school purchased and installed her Navigator™ system by the end of October of the first year of her participation in our study. Social support. Ms. C enjoyed a school culture with strong support for educational technology integration and a variety of readily available resources for mentoring and trouble shooting. In Ms. Cs district, the science curriculum specialist had previously received training on Texas Instruments products. In 9 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… addition, mathematics teachers at this middle school utilized the Navigator™ system in their instructional program. In Ms. Cs words: I am lucky enough to have people on my staff [who] know more than I do, so I have a great place to fall back when I need help. (3050.poi07.P124) There is a lady in my district who is going to be a TI trainer….She was so excited … that she came and set up my whole classroom and helped me. I have so much support from the math people. Everyone in my building in the math department uses Navigator™ everyday. (3050.TIAu06.P11) Ms. C also benefited from her building principal (R) who strongly supported educational technology innovation in her school. Principal R attended the International Teachers Teaching with Technology professional conference during Ms. C’s first year in the study to learn more about the technology. In addition he actively searched for and recruited a new science teacher participant at his school for the second year of our study. Despite Ms. C’s self-report of low technology skills, she displayed persistence and consistency in implementing the connected classroom technology. Her self-confidence, risk-taking tolerance, and ready access to technology support from other teachers in her building allowed her to try new things in her classroom. In addition to gathering support from teachers in her school, Ms C reported that her students who had experienced connected classroom technology in their mathematics classrooms were helpful. I’m very much a risk taker. I don’t care if I mess up in front of the kids. I’d like them to know that I’m trying something new, and I might do it wrong. (3050.poi07.P124) Because my students have this in math all the time and a lot of them had it last year in math, they are so knowledgeable about how Navigator™ works….they tell me what to do. (3050.TIAu06.P53) The connected classroom technology fit well within the existing school structure for Ms. C. She enjoyed both social and resource support from her district, building principal and teaching colleagues. The strong community support for innovative educational technology and mentoring by both colleagues and students provided a fertile environment for connected classroom technology implementation to occur. Ms. C – Congruence Ms. C found that implementation of the Navigator™ connected classroom system required her to make little change in her teaching practice. She felt able to follow the school mandated FOSS curriculum and noted easy integration of the new technology into this system. My curriculum is to use the FOSS kits. [There are] a lot of hands on labs with those. So, basically, every time we do anything with data and graphing, we just automatically implement it through the Navigator™. (3050.TIAu06.P23) In addition to implementing the FOSS curriculum with the Navigator™ system, Ms. C used the quick poll and learning check features of the system for classroom practice of state standardized test items. Analysis of district standardized test results revealed that students were unable to correctly interpret graphs. As a result of this finding, a district goal for the year in science was established to have students analyze line graphs every week. Ms. C used the Navigator™ system to comply with this specific learning goal. In addition, Ms. C. kept a data base of state achievement test questions published by her district for teachers to use as review questions. In her daily start-up, she selected one of these questions from the database and used the quick poll feature of Navigator™ to present the question to her students. 10 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… I am on the data team for my district where we really look at all of the data for the [State standardized test] questions….We really noticed in our district…that kids are not analyzing the graphs….We had terrible data as an entire district. So our district goal this year for science, is every week teachers are… having kids graph and analyze lines in graphs….to figure out what they mean….This is what my students are...doing through Navigator™. (3050.TIAu06.P29) In summary, Ms. C was able to integrate the connected classroom technology into the FOSS curriculum selected for use at her school. She perceived good congruence between this new technology and both her science curriculum, her standardized test review practice and district goals. With strong collegial mentoring and acceptance by her students for the new system, Ms. C found the connected classroom technology congruent with her classroom practices. Ms. C - Costs & Benefit Costs. The primary cost cited by Ms. C involved the difficulty of learning to use a new educational technology tool in her classroom. Her previous apprehension regarding computer based technologies and lack of familiarity with graphing calculators, the handheld device on which the Navigator™ system rests, represented potentially significant challenges for classroom integration. Ms. C. reported that initially she was uncertain that she would be able to remember enough from her summer workshop to be successful implementing the new system. I was a little skeptical for awhile, just getting everything. My district had to order me a laptop. And I actually ended up getting the Navigator™ before the laptop was ready. So there was a lot of anticipation. I was excited; then I was worried. You know, if I don’t do this soon, I’m going to forget everything. (3050.TIAu06.P11) Help from her curriculum specialist and other Navigator™ using teachers in her school provided immediate solutions for any problems that she encountered as she attempted to integrate the technology in her teaching. Ms. C recognized the importance of her social situation and mentoring resources. I feel for people…that were in the training, but are maybe the only person in their district. Because, I don’t know what they are doing when they get stuck….I feel like, I can just walk next door and go, hey, I need help. Or the kids can help me. (3050.TIAu06.P57) Benefits. The primary benefits cited by Ms. C from use of connected classroom technology included increased knowledge of student learning through improved formative assessment practices, students learning more about their own learning, and increased student engagement. Each of these benefits is described below. Increased knowledge of student learning. Although Ms. C was an experienced classroom teacher, classroom connectivity technology provided her with information about her students that she previously had not known. A key aspect of this experience was her realization that her ability to observe her students and determine what they were learning was not as keen as she had supposed. Kids, especially by this age level, the seventh or eighth grade, they have learned to play a game. They can know nothing and appear that they know a lot. I think it really 11 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… brought to light for me that I did not have a good sense of what they are learning by looking at them. (3050.TISp07.P90) I have taught for 18 years and I have been in seventh grade science for about 15 of the 18, and there are things that I have always been really sure that I think kids have understood completely. Now I see what they are thinking, and I am like, whoa, I am just amazed. (3050.TISp07.P40) Immediate feedback regarding student learning provided Ms. C with important information to guide classroom decision making. Student misconceptions or misunderstandings were more likely to surface with frequent targeted formative assessment questions. For example, Ms. C learned that her students needed review and re-teaching after a lesson on distance/time graphs using motion detectors. It also allows me to see right away, what things they know and what things they are not catching on to. Like … that one learning check today, I could tell right away that they still cannot tell the difference between a distance graph and a position graph and their meaning. (3050.poi07.P29) I know it helps the students, but sometimes I think it helps me more because I know right away whether I can go on, or whether we need to cover certain concepts of the learning again. (3050.poi07.P33) One thing I noticed … from what I saw with the quick polls, is if the [walker is at the reference point when the time starts] they don’t understand how the graph is going to change….Just because the time starts, the object doesn’t have to start moving….Many of them did not get that. (3050.poi07.P85) Ms. C also found the connected classroom technology beneficial for pinpointing with minimal embarrassment errors made by specific students. She used a learning check question to pre-assess her student’s knowledge of rotation and revolution before teaching an earth systems lesson. The knowledge she gained allowed her to differentiate her instruction to maximize instructional time for her students. We were doing earth and sun relationships … revolution versus rotation….But, we did a learn check, and …I was able to look on [my computer screen] and see who was making those mistakes still. And I was able to grab them. So it helped me because I could pinpoint without embarrassing them. (3050.TIAu06.P47) In another example, Ms. C reported using the connected classroom technology to help a student analyze data in an inquiry learning activity. When class temperature/time data were aggregated and displayed, one data set appeared as an outlier. Using the connected classroom technology, Ms. C identified the student who had submitted the questionable data and helped her diagnose her own error. There was really strange data that showed up once we aggregated the class. …When you put the cursor on the plot…and the data showed up, and [also] the child’s name. So I looked at the girl, and I said, “Do you know what you did? Look at this.” And she said, Oh, I put the temperature in before the time.” And I said, “Yes, and what’s wrong with that?” And she said, “The x information goes first. And I did it backwards”…But from that she learned what she was doing wrong. I would have never caught that if we were just having a classroom discussion. (3050.TIAu06.P33) 12 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… Students learning more about their own learning. In addition to Ms. C gathering and evaluating data to learn more about her students, she reported that connected classroom technology helped her students learn more about their own learning as well as the learning of their classmates. When student responses were displayed anonymously in a public way, every one in the class viewed a variety of student ideas. You know, you just see now exactly what they are thinking. And even they do. They are like, wow, I was the only one who didn’t know that. …I think it is really helpful for them, that immediate feedback. (3050.TISp07.P40) Classroom discussion of wrong answers provided opportunities for Ms. C and her students to explore student thinking about a particular topic. While Ms. C assumed that students would always prefer to remain anonymous, she was surprised to find that many students wanted to have their names revealed, even when their answer was incorrect. I am kind of surprised. I wasn’t quite sure about the showing of the kid’s names. I think the first thing I did, I hid their names, but then something came up, so I had to show it to figure out whose it was that was making the mistake. And the kids don’t seem to mind that…. You know they end up discussing what they see …after they send it. And they compare themselves to other kids. It leads to its own discussion. (3050.TIAu06.P37) When Ms. C compared her connected classroom to her previous teaching style she identified positive changes in the classroom atmosphere and her interactions with her students. She felt her students worked in groups more often and collaborated in their learning tasks. We definitely now are connected now in a way that I have not experienced before. …And the Navigator™ allowed that to occur…. Because the kids are always working in groups; they are always helping each other. …It just has established a completely different environment. (3050.TISp07.P80) Increased student engagement. Ms. C reported that use of the connected classroom technology engaged her students in traditional tasks such as review as well as inquiry learning activities. For example, Ms. C reported that her students requested review with the Navigator™ before quarterly standardized tests taken by all eighth graders in her district. Using the learning check feature of Navigator™™, students could answer multiple choice questions and receive immediate feedback regarding the correctness of their response. I know my eighth graders have a common assessment that we take for the district….All eighth graders take the same test four times a year. We have a big review day coming up ….And, they asked for it. They said, can we review with the Navigator™ and do the learn checks, because we pay more attention to it. (3050.TIAu06.P49) The screen capture feature of the Navigator™ system found utility for Ms. C as a way of increasing on-task behavior in her class. She used the refresh feature frequently to monitor student progress as they worked on calculator tasks. If I send them a learn check or a quick poll, I immediately go to screen capture. And every time…I walk by my computer, I just tap the refresh. It gives me an idea where they are. And actually it works really well to keep them on task because they really buy into the idea that I know what they are doing all the time. (3050.TISp07.P45) 13 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… Ms. C - Summary Overall Ms. C began the study fearful of the technology involved, but determined to persist with her students. Her school structure provided excellent support in both hardware and collegial relationships. Her curriculum supervisor and teaching colleagues were experienced with TI products and many already were using the Navigator™ system in their mathematics instruction. The principal at Ms. C’s school actively supported her participation in this research study. Ms. C demonstrated a high tolerance for risk taking in her classroom teaching and was not discouraged by temporary setbacks. When her school selected the FOSS program, Ms. C received specialized training for the new eighth grade curriculum and found the connected classroom technology highly compatible. Overall, Ms. C decided the benefits in increased formative assessment opportunities, improved student engagement in classroom tasks, targeted data regarding individual student learning, and support for inquiry learning provided by the Navigator™ compensated for the difficulties of learning a new technology system. Case Study 2: Ms D Ms. D: Context Ms. D had 5 years teaching experience in middle school science. She earned an undergraduate degree in animal science in preparation for her teaching career. Ms. D participated in a variety of professional development opportunities in her school community and was a teacher leader at her middle school in Texas. Ms. D was confident of her technology skills as well as her science content knowledge. Ms. D was the only one of the three teachers in this study who reported prior professional development with graphing calculators. The school where Ms. D worked was located inside a major urban center in Texas and housed a diverse student population with 92% of the students either Hispanic or Black and 82% reported as economically disadvantaged. The faculty at her middle school made the decision to separate the male and female students at the school into groups called pods. Each pod was assigned a team of teachers with a teacher leader. Ms. D served as the teacher leader for the seventh grade boys pod. Ms. D reported that the students at her school initially resisted the single sex assignment and demonstrated their displeasure through increased disciplinary problems during the fall semester when our project began. For many of the students in Ms. D’s science class, English was a second language. Classroom instruction occurred only in English, but student-student communication during observations was noted in both English and Spanish. Ms. D’s class sections included about 22-24 students. In the year prior to our study, Ms. D’s school was paired with Texas Instruments (TI) as a community partner school. As a result of this initiative, professionals from TI participated in science and mathematics outreach programs at the school including field trips, scientist visits, mentoring and tutoring, and a science and mathematics fair. Ms. D received training in TI graphing calculators as part of this project and acquired a mentor - another high school teacher located in Arkansas who led some of the teacher workshops that she attended. This mentor informed Ms. D about the CCMS research project and was instrumental in encouraging her to apply to participate in our study. In addition, teachers who participated in the training received free refurbished equipment. During the first year of the project, no other teacher at her school used the Navigator™ system. However, Ms. D successfully recruited two additional teachers at her school to join the project in the second year. The principal at Ms. D’s school was supportive of her participation in our project. 14 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… Ms. D. – Instrumentality Curriculum. Ms. D taught Science 7, a mixture of life science (first semester) and physical science (third quarter) and environmental science (fourth quarter) to the boys in her seventh grade pod. The faculty at Ms. D’s school worked together on teams tracking student progress, but each teacher designed and implemented his/her own unique classroom lessons. Technology resources. At her school, Ms. D had access to an old but functioning class set of TI 73 graphing calculators and a variety of class sets of compatible peripheral devices for making science related measurements including pH probes, voltage readers, and motion detectors. Ms. D borrowed an interactive white board and computer projector from the school media center. While the Navigator™ system does not require the use of an interactive white board, a computer projector is an important device for optimal use of the Navigator™ system. Low demand for this hardware from other teachers in her school allowed Ms. D to house this equipment in her classroom during the first year of our project. Ms. D experienced difficulty ordering her Navigator™ equipment as her district office lost the purchase order documents. She received her system in late November and then experienced difficulty with faulty cables and cable connections. A combination of refurbished graphing calculators, link cord failures and an inadequately configured computer resulted in start-up difficulties using the system. Social support. Although Ms. D’s school enjoyed a school/business collaborative arrangement with TI, no other teacher in her school used the Navigator™ system. Ms. D relied on the support of her mentor in Arkansas through email communication and called the TI support center when she needed help. The school structure at Ms. D’s middle school provided some support for her use of connected classroom technology. The prior relationship with Texas Instruments provided her access to both training and equipment and established a mentorship link with a TI instructor. Ms. D’s strong technology skills and resourcefulness were important elements in her eventual success in installing and using the connected classroom technology in her seventh grade science classroom. Her early struggles with installation and use of the system were evident in her interviews and during her classroom observations. Ms. D – Congruence Ms. D reported that connected classroom technology matched her teaching strategies in many ways. For example, Ms. D started every class with an opening exercise that she called a ‘bell ringer’. After greeting students at the door, Ms. D expected them to gather their calculator equipment and notebooks and be seated in the four-desk tables arranged in her room. The bell ringer questions were distributed through a Navigator™ learning check document and reviewed with the class analysis feature as the opening activity of each day’s lesson. Since Ms. D’s school was struggling to meet state academic standards, she paid close attention to proscribed learning objectives and individual student success on each. And [learning checks] are great because you can… put it by specific objectives and see where you are weak…and who is specifically weak in what area. That just narrows it down to each student by each objective…That is very good in backing up our…objectives for our lesson plans because we have to know exactly who is having problems with what… objectives so you can pinpoint it. (3026.TIAu07.P54) 15 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… In addition to opening exercise and review, Ms. D used connected classroom technology for students to review their individual homework assignments. Using the study guide feature of the technology, Ms. D would instruct her students to enter homework answers into a learning check document to receive instant feedback on their results. She sometimes (but not always) stored the electronic files with student responses. During the classroom observations, Ms. D introduced her students to motion detectors in an inquiry style lesson. Students first determined how their individual actions affected the distance/time line generated on the graphing calculator plot and then attempted to match their motion to reproduce distance/time graphs. Each student was able to work with an individual calculator and motion detector over two class periods. The screen capture component of the connected classroom technology allowed students to view the graphs produced by each member of the class and to recap the motion necessary to produce a particular trace. Ms. D’s school district has been focusing on student learning styles and accountability for learning tasks. Her school and district mentors and colleagues were encouraged to have students discuss how to solve problems and to take more responsibility for their individual learning progress. Ms. D observed that the aims and goals of her district and school were consistent with use of connected classroom technology. While Ms. C and her students (during their focus group interview) reported using the Navigator™ with probes for a variety of inquiry learning activities, the use of Navigator as a frequent, in-class assessment tool was identified by Ms. C as an important advantage of connected classroom technology. Ms. D - Costs/Benefit Costs. The primary cost cited by Ms. D related to equipment management and adapting her lessons to the new technology system. Ms. D felt rushed due to her perceived late acquisition of the connected classroom technology and challenged by old calculators, faulty connecting cords, and a computer that was not quite up to the task of interfacing with the new software. I got a late start. And trying to use it every day is hard. And …equipment management is hard. So once I felt more comfortable, [then] the more I used it of course…. It’s not hard to adapt, but it takes a little time to adapt your lessons. (3026.TISp.P52) Ms. D mentioned the need for additional planning required to use the connected classroom equipment. Due to repeated early equipment failures attributed to her old refurbished calculators and old computer, she routinely planned a back-up lesson in case things didn’t work out the way she intended. Ms. D acknowledged the need to be flexible and persistent in working with the technology. It’s quite a bit of planning before to get set up and ready. And during class…you have to …change your mind a lot. Because even if you may have tested stuff …, there’s [going to] be issues that don’t work. And you may have to…switch gears right in the middle of your lesson ….So, it is [important] to be quick on your feet. (3026.TISp.P75) Benefits Increased knowledge of student learning. Ms. D described her motivation to continue using the connected classroom system despite initial difficulties with her hardware. The primary reason she cited was the increased knowledge she gained regarding her student’s learning. Ms. D was surprised by what her students didn’t know and appreciative of timely and useful feedback to guide her lesson planning. Because I am surprised. I will give them a quiz which I think… they should know this. This is pretty basic stuff. And it comes back, and some kids on one question half of 16 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… them got it wrong. And I’m like whoa, it really opens my eyes and tells me that okay, they didn’t understand this. It really helps me quickly learn that I need to go back over a few things, rather than move on. ‘Cause usually when you give a paper and pencil quiz, and you get it back and you grade it, it’s like a day before you realize they didn’t understand this question at all. It’s timelier on the feedback. (3026.TISp.P67) Ms. D also appreciated the greater sense of individualized communication that the system afforded. She stated that in her connected classroom she could better identify individual student problems and gain important information to guide her instructional choices. It is just a great tool because you can narrow it down by topic and by student and you can really work with that student on what they are having trouble with and you can see if it is… one or two kids or if it is all of your kids. …It is great on…figuring out why they are missing something… or if it is… something you need to re-teach. (3026.TIAu07.P72) You do not know it until you have graded their papers days later… that they are having trouble. …Now you can know sooner and help them sooner rather than having to wait until they bomb a test or turn in their homework 3 days late. (3026.TIAu07.P76) The connected classroom provided Ms. D insights into how her students were learning and caused her to reevaluate her role in the classroom. She related that the device changed the studentteacher relationship in important ways, promoting discussion and encouraging students to take more responsibility for their own learning. With the Navigator™ you see their possibilities and their solutions and reasons why. So it is not a perfect yes or no answer, but now you have room for discussion. …It totally changes the dynamics…of the student-teacher relationship; … It is scary for a lot of teachers to give up that power, but that is where the district and where most schools are going now. They want the kids to have more power in decision-making; and taking charge of their own education. And so I think the Navigator™ helps them to do that. (3026.TIAu07.P82) Increased student engagement. The students in Ms. D’s class responded positively to the connected classroom technology. Ms. D reported that she valued the greater involvement of her students in classroom question-andanswer teaching episodes. The anonymous nature of student responses seemed to encourage students to respond even when they were unsure of their answers. During classroom observations, her students were cooperative and participated enthusiastically in the lessons. That is great because … every kid is involved… They like seeing that they are involved and it is confidential so they do not feel weird about jumping out and saying who they are if they are not sure about it. But …the kids just love it. You know, there have never been any problems. (3026.TIAu07.P56) 17 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… Ms. D -Summary Overall Ms. D experienced support from her principal and school for use of connected classroom technology. During the initial months of use, she struggled to cope with old calculators, unreliable cords, and a less than optimal computer for her system. Despite these initial trials, Ms. D persisted and learned how to incorporate the technology in her classroom teaching. By the time of her classroom observation, her students logged into the system seamlessly, completed learning check assessments without any verbal guidance from Ms. D, and successfully used Navigator to review distance/time graphs collected with motion detector probe ware. Clearly, this teacher was successful in establishing classroom routines that included connected classroom technology. While Ms. D mentioned use of the connected classroom technology for inquiry inspired lessons involving motion detectors and temperature probes, her primary use seemed to be for pre and post classroom assessments and for test or homework review. These traditional uses fit well with her prior teaching practice, her school goals to improve student scores on standardized tests, and required little change in her teaching style. By targeting her assessments to state learning objectives, Ms. D gained specific information regarding student learning gaps. Her surprise at what her students did not know represented an important insight. Ms. D focused primarily on the increased access to timely and targeted information about student learning as the primary benefit of the connected classroom system. As her school struggled to meet state mandates for student achievement, Ms. D identified increased frequency of formative assessment as an important advantage that the system supported. Ms. S - Case Study Ms. S - Context Ms. S was in her third year of teaching, but was a new teacher at her school. Her teacher preparation included undergraduate training with a degree in secondary science education. The high school where Ms. S taught physical science and chemistry was located in a suburban fringe of a large city in Ohio. The school population was primarily White (92%) with few students classified as economically disadvantaged (8%). Ms. S was assigned to a renovated room that had previously served as a home economics classroom. She had science tables for her students and sinks along one wall of the room. Her class sizes were the largest of the three teachers in this study with 26 and 30 students in the two groups enrolled in the project. Because this was Ms. S’s first year at this school, she was engaged in locating the necessary resources needed to teach science. Ms. S reported relatively high confidence in both her science content knowledge and her knowledge and experience with educational technology. She described previous use of probe ware in chemistry teaching and was familiar with the graphing calculator prior to joining the project. Ms. S – Instrumentality Curriculum. Since this was the first year that Ms. S taught physical science, she was creating new lessons each day. The physical science curriculum that Ms. S taught included one semester of introductory physics and one semester of introductory chemistry. During the first few months of use, Ms. S and her colleagues used their Navigator™ systems to pre-assess all ninth grade students in the building with a state standardized test. They gathered as a department to analyze student results, reviewing questions and using that information to inform their teaching decisions during the fall semester. We actually just did pre-[state standardized test], and so I had all of my freshman do it…I had them put their answers in learn check and then I put all four class results onto one class analysis, so that we could get a breakdown of each question for all four 18 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… classes combined. So that was neat. And actually we had the other teachers in the building do it, also, using Navigator™, and then we just combined all the teachers’ results onto one learn check, or one class analysis. (3062.TI.Au06.P10) Supporting technology resources. Although some students owned their own calculators, and Ms. S started with a partial class set, her relatively large class sizes (26-30 students) left her slightly short of sufficient equipment for each student to have an individual calculator. However, by second semester, Ms. S had sufficient calculators to allow each student to use their own equipment during her classes. In addition, Ms. S. enjoyed an interactive white board and dedicated computer projector in her classroom. During the first few months of the project, Ms. S struggled to arrange furniture, technology resources, and cords to connect all the equipment and allow her to move through her classroom as she wished. In her early interviews, Ms. S noted that she felt tethered to her computer screen and inhibited in her movements. By the end of the first semester, she was successful in procuring the necessary cables and supporting equipment to use the Navigator™ system. Social support. At least three mathematics teachers and one other science teacher at Ms. S’s high school had Navigator™ systems in their classrooms and used them regularly. Two of the mathematics teachers participated in the algebra 1 part of the CCMS field trial. Ms. S’s neighboring science teaching colleague was a TI instructor and taught the summer professional development week-long course that science teacher participants in the study attended. This colleague recruited her to join the project and served as an important mentor. The principal at this high school supported innovative technology uses and attended the Teachers Teaching with Technology conference during the year prior to Ms. S’s involvement in our project. This school environment included not only significant technology resources, but also a culture of technology innovation and experimentation. Ms. S - Congruence Although Ms. S had not participated in any graphing calculator professional development courses prior to participating in our project, she felt comfortable with both the graphing calculator and probe ware and had used both in her previous chemistry classrooms. In addition, Ms. S felt confident in her other technology skills. Ms. S initially worked diligently to set up her classroom to accommodate the new technology system and to establish routines with her freshmen students. Ms. S found connected classroom technology easier to integrate into content areas where she felt well informed. Since she was teaching physical science for the first time, some units required more background reading and preparation than others. In her interviews, she reported that when she had previously taught a topic and had existing lesson plans, she was more likely to make the effort to integrate the connected classroom technology. For topics that were new to her teaching repertoire, all of her pre-lesson energies were spent refreshing her knowledge or learning the material. Planned formative assessment represented an easy way for Ms. S to use the connected classroom technology. She used learning check for both graded and un-graded quizzes. After she mastered the class analysis feature of this component, she reported using the technology to collect quiz scores. Ms. S also used the screen capture feature of the connected classroom technology to facilitate data gathering during a lesson on acids and bases. Students used pH paper from a variety of sources and with a variety of pH ranges and as a result gathered similar but not identical pH data on a selection of household projects. Students could quickly see the data collected by others in the classroom using the screen capture feature. Discussion about accuracy and precision, multiple measurements, and averaging of class data demonstrated for students important aspects of scientific inquiry during this lesson. 19 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… Ms. S – Costs & Benefits Costs. For Ms. S the primary costs related to set-up of her classroom and establishing workable classroom routines with the connected classroom technology. With a supportive school environment and many colleagues to serve as mentors, she reported few difficulties with the system. Rather, she focused in her discussions and interviews on placement of the various technology components within her new classroom, classroom routines, students logging on, and managing the intricacies of transitioning from the calculator functions to Navigator™ functions and back. These initial costs seemed to be overcome by the time of her spring telephone interview. Ms. S observed that her perception of the difficulty of using the system decreased as she continued to gain experience and that her improving knowledge of the system facilitated her continued use. Indirectly Ms. S hinted at the time cost in using a new technology when she mentioned that she was more likely to use the system with units where she had prior teaching experience. When she planned for a topic that was new to her, she focused her energies on mastering the content and was less inclined to use the connected classroom technology. Benefits. Ms. S found connected classroom technology helped her learn more about her student’s learning. Early in her experiences she asked her students to take a practice state standardized test and used the class analysis feature of the Navigator™ technology to rapidly analyze their data. With her colleagues, who also tested their students, her department reviewed their ninth graders’ strengths and weaknesses in the first few months of the school year. With targeted information early in the school year, these teachers could implement teaching strategies to narrow the gaps between their state science objectives and their students’ performance. Ms. S identified rapid feedback as an important aspect of the technology. In addition to the convenience of rapid scoring of in-class quizzes, Ms. S mentioned that students were able to track their progress and ask questions before they had forgotten the lesson. Ms. S. also noted high levels of student engagement during the slide show presentation of student results that accompanies class analysis. Students were able to notice others who made similar errors. So from the teacher perspective, the grading is done in a heartbeat. And from the student perspective, I feel like if they can get that instant feedback, instead of a couple days later, when they don't remember what they chose anyway, I feel like this way they really understand it a lot better. And they can ask questions right away instead of [after] that time lapse where they forget everything. (3064.poi07.P73) Taking a quiz for them on Navigator™ is very simple. And then I get their grades instantly, so usually before they leave class that day I've had the answers up on the board and we've gone over it. So they get that instant feedback, they know what they did wrong or right, right away. (3064.poi07.P49) Ms. S used the connected classroom system to determine how successful students were on short assessments related to daily lessons. She indicated that with more frequent assessments, students asked more questions. She found the screen capture refresh feature helpful in checking student progress as well as for data display during inquiry lessons. In her spring telephone interview Ms. S stated that the Navigator™ helped students focus more on the science content and pay attention better. She identified the public display of knowledge as an important benefit for both her students and herself. Class discussion of open-ended responses helped her probe student understanding. Ms. S - Summary Overall Ms. S enjoyed a very supportive school culture for technology innovation. The administration and teachers at her school were already using Navigator™ when she joined our project. 20 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… Mentoring and troubleshooting was a matter of walking next door to speak with an accomplished Navigator™ user. While her class sizes were larger than either of the other teachers in this study, Ms. S’s students were economically advantaged and already scoring well on state standardized tests. Ms. S’s technology know-how helped her negotiate the initial start-up issues with the connected classroom system. After establishing new routines and re-organizing her room to accommodate the new furniture, she quickly began exploring use of the system for lessons where she felt particularly confident of the science topic. Ms. S identified frequent, rapid and public assessment and review as important benefits of the connected classroom technology. She observed that students were engaged in science tasks and asked more questions when they received rapid feedback on short in-class assessments. She immediately recognized and took advantage of the connected classroom technology to survey all ninth grade students to provide data for department review of student pre-achievement levels as a way to maximize scores on state standardized achievement measures. Physical Science Achievement Comparisons of student achievement for the students of our case study teachers must be made cautiously while remembering that Ms. D taught seventh graders most of whom spoke English as a second language, Ms. C taught low achieving eighth graders and Ms. S taught ‘regular’ ninth graders at schools with vastly different community wealth and resources. Only Ms. S had a science curriculum that included one semester of chemistry and a second semester of physics topics. Ms. D and Ms. C taught a wider range of science topics including some life and environmental science. Table 4 reports the pre and post achievement for each of these teachers. For comparison, we are reporting the post test data from the schools that participated in our pilot of the physical science assessment instrument. The population that participated in the pilot consisted of students from both suburban and urban schools and created a bimodal achievement profile, with suburban schools scoring above the urban students. The value reported for the pilot in Table 4 represents the mean of both groups. These data do not represent results of a randomized trial as the algebra 1 test results do and are presented here to give the reader a sense of the achievement levels of the students whose teachers are profiled in this article. Table 4. Student physical science achievement Test N Mean (SD)* Pilot Post 269 25.4 (8.1) Ms. C (8th grade) Pre 54 17.2 (4.4) Post 48 20.1 (5.5) Pre 23 17.3 (4.4) Post 24 19.4 (4.7) Pre 44 22.9 (5.9) Post 48 25.7 (6.3) Ms. D (7th grade) Ms. S (9th grade) * 42 maximum score Interestingly, Ms. C’s low achieving eighth graders and Ms. D’s low achieving seventh graders show very similar achievement patterns with this instrument. Ms. S’s pre scores are much higher, reflecting perhaps the affluence of the school community where she works as well as their ninth grade 21 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… school level. Post scores for Ms. S’s students are similar to the pilot data results. While differences in the school curriculum, age level and socio-economic levels of the schools are significant, students in all three classes showed gains in their mean achievement levels. Conclusions & Discussion Despite the wide differences in school contexts, the three teachers in this project were all successful at integration of connected classroom technology in their science teaching. Characteristics of these schools that may have contributed to this success include supportive collegial and structural supports in their school culture. Administrators in each of these three schools welcomed innovative educational technology and attempted to provide the necessary peripheral devices to support their teachers. Each of these three case study teachers eventually gathered a class set of compatible graphing calculators, a dedicated projector for their computer, a compatible computer system and an interactive white board. Ms. D, located at the most economically disadvantaged school and as a pioneer Navigator™ user in her school community, experienced the most difficulty with the technology due to her refurbished calculators and older computer. In addition, her mentor was not in her building forcing her to troubleshoot on her own. Ms. S and Ms. C both joined teams of connected classroom teachers in their school buildings. They enjoyed the social support of their colleagues and received quick answers to their technical problems. Two of the three teachers explicitly reported that the connected classroom technology provided them with information about their students that surprised them. The most veteran teacher remarked that she was amazed that her ability to informally assess her students by observing their classroom behavior to know whether they were learning did not provide her with accurate information. Teacher selective notice of some students, but not all, is a formative assessment practice identified by Bell and Cowie (2001). This ability to read students by viewing their body language or by listening to the responses of a few vocal students may provide teachers with inadequate and flawed data on which to base decisions about classroom instruction. The connected classroom provides responses from more students (often every student logged in will respond), providing the teacher with a more accurate reading of student understanding from a greater number of their students. Of the three participants in this study, Ms. C most clearly articulated the learning gains that she made as a result of integrating the connected classroom technology in her teaching. She also described specific plans to re-teach, rephrase explanations, offer individualized instruction, and to include other teachers on her team to help close the learning gaps she discovered for her students. All three teachers remarked that not only did they benefit from learning more about their students, but their students also learned more about their own and other’s learning in the class. Public display of anonymous answers promoted class discussion and revealed patterns in student wrong answers. Students asked more questions and received feedback about their learning soon after completing short assessments. Just-in-time information was viewed as powerful for both students and teachers in these classrooms. All three case study teachers observed increased student engagement and greater time on task in connected classrooms. They attributed this behavior to student liking of the new technology (perhaps an artifact of newness) as well as the opportunity for every student to respond to teacher probes. Students liked seeing their responses, often demanded to see their names even when their responses were incorrect, and participated actively in connected classroom activities. In addition, all three teachers appreciated the ability to use the screen capture component of the connected classroom system to track individual student work. This feature not only allowed them to view each student’s screen, but also provided them a way to monitor on and off task behavior. Two of the three teachers in this study were teaching physical science for the first time. They did not need to modify previous lessons to accommodate the Navigator™ technology, but rather designed their initial lessons integrating the connected classroom system. All three teachers used the 22 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… data collection, aggregation, and display features of the system to conduct inquiry lessons with their students. However, the more frequent use of the connected classroom was for daily pre and post assessments (planned formative assessment), homework checks, and spur of the moment questions related to lessons (interactive formative assessment) (Bell & Cowie, 2001). Implications and Future Work In this study, the connected classroom technology was sufficiently congruent with teachers’ practice, skills and values to promote implementation in their classrooms. From the perspective of Archer’s social theory (Archer, 1988), initially teachers adapted the new technology to their prior use patterns. The flexibility of the system allowed teachers to implement the connected classroom technology slowly, building confidence and testing its limits and capabilities. As their proficiency developed, teachers valued the new information they were able to access regarding student learning. Teachers particularly reported learning about the variety of student ideas and alternative conceptions, using planned and interactive formative assessment probes, and surprise at knowledge gaps revealed by students. In addition, the teachers used the connected classroom technology to support inquiry science activities with pH probes, temperature probes, and motion detectors. The data gathered by the connected classroom technology seemed to be more comprehensive and targeted at particular learning goals and individual students than the data teachers previously gathered based on their informal impressions of how students were receiving a particular lesson. Practical difficulties with gathering, aggregating and interpreting data in a timely manner under real classroom conditions contributes to the difficulty teachers experience implementing formative assessment practice (Assessment Reform Group, 1999; Black & Wiliam, 1998a, 1998b; Daws & Singh, 1996). In this study, rapid response technology-facilitated formative assessment appeared to reduce some of the barriers to the practice of formative assessment, informing teachers about their students as well as informing students about their own and other’s learning. Products of this research promise to aid in science conceptual development by improving teacher and student classroom formative assessment, classroom discourse, self-regulated learning, especially in the quality of questions asked, class discussion, and to expose and correct student alternative understandings. Technology-facilitated formative assessment represents a rich and unique resource for improving teacher practice in science classrooms. The case studies presented in this paper provide new knowledge regarding how school culture supports physical science teacher initial use of connected classroom technology. In addition, it reveals that each of these teachers persisted due to their perception of the benefits of increased understanding of their students, timely and targeted information regarding student learning, increased student engagement and involvement in classroom tasks, and easy adaptation of the technology to both inquiry lessons and more traditional assessment tasks. Detailed analysis of teacher formative assessment practice will reveal how technology-facilitated formative assessment contributes to teacher classroom decision making. References Abrahamson, L., Davidian, A., & Lippai, A. (2002, November 16-19). Wireless calculator networks Why they work, where they came from, and where they're going. Paper presented at the 13Pth PAnnual International Conference on Technology in Collegiate Mathematics, Atlanta, Georgia. Assessment Reform Group (1999). Assessment for learning: Beyond the black box. Cambridge: University of Cambridge, School of Education. Archer, M. (1988). Culture and agency: The place of culture in social theory. Cambridge: Cambridge University Press. Bell, B. & Cowie, B. (2001). Formative assessment and science education. In K. Tobin (Ed.) Science & technology education library (Vol. 12). Dordrecht: Kluwer Academic Publishers. 23 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… Black, P., & Wiliam, D. (1998a). Assessment and classroom learning. Assessment in Education, 5(1), 7-74. Black, P., & Wiliam, D. (1998b). Inside the black box: Raising standards through classroom assessment. London: King's College London. Cheung, D. (2006). TCSS: A new computer system for developing formative assessments. School Science Review, 88(322), 61-70. Council of Chief State School Officers. (2000). State Educational Assessment Center, State student assessment programs database, 1997-1998 school year. Available: http://nces.ed.gov Cowie, B. & Bell, B. (1999). A model of formative assessment in science education. Assessment in Education, 6(1), 101-116. Cuban, L. (1998) How schools change reforms: Redefining reform success and failure. Teachers College Record, 99(3), 453-477. Cuban, L. (2001). Oversold and underused: Computers in the classroom. Cambridge, MA: Harvard University Press. Cuban, L., Kirkpatrick, H., & Peck, C. (2001). High access and low use of technologies in high school classrooms: Explaining an apparent paradox. American Educational Research Journal, 38(4), 813-834. Daws, N. & Singh, B. (1996). Formative assessment: To what extent is its potential to enhance pupils’ science being realized? School Science Review, 77(281), 93-100. Daws N, Singh B (1999) Formative assessment strategies in secondary science. School Science Review 80: 71–78. Doyle, W. & Ponder, G.A. (1977) The practicality ethic in teacher decision-making, Interchange, 8(1), 1-12. Dufresne, R. J., Gerace, W. J., Leonard, W. J., Mestre, J. P., & Wenk, L. (1996). Using the Classtalk classroom communication system for promoting active learning in large lectures. Journal of Computing in Higher Education, 7, 3-47. Fies, C. & Marshall, J. (2006). Classroom response systems: A review of the literature. Journal of Science Education and Technology, 15(1), 101-109. Fuchs, L. S., & Fuchs, D. (1986). Effects of systematic formative evaluation: A meta-analysis. Exceptional Children, 53(3), 199-208. Keeley, P., Eberle, F. & Farrin, L. (2007). Uncovering student ideas in science: 25 Formative assessment probes (Volume 1). Arlington, VA: NSTA Press. Miles, M. B., & Huberman, A. M. (1994). An expanded sourcebook: Qualitative data analysis, 2nd Ed. Thousand Oaks: Sage. Owens, D. T., Demana , F., Abrahamson, A. L., Meagher, M., & Herman, M. (2004). Developing Pedagogy for Wireless Handheld computer Networks and Researching Teacher Professional Development (National Science Foundation Grant Report by The Ohio State Univ. Res. Fdtn. & Better Education Inc. No. ESI 01-23391 & ESI 01-23284). Washington, DC: Published by ERIC ED479499. Pape, S. J., Irving, K. E., Owens, D. T., Boscardin, C. K., Sanalan, V. A., Abrahamson, L., Kaya, S., & Shin, H. S. (in progress). The impact of classroom connectivity in promoting algebra I achievement: Results of a randomized control trial. Patton, M. Q. (1990). Qualitative evaluation and research methods, (2nd ed.). Newbury Park: Sage Publications. Priestley, M. (2005). Making the most of the Curriculum Review: Some reflections on supporting and sustaining change in schools. Scottish Educational Review, 37(1), 29-38. Priestley, M. & Sime, D. (2005). Formative assessment for all: A whole-school approach to pedagogic change. The Curriculum Journal, 16(4), 475-492. 24 of 25 Draft 2008.01.03 Technology-facilitated formative assessment… Roschelle, J., Penuel, W. R., & Abrahamson, L. (2004). The networked classroom. Educational Leadership, 61(5), 50-54. Ruiz-Primo, M. A. & Furtak, E. M. (2007). Exploring teachers’ informal formative assessment practices and students’ understanding in the context of scientific inquiry. Journal of Research in Science Teaching, 44(1). 57-84. Sanalan, V. A., Irving, K. E, Pape, S. J. & Owens, D. T. (2008, January). Classroom communication technology in science: A multi-faceted professional development program. Paper presented at the annual meeting of the Association for Science Teacher Education, St. Louis, MO. Torrance, H. & Pryor, J. (2001). Developing formative assessment in the classroom: Using action research to explore and modify theory. British Educational Research Journal, 27(5), 615-631. Yin, R. K. (2003). Case study research: Applied social science research methods series (3rd ed., Vol. 5). Thousand Oaks: Sage Publications. 25 of 25