Abstract
This study was designed as a confirmatory study of work on productive failure (Kapur, Cognition and Instruction, 26(3), 379–424, 2008). N = 177, 11th-grade science students were randomly assigned to solve either well- or ill-structured problems in a computer-supported collaborative learning (CSCL) environment without the provision of any external support structures or scaffolds. After group problem solving, all students individually solved well-structured problems followed by ill-structured problems. Compared to groups who solved well-structured problems, groups who solved ill-structured problems expectedly struggled with defining, analyzing, and solving the problems. However, despite failing in their collaborative problem-solving efforts, these students outperformed their counterparts from the well-structured condition on the individual near and far transfer measures subsequently, thereby confirming the productive failure hypothesis. Building on the previous study, additional analyses revealed that neither preexisting differences in prior knowledge nor the variation in group outcomes (quality of solutions produced) seemed to have had any significant effect on individual near and far transfer measures, lending support to the idea that it was the nature of the collaborative process that explained productive failure.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
As a rule of thumb, partial η2 = .01 is considered a small, .06 medium, and .14 a large effect size (Cohen, 1977).
The software program Multiple Episode Protocol Analysis (MEPA) developed by Dr. Gijsbert Erkens was used for carrying out the LSA. See http://edugate.fss.uu.nl/mepa/index.htm.
It is important to note that in the initial study (Kapur, 2008), LSA analysis was triangulated through an interactional analysis of discussion excerpts explaining the various transitions and feedback loops.
References
Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1), 1–14.
Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences, 12(3), 307–359.
Bakeman, R., & Gottman, J. M. (1997). Observing interaction: An introduction to sequential analysis. New York: Cambridge University Press.
Bielaczyc, K. (2006). Designing social infrastructure: Critical issues in creating learning environments with technology. The Journal of the Learning Sciences, 15(3), 301–329.
Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: A simple proposal with multiple implications. In A. Iran-Nejad, & P. D. Pearson (Eds.), Review of research in education, 24 (pp. 61–101). Washington, DC: American Educational Research Association.
Bromme, R., Hesse, F. W., & Spada, H. (2005). Barriers and biases in computer-mediated knowledge communication-and how they may be overcome. New York, NY: Springer.
Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32–42.
Chatterji, M. (2003). Designing and using tools for educational assessment. Boston: Allyn & Bacon.
Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. The Journal of the Learning Sciences, 6(3), 271–315.
Chi, M. T. H., Feltovich, P., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121–152.
Cho, K. L., & Jonassen, D. H. (2002). The effects of argumentation scaffolds on argumentation and problem solving. Educational Technology, Research and Development, 50(3), 5–22.
Cohen, J. (1977). Statistical power analysis for the behavioral sciences. New York: Academic Press.
Collins, H. (1985). Changing order. London: Sage.
Cronbach, L. J., & Snow, R. E. (1977). Aptitudes and instructional methods: A handbook for research on interactions. New York: Irvington.
de Groot, A. D. (1965). Thought and choice in chess. The Hague, NL: Mouton.
Dillenbourg, P. (2002). Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61–91). Heerlen, NL: Open Universiteit Nederland.
Dillenbourg, P., & Jermann, P. (2007). Designing integrative scripts. In F. Fischer, I. Kollar, H. Mandl, & J. M. Haake (Eds.), Scripting computer-supported collaborative learning (pp. 275–302). New York, NY: Springer.
Erkens, G., Kanselaar, G., Prangsma, M., & Jaspers, J. (2003). Computer support for collaborative and argumentative writing. In E. De Corte, L. Verschaffel, N. Entwistle, & J. van Merrienboer (Eds.), Powerful learning environments: Unravelling basic components and dimensions (pp. 157–176). Amsterdam: Elsevier Science.
Ertl, B., Kopp, B., & Mandl, H. (2007). Supporting collaborative learning in videoconferencing using collaboration scripts and content schemes. In F. Fischer, I. Kollar, H. Mandl, & J. M. Haake (Eds.), Scripting computer-supported collaborative learning (pp. 213–236). New York, NY: Springer.
Fischer, F., Kollar, I., Mandl, H., & Haake, J. (2007). Perspectives on collaboration scripts. In F. Fischer, H. Mandl, J. Haake, & I. Kollar (Eds.), Scripting computer-supported collaborative learning (pp. 1–10). New York, NY: Springer.
Fischer, F., & Mandl, H. (2005). Knowledge convergence in computer-supported collaborative learning: The role of external representation tools. The Journal of the Learning Sciences, 14(3), 405–441.
Ge, X., & Land, S. M. (2003). Scaffolding students’ problem-solving processes in an ill-structured task using question prompts and peer interactions. Educational Technology, Research and Development, 51(1), 21–38.
Giles, J. (2006). The trouble with replication. Nature, 442, 344–347.
Goel, V., & Pirolli, P. (1992). The structure of design problem spaces. Cognitive Science, 16, 395–429.
Hatano, G., & Inagaki, K. (1986). Two courses of expertise. In H. Stevenson, H. Azuma, & K. Hakuta (Eds.), Child Development and Education in Japan (pp. 262–272). New York: Freeman.
Hewitt, J. (2005). Towards an understanding of how threads die in asynchronous computer conferences. The Journal of the Learning Sciences, 14(4), 567–589.
Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16(3), 235–266.
Holland, J. H. (1995). Hidden order: How adaptation builds complexity. New York: Addison-Wesley.
Jonassen, D. H. (2000). Towards a design theory of problem solving. Educational Technology, Research and Development, 48(4), 63–85.
Jonassen, D. H., & Kwon, H. I. (2001). Communication patterns in computer-mediated vs. face-to-face group problem solving. Educational Technology, Research and Development, 49(1), 35–52.
Kapur, M. (2008). Productive failure. Cognition and Instruction, 26(3), 379–424.
Kapur, M., Dickson, L., & Toh, P. Y. (2008). Productive failure in mathematical problem solving. In B. C. Love, K. McRae, & V. M. Sloutsky (Eds.), Proceedings of the 30th Annual Conference of the Cognitive Science Society (pp. 1717–1722). Austin, TX: Cognitive Science Society.
Kapur, M., & Kinzer, C. (2007). The effect of problem type on collaborative problem solving in a synchronous computer-mediated environment. Educational Technology, Research and Development, 55(5), 439–459.
Kapur, M., Voiklis, J., & Kinzer, C. (2007). Sensitivities to early exchange in synchronous computer-supported collaborative learning (CSCL) groups. Computers and Education, 51, 54–66.
Kauffman, S. (1995). At home in the universe. New York: Oxford University Press.
King, A. (2007). Scripting collaborative learning processes: A cognitive perspective. In F. Fischer, I. Kollar, H. Mandl, & J. M. Haake (Eds.), Scripting computer-supported collaborative learning (pp. 13–38). New York, NY: Springer.
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86.
Kobbe, L., Weinberger, A., Dillenbourg, P., Harrer, A., Hamalainen, R., Hakkinen, P., & Fischer, F. (2007). Specifying computer-supported collaboration scripts. International Journal of Computer-Supported Collaborative Learning, 2(2–3), 211–224.
Kyllonen, P. C., & Lajoie, S. P. (2003). Reassessing aptitude: Introduction to a special issue in honor of Richard E. Snow. Educational Psychologist, 38, 79–83.
Lin, X., Hmelo, C., Kinzer, C., & Secules, T. J. (1999). Designing technology to support reflection. Educational Technology, Research and Development, 47(3), 43–62.
Lund, K., Molinari, G., Sejourne, A., & Baker, M. (2007). How do argumentation diagrams compare when students pairs use them as a means for debate or as a tool for representing debate? International Journal of Computer-Supported Collaborative Learning, 2(2–3), 273–296.
Marton, F. (2007). Sameness and difference in transfer. The Journal of the Learning Sciences, 15(4), 499–535.
McNamara, D. S. (2001). Reading both high-coherence and low-coherence texts: Effects of text sequence and prior knowledge. Canadian Journal of Experimental Psychology, 55(1), 51–62.
McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14(1), 1–43.
Mestre, J. P. (2005). Transfer of learning from a modern multidisciplinary perspective. Greenwich, CT: Information Age.
Mirza, N. M., Tartas, V., Perret-Clermont, A., & de Pietro, J. (2007). Using graphical tools in a phased activity for enhancing dialogical skills: An example with Digalo. International Journal of Computer-Supported Collaborative Learning (ijCSCL), 2(2–3), 247–272.
Petroski, H. (2006). Success through failure: The paradox of design. Princeton, NJ: Princeton University Press.
Poole, M. S., & Holmes, M. E. (1995). Decision development in computer-assisted group decision making. Human Communications Research, 22(1), 90–127.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical Linear Models. Thousand Oaks, CA: Sage Publications.
Reiser, B. J. (2004). Scaffolding complex learning: The mechanisms of structuring and problematizing student work. The Journal of the Learning Sciences, 13(3), 423–451.
Rummel, N., & Spada, H. (2007). Can people learn in computer-mediated collaboration by following a script? In F. Fischer, I. Kollar, H. Mandl, & J. M. Haake (Eds.), Scripting computer-supported collaborative learning (pp. 39–56). New York, NY: Springer.
Sandberg, I. (1994). Human competence at work: An interpretative approach. Göteborg, Sweden: BAS.
Sandoval, W. A., & Millwood, K. A. (2005). The quality of students’ use of evidence in written scientific explanations. Cognition and Instruction, 23(1), 23–55.
Scardamalia, M., & Bereiter, C. (2003). Knowledge building. In J. W. Guthrie (Ed.), Encyclopedia of Education. New York, NY: Macmillan Reference.
Schellens, T., Van Keer, H., De Wever, B., & Valcke, M. (2007). Scripting by assigning roles: Does it improve knowledge construction in asynchronous discussion groups? International Journal of Computer-Supported Collaborative Learning (ijCSCL), 2(2–3), 225–246.
Schwartz, D. L. (1995). The emergence of abstract dyad representations in dyad problem solving. The Journal of the Learning Sciences, 4(3), 321–354.
Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16(4), 475–522.
Schwartz, D. L., Bransford, J. D., & Sears, D. (2005). Efficiency and innovation in transfer. In J. P. Mestre (Ed.), Transfer of learning from a modern multidisciplinary perspective (pp. 1–52). Greenwich, CT: Information Age Publishing.
Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22(2), 129–184.
Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis. London: Sage Publications.
Spiro, R. J., Feltovich, R. P., Jacobson, M. J., & Coulson, R. L. (1992). Cognitive flexibility, constructivism, and hypertext. In T. M. Duffy, & D. H. Jonassen (Eds.), Constructivism and the technology of instruction: A conversation (pp. 1–5). Hillsdale, NJ : Erlbaum.
Stahl, G. (2005). Group cognition in computer-assisted collaborative learning. Journal of Computer Assisted Learning, 21, 79–90.
Stahl, G. (2007). Scripting group cognition. In F. Fischer, I. Kollar, H. Mandl, & J. M. Haake (Eds.), Scripting computer-supported collaborative learning (pp. 327–336). New York, NY: Springer.
Suthers, D. D. (2006). Technology affordances for intersubjective meaning making: A research agenda for CSCL. International Journal of Computer-Supported Collaborative Learning (ijCSCL), 1(3), 315–337.
Suthers, D. D., & Hundhausen, C. (2003). An empirical study of the effects of representational guidance on collaborative learning. The Journal of the Learning Sciences, 12(2), 183–219.
VanLehn, K. (1999). Rule learning events in the acquisition of a complex skill: An evaluation of cascade. The Journal of the Learning Sciences, 8(1), 71–125.
VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. B. (2003). Why do only some events cause learning during human tutoring? Cognition and Instruction, 21(3), 209–249.
Voss, J. F. (1988). Problem solving and reasoning in ill-structured domains. In C. Antaki (Ed.), Analyzing everyday explanation: A casebook of methods pp. 74–93. London: Sage Publications.
Voss, J. F. (2005). Toulmin’s model and the solving of ill-structured problems. Argumentation, 19, 321–329.
Wampold, B. E. (1992). The intensive examination of social interaction. In T.R. Kratochwill, & J.R. Levin (Eds.), Single-case research design and analysis: New directions for psychology and education (pp. 93–131). Hillsdale, NJ: Erlbaum.
Weinberger, A., Stegmann, K., Fischer, F., & Mandl, H. (2007). Scripting argumentative knowledge construction in computer-supported learning environments. In F. Fischer, I. Kollar, H. Mandl, & J. M. Haake (Eds.), Scripting computer-supported collaborative learning (pp. 191–212). New York, NY: Springer.
Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry and Allied Disciplines, 17, 89–100.
Acknowledgements
The research reported in this paper was funded in part by the Spencer Research Training Grant and the Education Policy Research Fellowship from Teachers College, Columbia University to the first author. The authors would like to thank the students, teachers, and principals of the participating schools for their support for this project. We are also grateful to David Hung, Donald J. Cunningham, Katerine Bielaczyc, Katherine Anderson, Liam Rourke, Michael Jacobson, Rebecca Mancy, Rogers Hall, Sarah Davis, Steven Zuiker, and John Voiklis for their insightful comments and suggestions.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix A–Sample Items from the 25-item MCQ Pretest
Two cars having different weights are traveling on a level surface with different but constant velocities. Within the same distance, greater force will always be required to stop the car with the greater
(A) weight (B) velocity
(C) kinetic energy (D) momentum
A 5 kg block is resting on a rough horizontal plane. The coefficient of friction between the block and the plane is 0.8. A 50 N force parallel to the plane is applied on the block for 10 s and then removed. The block eventually comes to a stop. Assuming g = 10 ms−2 and that the coefficient of friction does not change, the total distance traveled by the block equals
(A) 20 m (B) 25 m
(C) 100 m (D) 125 m
A car starts moving from rest in a straight line with a constant acceleration of 5 ms−2, then at constant velocity, and finally decelerating at the rate of 5 ms−2 before coming to a stop. If the total time of motion equals 5 s and the average speed for the entire motion equals 4 ms−1, how long does the car move at constant velocity?
(A) 1 s (B) 2 s
(C) 3 s (D) 4 s
Appendix B–Collaborative Phase Problem Scenarios
Ill-structured Problem 1
You have recently been hired as a lawyer for a prestigious law firm. On your first day, you are sent to meet with an important client who has been fined for speeding. Opening your work file, you find your assignment:
Dear new lawyer,
This morning, I received a call from Mr. Gupta asking me for help. According to him, he almost ran over a small boy this morning in downtown Ghaziabad and was fined for speeding. He insists that he was not. He says that the boy suddenly ran on to the road and he braked very hard and managed to avoid an accident. However, this was enough for a policeman who happened to be there to fine him Rs. 20,000 for speeding. Mr. Gupta is a very important client of our firm and we must do our best to help him. I trust you will give this case your best effort. I am attaching his file for your reference.
I am meeting with Mr. Gupta later this evening. So, I need you to investigate this case and submit your report to me with your analyses and recommendation by today.
Sincerely,Nitin Sharma
Senior Partner
PS–Please note that the word of law is very clear on this. A person is speeding if and only if he is driving above the legal speed limit of the road. No exceptions.
CLIENT FILE
Name: Mr. Amit Gupta
Age: 52 years
Driving Experience: 34 years
Prior Traffic Violations: 1981 (Fined for speeding, Rs. 500),
1993 (Fined for drunk driving, Rs. 10,000)
To carry out your investigation, you go through a number of steps such as a) interviewing an eyewitness, b) analyzing the incident report filed by traffic police, c) accessing the medical examination reports, and d) interviewing the mechanic who inspected the car after the incident.
EYEWITNESS’ ACCOUNT
“I was walking on the roadside pavement. I don’t recall the traffic on the road to be particularly heavy. Suddenly, I noticed a small boy run out on to the road chasing a cricket ball. The next thing I heard was a loud screeching sound. I realized that it came from an Ambassador car skidding to a stop in order to avoid running the boy over. The boy was very lucky to have escaped any injury. I think the boy took about 3 s to cross the road, but I don’t think he looked at the traffic before crossing the road. He was just chasing the ball!”
TRAFFIC POLICE INCIDENT REPORT
-
Traffic conditions: Normal
-
Weather conditions: Bright and sunny; dry road
-
No evidence of a collision between the car and the boy.
-
Number of passengers in the car besides the driver: None
-
Evidence of skid marks: about 15 m
-
Speed limit on the road: 55kmph
-
Width of the road: about 4.5 m
MEDICAL EXAMINATION REPORT
General Comments:
Neither the driver nor the boy sustained any physical injury whatsoever.
Results of the car driver’s medical tests
-
BP (Blood Pressure) = 110/80
-
HR (Heart Rate) = 80
-
Weight = 75 kg
-
Reaction Time = 0.8 s on an average
-
Drug/Alcohol Screen = Negative
INTERVIEW WITH THE MECHANIC
You: What can you say about the condition of the car from your inspection?
Mechanic: Well, this is a heavy car weighing about 1,570 kg and I can clearly see some wear and tear of the tires and the braking system. The braking fluid is also running out. As a result, the traction between the tires and the road does not seem to be as good as it can be.
You: Oh! Does this mean the car was not maintained properly?
Mechanic: Not really. You see, the traction also depends on the condition of the road. The coefficient of friction between the car’s tires and the road is usually between 0.6 and 0.7. So, given the city’s roads, the level of traction not being as good is quite understandable.
You: So, what are you saying?
Mechanic: What I’m saying is that although the traction is not as good as it could have been, this is quite normal in Ghaziabad. Also, it is hard to tell how much of the wear and tear happened during the skidding itself.
You: OK. Thank you for your time.
Well-structured version of ill-structured problem 1
You are a lawyer in a prestigious law firm. You’ve been assigned the following case:
A man was driving his car when, suddenly, a small boy ran out on to the road chasing a ball. He slammed on the brakes and skidded to a stop, leaving a 15 m long skid mark on the road. Luckily the boy was not hurt, but a policeman watching from the sidewalk walked over and fined the man for speeding. An investigation found out that the speed limit on the road is 55kmph. It also determined that the coefficient of friction between the tires and the road was 0.6. The man’s mass was 75 kg and his reaction time, on average, was found to be about 0.8 s. The car’s information manual indicated the mass of the car to be 1,570 kg. Witnesses say that the boy took about 3 s to cross the 4.5 m wide road.
As the man’s lawyer, will you fight the fine in court? Present your case as best you can.
Rights and permissions
About this article
Cite this article
Kapur, M., Kinzer, C.K. Productive failure in CSCL groups. Computer Supported Learning 4, 21–46 (2009). https://doi.org/10.1007/s11412-008-9059-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11412-008-9059-z