Is interactivity a good thing? Assessing its benefits for learning

Is Interactivity a Good Thing? Assessing its benefits for learning. Nuno Otero1, Yvonne Rogers and Benedict du Boulay. School of Cognitive and Computing Sciences, University of Sussex, Brighton, BN1 9QH, UK. Email addresses: {nunop, yvonner, bend}@cogs.susx.ac.uk Abstract This paper investigates whether adding interactivity to diagrammatic representations aids learning. More specifically, the analyses focus on the extent to which 3D interactive diagrams facilitate the comprehension of a geometry concept, the stereographic projection, that is usually taught using 2D diagrams. Four Interactive Learning Environments (ILE) were built in order to test the effectiveness of different types of diagrammatic representation. Eighty undergraduate students took part in the study, each using one of the ILE versions to explore the concept of the stereographic projection and its application in crystallography and answering a multiple-choice test. The results suggest an interaction between previous knowledge, spatial ability and type of ILE. 1. Introduction. Interactive graphical representations have been used and promoted to facilitate learning (Cheng, 1999). However, little is known about what kinds of graphical representations and what forms of interactivity are effective (Rogers, 1999). The present research explores how external cognition affects learning (for a review see Scaife and Rogers, 1996) and investigates whether providing 3D interactive graphical representations in problems that require the depiction of 3D properties is beneficial for the learning process. The domain chosen was geometry but the issues raised are more general. Geometry makes use of many graphical representations, particularly diagrams. Many studies have addressed the problem of investigating their benefits, especially in problem solving (Larkin and Simon, 1987; Koedinger and Anderson, 1990). The seminal paper by Larkin and Simon (1987) triggered interest in the study of how different external representations facilitate solving different types of task. Using a geometry problem example, they concluded that a diagram was advantageous to the problem solving at hand. According to these authors, a diagrammatic representation is a data structure that indexes information by location. Its utilisation may be preferable to the use of a sentential representation by easing search, decreasing cognitive load associated with labelling and easing recognition of necessary elements for a correct inference. Koedinger and Anderson (1990) propose that geometry experts do not plan geometry problem solving in a step by step manner. Instead, the planning is done at a more abstract level, skipping less important steps and employing diagram configurations. Koedinger and Anderson (1990) also argue that this strategy enables experts to transform a deductive task into an inference one. In terms of the application of the findings to geometry instruction, they suggest that there is space for introducing the teaching of diagram configuration schemas, capitalising on the benefits of using diagrammatic representations. Our study is not comparing sentential and diagrammatic representations. The focus of the research is on the comparison between different diagrammatic representations, considering how additional interactive properties might change the learning task. The stereographic projection was chosen as the application domain. It is a one-to-one transformation from the surface of the sphere (the surface of a 3D geometric entity), less one point, onto a plane (by definition a 2D geometric entity)2. In general terms, the concept belongs to the geometry domain, but in the case of this study there is an emphasis on one of its particular applications: the problem of studying symmetry relationships in crystallography. The use of the stereographic projection in crystallography involves explaining the process of representing the angular relationships between the faces of a crystal using a 2D representation called a stereogram. Some of the graphical representations in textbooks explaining the concept are 2D diagrammatic representations that, through the use of pictorial cues are intended to show 3D "objects" (for example, Borchardt-Ott, 1995). The issue 1 Acknowledgements: Nuno Otero research is being supported by a grant from Fundação para a Ciência e a Tecnologia, Praxis XXI, BD/15717/98. We would like to thank the support given for the experiment by the Departamento de Matemática, Faculdade de Ciências da Universidade de Lisboa and Escola Superior de Comunicação Social (Portugal) and Monica Dias, Maria Manuel Torres and Miguel Gaspar for their advice on building the ILEs. 2 See Figure 1, diagram 1, below. here is whether these representations are more easily understood when 3D interactive diagrams are employed, considering that students usually have some difficulties learning the concept and manipulating the resulting 2D final representations, the stereograms. To make 3D graphical representations in a computer display interactive, one possible strategy is to allow the user to rotate the virtual objects and perceive their spatial properties. In this case, the interactive property provided is the possibility to manipulate the whole representation. However, the manipulation of a diagram could also comprise the manipulation of individual elements of the diagram. We also considered Cox's (1999) suggestion that the successful use of an external representation is dependent "…upon a complex interaction between (a) the properties of the representation, (b) the demands of the task and (c) within-subjects factors such as prior knowledge and cognitive style." (p. 343-344). In particular, Kirby and Boulter (1999) report that previous geometry knowledge and level of spatial ability are important variables in the prediction of geometry test scores. 2. Hypothesis Our experimental hypothesis were based on the belief that: (a) providing 3D versions of the diagrams could "offload" graphical interpretation and mental rotation tasks that seem to underpin understanding the nature of the problem at hand, and (b) the possibility to manipulate the diagrams' elements could explicitly show important features of the relationship between the elements of the representation, which in turn are central to the understanding of the concept in question. The hypotheses were: 1) The more interactive ILEs will provide better immediate learning results than the less interactive ILEs. By immediate learning results we mean the results from answering multiple-choice questions about the concepts being learned on the ILE. 2) Subjects with less knowledge of geometry and less developed spatial skills will benefit more from the use of the 3D interactive ILE, achieving better scores in the multiple-choice test than those using the other three systems. 3. Methodology 3.1 The Interactive Learning Environments. Four Interactive Learning Environments (ILEs) were created with different graphical representations. Each learning environment tackled the same concept in the same manner, except for differences in the diagrams and in interactivity. In total, the ILEs had 15 diagrams. Two 2D systems were built differing in the possibility for learners to manipulate the diagrams' elements. Two 3D systems were also built again differing in the possibility for learners to manipulate diagrams' elements. The four systems were named 2D, 2DI, 3D and 3DI3. The concept of the stereographic projection was broken into six explanatory steps, each containing a piece of text and corresponding diagram(s). At the end of each explanatory text/step there were multiple choice questions that the learner had to answer on a separate sheet (the multiple-choice test). There was some overlap in the information displayed in each of the ILEs between the diagrams and the text. That is, in some cases the information displayed in the diagram illustrated what was being referred to in the text, but in other cases there was no overlap. The text employed a non-quantitative approach to the explanation as there were no algebraic or analytical expressions present. The multiple-choice test reflected this design decision, addressing issues of definition and of the visualisation of properties closely related to the understanding of key issues of the stereographic projection and its application. Since the questions of the multiple-choice test were present while subjects were interacting with the ILEs, they are regarded as learning goals of subjects. The design of the ILEs and writing of the content was informed by a maths teacher, a geology teacher and a maths post-graduate. Prior to the experiment, a pilot test with 3 subjects was run focusing on possible design and usability issues. The data collected fed into a re-design of the content layout and navigability of the ILEs. 3 The 3D IlEs were built using VRML and the 2DI ILE was programmed in Director. Projection line A a r - the projection plane S (1) Diagram similar to the ones displayed in textbooks. (2) The corresponding3DI diagram in a different perspective with (A) and (a) in different positions. Legend: If we consider an interactive version of the ILE then: (i) the projection line is a manipulable line that can be rotated along the fixed point S; (ii) the (A) dot corresponds to the point on the sphere to be projected and moves accordingly whenever the line is manipulated; (iii) the (a) dot corresponds to the point projected in the projection plane (r) and moves accordingly whenever the line is manipulated. Figure 1 - Examples of two types of diagrams, 2D (1) and 3D (2), showing the basic principle of the stereographic projection. 3.2 Participants Eighty subjects, 42 first year undergraduates studying a geology course and 38 from a mathematics course, were randomly assigned to one of the four ILEs. The two groups were included to explore the possibility that coming from a different course could reveal distinct interaction patterns and/or responses to the multiple-choice and post tests, which in turn could lead to exposing effects of prior knowledge. 3.3 Procedure The subjects were instructed to explore the geometry concept with the ILE they were allocated to and answer the questions displayed at the end of each text frame. A brief explanation of the functionality of the different ILEs was also given. No constraints were imposed on the time available or sequencing of the exploration. Prior to the use of the ILEs, the subjects were tested on their spatial ability through the Paper Folding Test (PFT) (Ekstrom, French, Harman, 1979) and their geometry knowledge through a geometry test (GT) similar to the GRE geometry module (for example, see Stewart, O'Toole, 1999). Finally, a post-test was completed that consisted of 17 questions about the concept taught. Forty subjects were video taped with a camera focusing on the computer screen. This paper addresses itself only to the hypothesis stated above. 4. Results 4.1 Initial considerations regarding the co-variates. Considering the possible effects of spatial ability and geometry knowledge in the performance with the ILEs, an initial preamble about the co-variates seems helpful. The PFT was revealed to be a good predictor of the outcome on the multiple-choice test. However, the GT was not. Hence the impact of learner's previous geometry knowledge was not properly uncovered. Two possible conclusions could be drawn: (i) prior geometry knowledge does not affect much the comprehension of the concept taking into account the way it is presented in the ILEs or, (ii) the GT is not adequate to study the implications of prior geometry knowledge in this particular problem, because it is not sufficiently focused on the concept in question. We are inclined to accept (ii) and in section 4.4 we investigate the differences between the maths and geology students, trying to assess the importance of background knowledge. Nevertheless, while there was a high correlation between the GT and the PFT for the geology students; this correlation was not found for the maths students. This different dependency relationship for the two student groups was also obtained, in general, between the PFT and the multiple-choice test, suggesting that the two phenomena might have a common basis related to previous knowledge. 4.2 Testing hypothesis 1: The more interactive ILEs will provide better immediate learning results than the less interactive ILEs. By immediate learning results we mean the results from answering multiple-choice questions about the concepts being learned on the ILE. To test hypothesis 1 an ANCOVA was run on the data collected with the type of students' course and ILE as independent variables, the results of the multiple-choice test as dependent variable and PF and GT tests results as covariates. The results obtained indicate that hypothesis 1 cannot be confirmed, as no significant effect for the type of ILE, type of course or interaction between ILE and type of students' course was found. Nevertheless, an interesting trend was observed concerning differences between the geology and mathematics students. The performance of the geology students was better in the 2D than in the 2DI ILE, approximately the same when comparing the 2DI and 3D and worse in the 3DI ILE. The mathematics students, in their turn, performed better on the 2DI than on the 2D, and even better on the 3D, but scored worse on the 3DI. We should interpret these results with extra care since the ANCOVA model ran assumes a similar effect of the co-variates for all the groups and we already indicated in the previous section that maths and geology students differ in their relation between spatial ability and the multiplechoice test. In fact it could be the case that different ILEs might be beneficial to different clusters of students with different background knowledge and levels of spatial ability. This topic will be explored further in section 4.4. 4.3 Testing hypothesis 2: Subjects with less knowledge of geometry and less developed spatial skills will benefit more from the use of the 3D interactive ILE, achieving better scores in the multiple-choice test than those using the other three systems. To test hypothesis 2, subjects' scores in the GT and PFT tests were ranked and three groups were formed: (i) lowlow - which is constituted by low scoring GT and PFT students; (ii) medium - that aggregates the medium scoring students in both tests (high-low, low-high) and (iii) high-high - composed by high scoring GT and PFT students. In this case, due to the small number of observations, we could not subdivide these three groups into maths and geology students. Hypothesis 2 was also not confirmed. The ANOVA showed the performance with 3DI ILE being no better than with other ILEs for lower spatial ability and geometry learners. However, we should re-emphasise that the GT did not seem to be a good predictor of performance and so the groups formed do not help to uncover part of the relationship under study. 4.4 Further analysis Summarising, the results indicate that there is no "best ILE fits all" conclusion. In fact, the performance on the multiple-choice test seems to be affected by an interaction between background knowledge, level of spatial ability and type of ILE. In order to investigate these issues further, exploratory regression models were computed testing the relationship between the PFT and the multiple-choice test for the different groups. Degree: Maths Degree: Geology 14 14 12 12 10 10 8 ILE 6 3DI 3D 4 2DI 2 2D 2 PFT 4 6 8 10 12 14 16 18 20 Multiple choice test Multiple choice test 8 ILE 6 3DI 3D 4 2DI 2D 2 2 4 6 8 10 12 14 16 18 20 PFT 4 Figure 2 - Graph plotting the relationship between PFT and the multiple-choice test for the geology and maths students in the different ILEs . Again, we stress out that the results are highly exploratory due to the sample size and must be taken cautiously. Some of the interesting trends observed were: • Taking all the subjects together, we observe that performance with 3D ILEs seem to be less dependent on spatial ability than 2D ones. If this is confirmed in future experiments the implications might be important, since using 3D could help learners with less developed spatial ability. • Subjects from the different programmes (maths and geology) show different relationships between the results of the multiple-choice test and spatial ability on the different ILEs (see Figure 2). The maths students seem to be less dependent on their spatial ability than geology students. Moreover, if we separate the questions in the multiple-choice test based on whether the information to look for is in the text or in the diagram, additional differences between maths and geology students and between types of tasks are suggested. This seems to indicate an effect of background knowledge. Probably the maths students were applying their knowledge of geometry and related diagrammatic representations but nevertheless were not able to take full advantage of this 4 The lines displayed in the scatterplots in figure 2 are intended to illustrate the different trends (they are not statistically significant). in order to systematically outperform their geology colleagues. An interesting issue for further research is to investigate if the knowledge of using static diagrams interferes with the use of its interactive versions. 5 Conclusions The best answer to the question in the title of this paper, "Is Interactivity a Good Thing?", is probably that the manipulation of representations in the forms explored by these ILEs does not assure learning gains for all types of learners. Moreover, just adding interactivity does not seem to be an efficient way of building ILEs. The results concerning the effects of spatial ability and background on performance point that learning is affected by a complex interaction of representations' properties, task demands and within-subjects factors (cf. Cox, 1999). Focusing on the ILEs under study, it seems clear that the 3DI system did not provide the expected learning gains. Why? Possible answers might be: a) This type of concept in an ILE with a sentential representation with the characteristics of the one employed and assessed in a similar basis is best explained with 2D diagrams. The design solution is quite straightforward: restrict the use of interactive diagrams. However, an underlying research topic would be to see if changing the textual representation and the questions presented (including, for example, analytical and algebraic expressions) would alter the balance of the results. b) The students did not use the ILEs in the way intended, for example, not taking advantage of the more interactive versions. In this case, it seems that appropriate feedback and help might be needed, focusing on fostering clusters of "efficient" learners' actions, enabling them to extract important information to reason. For example, when the learner is manipulating a certain element of the diagram, the ILE could display information about what were the important aspects related. Moreover, if the learner was not performing a certain meaningful action, the ILE could trigger an animation showing the possibility and providing information about the relevance of such action. c) Subjects' might have difficulties in co-ordinating text and interactive diagrams. This topic clearly needs further study and relates to the issue of the use and design of multiple external representations in ILEs (for a review see Ainsworth, 1999). Important points are to know how and when to support the translation between different external representations. Providing dynamic linking is one of the possible design solutions. d) Learners focused too much on the diagrams and gave inappropriate attention to the important information in the text. In this case a possible remedy would be to mix the design approaches considered in (b) and (c), where the learner could get feedback about his/her actions but the information could be directed to the sentential representation. It seems that the issues referred to above are artificially separated. It is more likely that, in order to increase the efficiency of an ILE, a mélange of such approaches and design solutions needs to be considered. Such issues are clearly important and additional investigation is required, especially if one wants to take advantage of the promising hint that performance on 3D ILEs in this type of problem seem to be less dependant on spatial ability. References Ainsworth, S. (1999). Designing effective multi-representational learning systems (Technical report nº 58). ESRC Centre for Research in Development, Instruction and Training: University of Nottingham. Borchardt-Ott, Walter (1995). Crystallography. Berlin: Springer Verlag. Cheng, P.C (1999) Interactive law encoding diagrams for learning and instruction. Learning and Instruction, 9(4), 309-326. Cox, R. (1999). Representation construction, externalised cognition and individual differences. Learning and Instruction, 9(4), 342-363. Ekstrom, R.B.; French, J.W.; Harman, H.H. (1979). Cognitive factors: their identification and replication. Forth Worth, Tex: Texas Christian University Press [for] The Society of Multivariate Experimental Psychology. Kirby, J. R.; Boulter, D. R. (1999). Spatial ability and transformational geometry. European Journal of Psychology of Education, Vol. XIV, nº2, 283-294. Koedinger, K.R., & Anderson, J.R. (1990). Abstract planning and perceptual chunks: Elements of expertise in geometry. Cognitive Science, 14, 511-550. Larkin, J.; Simon, H. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65-99. Rogers, Y. (1999). What is different about interactive graphical representations? Learning and Instruction, 9, 419-425. Scaife, M; Rogers, Y. (1996). External Cognition: how do graphical representations work? International Journal of HumanComputer Studies, 45, 185-213. Stewart, M.; O'Toole, F. (1999). Arco Teach Yourself the GRE in 24 Hours. IDG Books Worldwide.