The Effect (or Lack Thereof) of Spatial Skills Training in a Mid-Major Computing Course

Spatial skills, which we define using Margulieux’s definition [18], are often conflated with other related terms. While we acknowledge that there are distinctions between spatial visualization, spatial ability, and spatial skills, we adopt the use of the term spatial skills to signify its malleability. In other words, spatial skills can be changed, and we choose to emphasize that with our word choice.

While we refer to spatial skills broadly in this work, it is important to understand that there are different types of spatial skills and different theories of how these types of spatial skills can be categorized. Linn and Petersen refer to three categories: spatial perception, mental rotation, and spatial visualization [14]. Uttal et al. defines four types of spatial skills along the dimensions of intrinsic-extrinsic and static-dynamic [33], which the Linn and Petersen categories can map onto. Each of the four categories could be interpreted as follows:

These categorizations are, in part, where the definitions and word choice in spatial skills work diverge. While some work we mention here will refer to different terms or aspects of spatial skills (e.g., mental rotational ability), we see them all as connected to this core concept of a mental capacity that involves symbols or objects.

Spatial skills have consistently been shown to predict achievement in STEM courses [34]. Although the connection between spatial skills and achievement in STEM settings may be surprising, spatial skills can be trained to the point of improving both spatial skills and STEM outcomes [33]. Uttal et al. conducted a meta-analysis of studies that implemented various spatial training techniques [33]. Overall, they found training to have a moderate effect on spatial skills (g = 0.47), but they also explored different types of training types: courses, video games, and spatial tasks. Courses, which had an effect size of g = 0.41, include those developed by Sorby and Baartmans to improve spatial skills among engineering students [30]. This course consists of ten weeks of activities across a range of 3-D spatial skills and is accompanied by a workbook for students to draw isometric, orthographic, and rotated shapes. This particular spatial skills training course has been shown to have a positive impact on spatial skills and STEM outcomes [29].

As there are different categories of spatial skills, so too are there different types of spatial skills training to cater to the different skills. The workbooks described above, for example, are focused on intrinsic-dynamic spatial skills training, as they define objects that participants are then asked to move. Some spatial skills categories may respond less to training, including the intrinsic-static category [33]. These skills are more about applying principles or rules to an object than moving the object, which may be harder to train.

There is evidence from the aforementioned meta-analysis of spatial skills training studies that students with lower spatial skills benefit more from training [33]. This makes logical sense as these students have more room to grow in their spatial skills, so any efforts to improve their skill sets could see more improvement than their higher spatial skills peers. As a result, many studies in STEM education focus specifically on students with low spatial skills as determined by a preliminary spatial skills test at the start of the study [22, 29]. Studies that choose to give spatial skills training to all students still find a larger improvement for students that initially scored lower on a spatial skills test [17].

Over the past twenty years, there have been various efforts to explore the relationship between spatial skills and CS. One of the first of these works found that spatial visualization skills, as measured via a paper folding test, were correlated with success in an introductory computing course [8]. Other work similarly found correlations between mental rotation ability, as measured by a mental rotation test, and success in introductory computing for Master’s students [11]. Cooper et al. continued to confirm a connection between spatial skills and achievement in CS, this time in high school students [5]. In another study, spatial skills, measured at the start of the semester, were able to predict student computing knowledge by the end of the semester [2].

Despite this abundance of evidence that there exists a relationship between spatial skills and CS ability, it remains unclear why that connection exists. The prevailing theory, Margulieux’s Spatial Encoding Strategy (SPeS), hypothesizes that having better spatial skills leads to better strategies for storing information and orientation, which would aid computing students who are actively engaged in code comprehension and debugging tasks [18]. The SPeS theory has been supported by recent research which found, through a think-aloud study, that students with higher spatial skills were more able to demonstrate advanced chunking (information storing) and encoding (orientation) techniques; students with lower spatial skills were less likely to demonstrate these skills [26].

The connection between CS and spatial skills may lie in a particular aspect of computing. There have been fMRI studies that indicate that the same region of the brain is activated for spatial operations and for data structure tasks [10]. Another neuroimaging study found that students at the beginning of a programming course have certain neural activation patterns while completing coding and spatial reasoning tasks [7]. These patterns can predict performance on a programming assessment at the end of the term [7]. Consequently, not all aspects of a computing program may equally correspond to spatial skills. There was no correlation found between growth in spatial skills and courses with limited computational models, including human-computer interaction and ethics courses [25]. Meanwhile, when spatial skills was measured in a computer graphics course, students’ mental rotation and spatial visualization abilities correlated with their success in the course [15].

Beyond just a relationship between skills in spatial and computing domains, it has also consistently been shown that training spatial skills in an introductory computing course has positive effects on student outcomes [2, 22, 27]. However, there is some evidence that when this training is offered in a CS curriculum may matter: Parkinson and Cutts found a larger impact of spatial skills training in a CS0 course (typically geared towards students not majoring in CS) when compared to an introductory CS course, a finding that surprised the authors [24]. Further, spatial skills training in the second semester of a year-long introductory computing course was found to be less effective than in the first semester [27]. This is important to consider in conjunction with evidence that spatial skills training can have a larger impact on students with low spatial skills [33], providing evidence for why earlier spatial skills interventions might be more beneficial for students. It should also be noted that students tend to find spatial skills training challenging, occasionally guessing answers to complete the task [17]. However, to our knowledge, no further analyses have been conducted regarding students’ engagement with the spatial skills training materials in this manner.

Spatial skills have also been shown to relate to other factors that may play a role in our CS classrooms and research. In one study, students with higher mental rotation ability were able to complete a code comprehension task quicker and more efficiently than students with lower mental rotation ability [12]. Other studies have found that spatial skills training may have a greater impact on students of different races and socioeconomic statuses [5] and on women students [17]. The connection between spatial skills and socioeconomic status has since been explored, and generally supported, in other studies [17, 21]. Parkinson and Cutts found that spatial skills increased with the level of academic achievement in CS; that is, master’s students in computing had higher spatial skills than honors undergraduate students, who had higher spatial skills than first-year students [23]. Further, spatial skills tend to increase over the course of study in a computing program [25].

Our present study seeks to build on this prior literature around spatial skills training in CS classrooms while also pushing the boundaries of what is known about this phenomenon in our field. While most of the above studies focus on introductory courses, we seek evidence as to whether spatial skills training can be effective later in a CS major. We also continue the efforts to understand what factors correspond to spatial skills so that we may know what could help, or hinder, students throughout this process. Given the related work on this topic, we present the following hypotheses in regard to our two research questions:

Our study was based at a large public university in the United States. In particular, we focused on a course on software engineering, which is required for CS majors at the institution where this study was conducted. The course is the last core requirement in the major sequence, meaning all students majoring in CS take the course, generally before they enroll in elective courses but after they have completed the introductory course sequence. The course does not contain any programming, focusing instead on the other phases of the software life cycle (e.g., requirements, design, testing). Conveniently, this course also allows us to remove the direct impact of programming influencing spatial skills. However, students are enrolled in multiple courses in a term and are often simultaneously enrolled in data structures courses along with this software engineering course. To control for the effect of these courses on the students in the study, we asked students to report what other courses they were enrolled in during the same semester.

Our study consisted of a control group and a treatment group. The delineation between the two groups can be found in Table 1. Each group was a different section of the same course on software engineering, described in Section 3.1, and received the same lectures and assignments and completed the same final project. Both groups took the same pre- and post-tests of CS and spatial skills knowledge and filled out a demographics survey at the end of the course. In the intervening ten weeks, the treatment group engaged in spatial skills training activities, including a weekly video and quiz to be completed outside of class and workbook pages to be completed during class. The control group received additional material related to the course, including readings on software engineering to be completed as homework and activities based on those readings to complete during class. This material was explicitly designed to not overlap with the spatial skills training modules; rather, these materials provided more practice with course content above what is normally delivered to the students. The two conditions were designed to control for time-on-task, both during and outside of class time. All of the out-of-class activities, such as the quizzes, videos, and readings, as well as the pre- and post-tests and demographic surveys, were delivered through the learning management system that the course uses to post the schedule and assignments.

The CS assessment used for the pre- and post-CS test in this study was SCS1, which is a pseudocode-based assessment designed for use at the end of an introductory course in computing [20]. Given that students in the course in this study had all completed the first CS course, but no other course was guaranteed to be completed, the use of SCS1 was deemed appropriate. Students were given 60 minutes to complete the 27 questions on the assessment, as advised by the creators of SCS1 [20]. The pre- and post-spatial skills test was the Purdue Spatial Visualization Test on Rotations (PSVT:R) [9]. This test has become the standard for spatial skills research and has been used by many studies in CS education to measure spatial skills gains [22]. Students were given 20 minutes to complete the 30 questions on the assessment. The software engineering course in this study has an emphasis on diagrams (UML class diagrams, software architecture diagrams, use case diagrams, etc.). However, these spatial skills would fall under the extrinsic-static category, as discussed in Section 2.1, as opposed to the intrinsic-dynamic training that tested with the PSVT:R. Thus there was no expectation that the control group activities could lead to improved performance on the spatial skills measure.

The spatial skills training workbooks, videos, and quizzes were developed by Sheryl Sorby [31]. Similar to the PSVT:R, these training materials are thoroughly used in computing education research [22]. These materials were chosen, in part, to allow for comparison between the present work and the past literature on spatial skills and CS. As discussed in Section 2.2, this workbook asks students to draw shapes from different angles, making this an intrinsic-dynamic focused training.

The instructor of the course section that served as the control group was the lead researcher of this study. They developed the class activities that were used to control for time-on-task, given their familiarity with the spatial skills training materials. This allowed for minimal overlap between the additional materials seen between the two sections, and thus the two groups of this study. That is to say, the control group was assured to not receive incidental spatial skills training akin to what the treatment section was receiving. The instructor of the treatment group was not involved in the research, nor an expert in computing education research. The treatment group instructor was given a script to introduce the spatial skills activity, emphasizing the importance of these skills in the students’ computing courses and their ability to improve them through the activities. The two instructors had weekly meetings to monitor the status of the project and address any questions or issues with the activities. No teaching assistants were involved with this work, either in the administration of activities or the grading of assessments.

This study was approved by the local ethics approval board. Students were unaware that the sections would be different prior to enrolling in the course, nor were they aware of the research study being conducted. While all students enrolled in each section participated in the respective activities, only students who broadly consented to the use of their class data in a research study were included in our analysis. Ultimately, 51 students in the control and 31 students in the treatment section consented to be in this study. The completion of all of the activities and assessments was counted as participation points for the course; these elements were not graded for correctness. Not every student completed every activity in their section, so for statistical analyses that require paired samples, the sample size may be smaller than the number of students in that section that consented. The rate of students who receive a D, F, or Withdraw from the course, often known as the DFW rate, for the two sections was similar; given the course is required for the CS major, no students dropped from either section during the term. The control group section had one student who received an F at the end of the course, while the treatment group section had two students who received an F in the course. Besides these students, the average final course grade for the students in the control group was 92 and for the treatment group was 88 (out of 100). A summary of our study population can be found in Table 2.

We provide a summary of our dataset in Table 3, both for the whole sample and for the students who had low scores on their pre-test of spatial skills (a threshold of 60% correct is typically applied in the literature, as seen in [3, 30]). Timing data is provided only on average and only for the entire study population. Due to constraints with the learning management system used to deliver the intervention, we do not have finer grain timing data than averages for each section.

We analyzed our data using different statistical techniques depending on the research question being answered. To begin all analyses, we tested our data for normality using a Shapiro-Wilks test. We found that not all of our data were normally distributed, and thus chose to use non-parametric analyses when appropriate.

To answer our first research question, regarding the impact of spatial skills training on spatial skills and CS ability, we looked at differences within and between each study group. To analyze differences within each group, we performed Wilcoxon Signed-Rank Tests to compare the paired pre- and post-test scores on the spatial and computing assessments. We did this for all students in each group, and then again for the students that had low spatial skills as determined by their score on the pre-test. Since we could not divide students into sections and only apply the treatment to low spatial skills students, we chose to conduct post-hoc analyses to understand the differential impact of the training on these students. Due to conducting two comparisons within each group, we applied a Bonferroni correction to our significance level, such that p < (0.05/2) = p < 0.025. To analyze the differences between the control and treatment groups, we used an Analysis of Covariance (ANCOVA) to understand the impact of the study condition on the post-test scores, controlling for the pre-test scores.

To answer our second research question, regarding the other factors that influence the impact of spatial skills training, we ran an ANCOVA to explore the effect of gender and concurrent enrollment in a data structures course. We also conducted a multiple regression to understand the impact of attempts on the spatial skills workbooks (discussed in Section 3.3.1) on the post-test spatial skills scores. All of our analyses were performed using R version 4.3.0.

We began to analyze the impact of spatial skills training by examining the differences in the pre- and post-tests of spatial skills and CS knowledge. We found a statistically significant decrease between the pre- and post-tests of spatial skills (Z = -2.821, p = 0.005) for the control group. We did not find a statistically significance difference for the CS test for the control group, as seen in Table 4. Additionally, we did not find statistically significant differences within the treatment group. Further, when looking only at the low spatial skills students in both groups, we do not find statistically significant changes in either spatial skills or CS knowledge for either group.

We also ran an ANCOVA to control for the pre-test spatial scores in the post-test spatial scores and consider the impact of the two conditions of the study. As shown in Table 5, we do not find that the condition (i.e., control versus treatment group) plays any significant role in the final scores. However, the pre-test spatial scores were a significant factor in the post-test spatial scores. Similar results were found for the CS scores.

To understand the effect of certain factors, as shown in prior literature, on spatial skills and spatial skills training, we conducted an ANCOVA on our data across the two study groups, as shown in Table 6. We find that, when controlling for the condition of the study, the student’s gender and whether or not the student is in a data structures course do not statistically significantly influence the change in spatial skills. To further understand this finding, we also ran a Mann-Whitney U test to model the effects of gender on the pre-test spatial skills scores. We found that gender was a significant factor in the pre-test spatial scores across both conditions (U = 196, p = 0.0497), but not a significant factor for the post-test scores (U = 262.5, p = 0.8554). We did not further analyze the effect of data structures enrollment on the pre-test scores given that students had not completed any portion of the data structures course at the time that they completed the pre-test for this study.

Within the treatment group, we further analyzed the impact of the spatial skills training on their post-test scores. We ran multiple regression analyses on the impact of the ratio of student attempts in the spatial skills workbooks (as described in Section 3.3.1) on the post-test spatial skills scores when controlling for pre-test spatial skills scores. The results shown in Table 7 indicate a significant effect of trying on the workbooks when considering all students in the treatment group. However, when focusing on the students who scored low on the spatial skills pre-test, the impact of attempting the workbook is no longer significant.

We conducted further regression analyses exploring the relationship between the overall workbook scores, final course grades, and post-spatial skills scores. We also factored in a quantified variable representing how often the student self-reported playing with spatial toys (e.g., Legos, Minecraft, etc.). None of these variables, either individually or in conjunction with other variables, for the entire treatment condition or the low spatial skill student group were significant factors in our models.

Our analysis did not find statistically significant differences in the post-test of spatial skills or CS knowledge for either the control or treatment group after controlling for pre-test scores. However, we did find that the control group saw a statistically significant decrease in spatial skills (ΔSpS = −0.14, p = 0.005). Meanwhile, the treatment group saw smaller, not statistically significant changes: ΔSpS = −0.027 (p = 0.670) and ΔCS = 0.003 (p = 0.829).

Based on prior work [22, 25], we had no reason to expect a decrease in spatial skills in either group. However, given what we saw in our data, while the spatial skills training did not improve spatial skills in the treatment group, it seems to have prevented a decline in spatial skills that was seen in the control group. However, this is an area that would benefit from further research with a larger dataset to untangle the impact of spatial skills training at this point in a computing major.

While it is possible that the spatial skills truly declined in the control group, it is also possible that the decline is due to the students not taking the post-test seriously. The post-test did not directly connect back to anything they saw that semester except for the pre-test, it was graded for completeness and not correctness, and they took less time to complete the post-test than the pre-test (as seen in Table 3). However, this pattern was also true of the treatment group. If the control group did not take the post-test seriously, the treatment group may not have either, which would still mean that they took it under the same conditions. However, we only see a decrease in spatial skills for the control group, not the treatment group.

Further, the only significant factor we found predicting students’ spatial skills post-test was a student’s pre-test score in spatial skills, regardless of the study condition. This finding is similar, though distinct, from prior work that showed students’ prior spatial ability predict their CS scores above and beyond other factors [2]. This replicated finding gives us further confidence that the students did not take the post-test any less seriously than they took the pre-test.

While in some respects our findings are surprising given the prior work on the positive effect of spatial skills training in CS [3, 22, 27], all of the prior work focused on introductory computing courses. This study is centered outside of that context, in a course required for the major but beyond the introductory CS course sequence. This work provides preliminary evidence that spatial skills training interventions are most effective early in the major. Our findings are also similar to recent work that indicated that students’ spatial skills gains were not significant in the second semester of an introductory CS course sequence [27]. These findings could mean that waiting until the students are in their second or third year of study is too late to intervene.

This might be because novices need more support to grow spatial skills [18], perhaps because those who persist in CS programs and careers incidentally have greater spatial skills as they attain higher levels of academic achievement [23]. Additionally, lower spatial skills students have more to gain from spatial skills training [17, 33]. If students early in the computing major have lower spatial skills, need more support, and have more room for improvement, then it stands to reason that spatial skills training would most benefit introductory CS students.

Another possibility that would make spatial skills training less effective outside of introductory computing is the idea that everything is novel to students in an introductory course and less so as they progress in their degree program. In an introductory course, students may be more receptive to activities that are tangential to the course, primarily because they cannot recognize it as tangential. The more CS courses that a student takes, the more they may recognize what does, and does not, belong in a computing course. If a student has taken two or more CS courses, and no explicit spatial skills connections have been made, they may be less motivated to fully engage when a spatial skills training is presented to them. This, in turn, may inhibit their ability to grow their spatial skills effectively.

Prior research has offered spatial skills training as a type of remediation course for students who score below 60% on a spatial skills test at the start of a term. Past research demonstrates that low spatial skills students benefit the most from spatial skills training [17, 22, 33], but our research did not replicate this finding. However, when we begin to look further into the effort put into the training, as we do in the next section, we can start to understand where the boundaries are of the effectiveness of spatial skills training.

We did not have the ability to direct only low spatial skills students into the spatial skills training, but we did look at their specific performance through a post-hoc analysis. Our analyses do not indicate statistically significant changes in any knowledge set over the course of the semester within either group. Further, the changes for the low spatial skills students within the two groups were not statistically significant either, for spatial skills or CS knowledge.

Similarly to the previous discussion regarding the decline in spatial skills, one hypothesis for the spatial skills intervention not improving spatial skills is that the students did not take the activities seriously, since it was graded for completion, not correctness, for class credit. The best proxy we have for the engagement students had with the post-test is the amount that students engaged with the training materials in the treatment condition. When modeling the impact of the attempts on the spatial skills workbook on post-test spatial skills scores, while controlling for pre-test spatial skills scores, we see that how much a student tried to complete a sketch does statistically significantly predict their post-test scores. However, this effect disappears when only looking within the low spatial skills group. This is especially discouraging because it implies that no matter how much a student with low spatial skills student tried on the workbooks, it did not impact their post-test spatial skills score. There are likely other factors at play that can support, or deter, low spatial skills students from improving their spatial skills, such as the type of training or the context of the training.

Prior work indicates that the instructor and teaching assistants can make a difference in the effectiveness of spatial skills training [22]. In our study, the condition is a proxy for the instructor. While the lead researcher taught the control group to make sure they did not bias the outcomes of the treatment group, the consequence was that the opposite occurred, with the treatment group instructor less familiar with spatial skills training. It is possible that an instructor who was more invested in the spatial skills training would have led to the training impacting outcomes more.

To gauge students’ CS knowledge over time, we used a validated measure of learning in CS called SCS1 [20]. However, this measure is only validated for the content of introductory CS courses. Meanwhile, we administered this assessment outside of an introductory course. Our reasoning was that the students in the mid-major course should know the content that is covered in SCS1. Additionally, we recognize that this assessment is measuring a certain subset of computing knowledge that may not be the same subset of computing that is most malleable through spatial skills training [7]. In other words, spatial skills training may be affecting mid-major student performance in CS, but it may not be affecting their performance on that subset of CS. The most evidence we have right now for where spatial skills interplay with computing is within data structures problems [10, 25], which are only covered in SCS1 in part, with the inclusion of arrays. It is possible that another assessment, such as the Basic Data Structures Inventory (BDSI) [28], would be more appropriate to use in studies of spatial skills impacts on computing outcomes.

Part of the appeal of situating this study in this course was the course focus on diagrams: UML class diagrams, architecture diagrams, use case flows, etc. While the treatment condition received spatial skills training, the control condition received more course-related activities. This inherently meant that the control group received more time working with diagrams. Given the distinct, statistically significant drop in spatial skills among the control group, it seems that the control activities were worse for spatial skills, as measured by the PSVT:R, than the spatial skills training. Our findings can be explained by the differences in spatial skills and processes that each condition received [33]. When completing the spatial skills workbooks, students were completing intrinsic, dynamic tasks, which typically involved the rotation of a singular object. However, the students in the control condition were completing extrinsic, static tasks, involving no rotation of an object, but rather understanding the relationship between the objects. These different categories of spatial skills, as described in Section 2.1, are distinct and serve different purposes. Further, the PSVT:R specifically measures intrinsic dynamic spatial skills and thus catered more towards the treatment condition. Our research begins to prod these differences in the context of CS education, and we encourage future work to continue to disentangle the effects of these different spatial skills in our domain.

Our study presents a novel approach to studying spatial skills training in a CS program. For one, we focused on a non-introductory course. Although some research has been done on individuals in CS outside of the first computing course (e.g., [23, 26]), the area is due for further exploration. Additionally, our study is based on a non-programming, software engineering course. We do not currently understand what aspect of CS leads to a connection in spatial skills, especially since CS is such a virtual domain. By expanding the research on spatial skills to other areas of CS, we can add to the existing SPeS theory [18]. For example, our results indicate that a non-programming, diagram-based course does not inherently improve students’ spatial skills, perhaps because they are not forced to use encoding strategies as much in the in-class activities they complete as they would if they did programming tasks instead. Further, SPeS offers an explanation that, as learners progress from novices to experts, they rely less on problem-solving skills, such as spatial skills. As our study is conducted mid-major, not with novice students, spatial skills training may be less relevant or effective given the students’ progression into more domain-specific courses.

To the best of our knowledge, our work is the first to create a rubric for scoring the spatial skills workbooks and incorporate those scores into the analysis. Further, we present our novel approach to codifying and quantifying what it means for students to put forth effort and try on the sketches in the workbook. While knowing whether spatial skills training works or not is valuable, it is also valuable to understand why it may be less effective when no impact is observed. Analyzing the student sketches further can provide deeper insight into how their spatial skills are evolving, or not, throughout the intervention.

Many CS education research studies on spatial skills use the same spatial skills test (PSVT:R), which we use as well. However, there is no standard measure of CS knowledge used in these studies. Oftentimes, a final exam or final course grade is used as a proxy for CS knowledge. Our study is the first to use the pseudocode-based SCS1 assessment in its entirety. Though other studies have used subsets of SCS1 [2, 3, 21], those uses are further outside of the context for which SCS1 was validated [19]. SCS1 may not be appropriate in all studies of spatial skills and CS, and its timing (one hour) and difficulty level may be deterrents to its use. Regardless of which assessment is used, we encourage other CS Education researchers to move towards standardized measures of CS learning. This will aid in comparing results across studies and better inform the community of the effects of spatial skills training.

Although we did not see gender as a significant factor in post-test spatial skills scores, we did find that gender was a significant factor in pre-test spatial skills scores in both study conditions. Given the significance of gender at the start of the term, combined with the lack of significance at the end of the term, we can hypothesize that the gaps in spatial skills between men and women students shrink over the semester. This is similar to prior findings shown in the literature [17, 33]. However, it should be noted that our data set is too small, with only eight females in the control condition and five females in the treatment condition, to dig much deeper into the differential effects of the training among the genders.

Our divergence from prior findings on the effects of spatial skills training could be due to the new context of the training being beyond an introductory CS course. However, we should also consider the fidelity of the implementation of the intervention used. As the study was being carried out, the instructor of the treatment group observed that students did not talk to each other while completing the spatial skills workbooks. The students also did not make use of toy blocks, which were made available to them as an aid for visualizing the rotations of shapes. Increasing student interactions and use of visual aids may help in the training having a greater effect on spatial skills at the end of the intervention.

Our sample size, while large enough for the statistical analyses we used in this paper, is limiting. There is not enough data to cross-cut it any further, including looking more into the gender differences in spatial skills, especially among the low spatial skills students. A larger sample size is critical to dive deeper into the effects of the various factors that could be hindering students from developing their spatial skills.

Our analysis of sketches in the workbooks, including whether a student tried or not, does not fully capture a student’s understanding, intent, and attempt to understand a problem. To do this, we would need to conduct cognitive interviews as students think out loud through the workbook. We could also ascertain their effort level with a self-report survey, asking students after each workbook session how much they felt like they tried on the activity, similar to Ly et al. survey of guessing and cheating [17]. However, given that the observations about trying on a sketch were gathered in a post-hoc analysis, we did not include any element of analyzing the sketches in this way. Further work should be done in this vein if we are to accurately ascertain when spatial skills training works, and when it does not, in a CS major.

This paper primarily presents null results. However, that does not imply that the contributions of this work are also null. While these findings run askew from prior work on spatial skills in computing, our study provides promising directions for future research. Understanding when spatial skills training is the most effective is beneficial for departments and professors contemplating the inclusion of spatial skills in their curricula. Further, if we understand when spatial skills training has the most impact, then our students can receive optimal timing of this additional instruction, aiding in their future success in CS and STEM fields.

We previously discussed how our work builds on prior findings and, in turn, how our work can be built on in the future. Beyond what has already been mentioned, we also encourage future endeavors in spatial skills in the CS learning context to explore other spatial skills training methods that have been shown to be effective in other fields, as shown in [33]. While the standard method in our community has been to use a course, further research is needed to determine if video games (e.g., Tetris) or spatial tasks (e.g., animating 3D media) is similarly, or more, effective with our students. Other research has found that technical reading tasks may be more beneficial for CS students than spatial skills training [6], which also deserves further investigation. We should do what we can to discover what works best to support all of our CS students to succeed.