Opportunities and Risks for Game-Inspired Design of Adaptive Instructional Systems

As the prevalence of games continues to increase within popular culture, educators and instructional developers continue to look to games as a model for successful design of interactive media. Games are one of many media-based tools with a number of potential benefits for learners. It stands to reason that, because video games are fun and engaging, applying elements of video games to instructional contexts will therefore make instruction fun and engaging [1–3]. Though, the game label is often used as an umbrella term for a variety of implementation strategies; for example, the label might describe the technology used within the instructional program, certain rules that govern instructional activities, the manner in which learner progression is measured/tracked, or collaborative features of the instructional environment.

Indeed, the influence of games within learning environments takes many forms, including serious games, interactive virtual environments, social media frameworks, or the use of game mechanics within non-gaming contexts (sometimes referred to as gamification). Given the various strategies for the design of game-inspired instruction, great care must be taken to ensure that the impact of those strategies is positive, such as supporting learner motivation and/or engagement with instructional content. Additionally, it is important to consider the influence of learners’ existing knowledge on user-system interaction with different game design elements and technologies. Further, what considerations must be made in transitioning game-inspired strategies from traditional computer-based training to adaptive tutoring systems? The current discussion provides an overview of these topics, through a review of relevant interdisciplinary literature. This paper will offer suggestions for selecting appropriate game-inspired features in instructional design, guidance for avoiding potential pitfalls when working with game elements, and identify vectors for future research and discussion in integrating game-inspired features with adaptive tutoring systems.

Video games are often leveraged in or developed for instructional systems. Though, simply adding game software to an instructional system does not necessarily lead to benefits associated with the playing of a game. Consider the distinction between games as software products versus activities characterized as games. Juul [4] proposed a functional definition of a game which is sufficient and relevant for the current discussion: “A game is a rule-based formal system with a variable and quantifiable outcome, where different outcomes are assigned different values, the player exerts effort in order to influence the outcome, the player feels attached to the outcome, and the consequences of the activity are optional and negotiable.” The criteria are not comprehensive; rather they articulate core characteristics that are common to games. One characteristic, for example, that is conspicuously missing from this list is narrative [2]. While narrative is a powerful element of a game’s design, it is not central in defining an activity as a game.

These characteristics can also be used to identify edge cases, or activities that meet some of these criteria, but not other criteria. Children’s sandboxes, for instance, are intended to be freely explored; there are no rules that govern the activity, and the outcomes of the activity do not have defined values. A children’s sandbox might be more accurately described as semi-structured play. Even in a structured video game, players can choose to ignore rules to pursue other interests (e.g., exploration, goofing off). Thus, the context surrounding the activity can change its classification. For example, if the game, Grand Theft Auto were used as a training environment to teach students about obeying traffic laws and safe driving, then the user experience may be more simulation than game-like. Therefore, gameness, as it were, might be more accurately described as an emergent property from the interaction between the context (users and environment) and the system (software, physical objects, etc.).

To that end, game characteristics may seem to sometimes be at odds with instructional goals, where consequences of performance outcomes may not be optional or negotiable (e.g., poor grades requiring remediation), or the learner may not feel attached to said outcomes (e.g., lack of interest, non-voluntary participation). Game software can be educational by design, such is the case in serious games [2]. However, traditional games can lose their gameness as more of their defining characteristics are altered or eschewed. For instance, game-players build game expertise by learning from failures associated with attempting different strategies (including goofing-off) [5]. If failure tolerance is low within a learning environment, then the potential value added by games, like autonomy or fun, might be compromised. Just as illustrations or slide stacks are used to generate learner interest [6], games are one of many available tools for instructional design. Developers should consider the balance between the playing of a game and achieving instructional objectives, as well as how closely the two align with one another in order to effectively leverage games in instruction.

For instructional contexts in which game-playing is not desired, game-based technology components still provide rich simulation opportunities to encourage scenario-based critical thinking, exploration, and collaboration in serious settings. When leveraging individual game-based technologies outside of traditional gaming contexts, it is important to consider learners’ mental models, which significantly influence system perception and user interaction.

Rouse and Morris [7] explained that mental models “are the mechanisms whereby humans are able to generate descriptions of purpose and form, explanations of system functioning and observed system states, and predictions of future states” (p. 7). Mental models influence users’ expectations regarding a system’s functionality and guide user interaction behavior [8]. An individual’s mental model regarding a particular system is influenced by past experiences and perceived similarity of other systems to the target system. Further, human mental models do not to be complete or even accurate in order to be applied to a specific system interaction [9].

Mental model theory applies directly to learner understanding of and interaction with instructional systems. Individual mental models make it possible for a learner to perceive a system in a way that was unintended by the designer. For instance, it may prove difficult to instill a sense of seriousness in learners if they have prior experience with the playful aspects of a COTS game that was modified for use with an instructional program. Alternatively, learners can experience similar affect with a system for different reasons (e.g., reward structure, learner autonomy), or can respond the same game-like elements in different ways [10].

Regarding system usability, non-game virtual-world simulation products can choose to evoke specific interaction mental models through utilizing game elements familiar to frequent gamers (e.g., mini-map, health bar), thus potentially reducing the time needed to explain controls or rules regarding certain aspects of instructional systems. Placing heart icons at the top corner of the UI, for example, would indicate to a learner that the number of acceptable errors within a session is limited.

Users with prior game experience will expect game-based elements to function in specific ways; therefore, deviation from expected behavior is undesirable and may lead to unintended user confusion or frustration. Dedicated gamers are believed to have a set of meta-knowledge that transfers between games [5], which might include avatar behaviors (e.g., jump, run, crouch) or properties of objects (e.g., red barrels explode). This meta-knowledge also includes expectations regarding how controller buttons are mapped to user actions in certain game genres, such as pulling the trigger to shoot in first-person shooter (FPS) games. Less-frequent game players may also impose different mental model interaction paradigms, based on their own unique experiences to novel instructional systems, such as touch-based and swipe-input gestures for mobile applications.

Anticipating the way in which learners will perceive and react to certain aspects of game-inspired instructional elements, including non-game simulated environments, requires designers and developers to understand learner’s mental models. When a learner interacts with an instructional system, there are likely multiple mental models in play. The learner is working to build a mental model of domain knowledge, while maintaining a mental model to interact with the actual learning system used to facilitate the domain content, in addition to other mental models required to interact with technology systems, including games and simulations, integrated within the learning system. Streitz [11] identified the distinction between domain knowledge mental models and system use mental models as the content problem and the interaction problem, respectively. System design with consideration for users’ multiple mental models, including domain knowledge acquisition, expands upon user-centered design, as described by Quintana et al. [12] as learner-centered design.

Meanwhile, developers (engineers, designers, authors, etc.) have their own mental models of how their instructional systems should function. However, it is a mistake to assume that end-users have the same skills and expertise as those that designed and developed the instructional game and/or system. Recognizing the differences between developers’ and learners’ mental models is paramount. When possible, game-inspired instructional designs should align with existing user models, in order to reduce the time and resources needed for a learner to understand how to interact with the system. Therefore, in the same way that learner analysis is a critical step in the systematic design of instruction [13], so too is game user research (for an overview, see Isbister and Schaffer [14]) in evaluating the design implementation of instructional systems including games or game-like features.

Well-designed and tested games are successful because they are designed to engage players [1]; but, it is usually not the purpose of game-inspired instructional design to generate a commercially successful game. Games can also serve as a means to initially draw users’ attention toward a system simply based on interest in games [15]; though, it is important to recognize the difference between gaining initial interest through compelling sensory stimulation, and prolonged sustained engagement. Further, it is also not always practical or appropriate to instantiate a game or fully realized virtual environment within an instructional system. Rather, the impetus for a game-inspired instructional design might be to develop a compelling experience that engages and sustains learner motivation. With those considerations in mind, it is increasingly becoming common to see individual game design elements applied to non-gaming instructional content and systems.

4.1 Design for Motivation

Student motivation is an important driving force behind learning. For instance, motivation has been shown to have a positive relationship with performance outcomes, both in traditional [16] and computer-based training environments [17]. Prior research demonstrated that human instructors positively influence learner motivation [18], but motivating students in computer-based environments has been noted as a challenge to the wide-spread implementation of self-directed e-learning environments [19].

Additionally, the source of learners’ motivation is relevant to achieving instructional performance outcomes. Ryan and Deci [20] distinguished between extrinsic motivation, “the performance of an activity in order to attain some separable outcome” and intrinsic motivation, “which refers to doing an activity for the inherent satisfaction of the activity itself” (p. 71). Prior research indicated that intrinsically motivated students can experience greater engagement in learning [21], engage in more exploratory behaviors within instructional content [22], and can exhibit better learning and performance [23]. Some externally motivating factors can be internalized by students, to a similar or complimentary effect as internally motivating factors, which suggests that the two types of motivation are not mutually exclusive [24].

Intrinsic motivation is at the foundation of self-determination theory (SDT), which highlights the importance of self-motivation in well-being and personal growth [20, 25]. SDT identifies three fundamental psychological needs, which form the basis for self-motivation when met. These needs are: 1. Competence, or the need for mastery and accomplishment; 2. Autonomy, or the feeling that one’s actions are aligned with one’s self and made of one’s own volition; and 3. Relatedness, or the desire to connect and interact with others [20, 26].

Leveraging SDT in the study of games, Rigby and Ryan [27] described the ways in which games stimulate and sustain intrinsic motivation in players, through the psychological need satisfaction of competence, autonomy, and relatedness, respectively. For example, in a role-playing game (RPG), players’ need for autonomy is satisfied by the ability to customize the physical appearance of the player-character, and make decisions within the narrative that influence the game world. In a puzzle game, competence needs are satisfied thorough visual and auditory feedback and score bonuses or multipliers that acknowledge player mastery. Finally, player needs for relatedness are met in action games by sophisticated NPC (non-player controlled) agents that interact with the player, as well as through social aspects of gaming that occur with other players in online competitive and cooperative gameplay. Such elements can be integrated into existing instructional systems, without a complete game environment.

4.2 Game Design Elements

Gamification refers to the approach of applying game-elements to non-gaming contexts, with the goals of influencing behavior, improving motivation, and/or sustaining engagement [28]. Elements of gamification include achievements, levels, leaderboards, points, badges, and virtual currency. Gamification is applied in a variety of domains including social networking, web design, business, and learning. From a business perspective, Zichermann and Cuningham [29] described gamification as the use of game mechanics to engage users and solve problems. However, the academic validation of gamification is still in early-stages. A recent literature review revealed mixed results for gamification; the researchers found that gamification has different effects in different settings, can be polarizing to users, can encourage unintended behavior, and may undermine users’ intrinsic motivation [30]. (A broad discussion of the state of gamification can be found in Fuchs et al. [31]).

Certain game elements are sometimes implemented to capitalize on the psychology of human behavior, and do not necessarily coincide with SDT needs, described earlier. In some free-to-play and social network games, for example, real-world time limitations, carrot dangling, and social notifications serve to artificially delay or diminish reward distribution in order to compel players to return to the game in regular intervals [32]. Alternatively, these systems offer users the ability to spend real-world money to accelerate game progress [33, 34]. While the idea of getting learners addicted to frequent, yet minimal reward-loops within an instructional system may sound appealing, it is more likely students will game the system [35] in search of the next extrinsic reward satisfaction (e.g., achievements or social status on leaderboards), thus becoming disengaged with the actual instructional content [36].

In a meta-analysis conducted by Deci and colleagues [37], the researchers found that extrinsic rewards can undermine intrinsic motivation. However, the specific effects for various types of rewards (verbal, tangible, etc.) tended to vary among men, women, and children, respectively. The authors suggested that the negative impact of rewards on intrinsic motivation may be due to the nature by which rewards control user behavior and thwart user volition under certain conditions [38]. Though, extrinsically motivating features may be necessary when learners are not intrinsically motivated to interact with an instructional system. For instance, Thom and colleagues [39] examined an enterprise social network system, which incentivized participation through points and leaderboards. When those features were removed from the system, participation within the social network significantly decreased.

Of course, games have more to offer instructional systems than badges and micro-transactions. Allowing the learner to customize non-critical elements of the user experience supports the need for autonomy. Customization may take the form of the learners’ avatar, or the style and layout of the interface (e.g., backgrounds or themes). Stories and narratives can be used to help learners sequence and organize knowledge [40]. For non-sequential learning modules, allowing learner autonomy in course navigation can encourage self-regulated exploration of the instructional content; though, there is a potential danger of losing the attention of learners due to apathy or off-task exploration in open-ended environments [41].

Given the variety of ways in which games and instructional systems can intersect, game technology and game design elements offer both opportunities and risks in the design of adaptive tutoring systems. Adaptive tutoring systems, sometimes referred to as intelligent tutoring systems, have been described as “computer-based learning systems which attempt to adapt to the needs of learners” (p. 350) [42], or a computer system that customizes instruction and/or feedback to learners [43]. The system is composed of a series of models or modules that interact with one another. At the core of the system are four models/modules: the learner/student model, the pedagogical/instructional model, the domain knowledge model, and the user interface/communications model. Compared with traditional computer based training, adaptive tutoring systems possess the unique quality of adapting instruction to meet the idiosyncratic needs of the learner. Similarly, these systems can support the affective and motivational needs of the learner as well [17, 44].

5.1 Adaptive Assistance

Many games are not adaptive by nature. They do not alter or change their presentation or challenge based on the player’s performance. Typically, the player adapts to the game (not the other way around) to meet increasingly difficult challenges. For instance, memorizing the patterns of enemies and locations of power-up items are part of a player adaption strategy. Likewise, game-players are typically not given the option to skip levels within a video game based on current performance or pre-test evaluations. Single-player video games often offer a difficulty selection option prior to starting a campaign (i.e., easy, medium, hard), though the selection is made by the player and not adaptively changed over the course of gameplay. Challenge adaptation at the macro-level may not be a practical option when teaching to a criterion; however, it might be useful in the identification of novice, journeymen, and expert performers for intervention strategies and subsequent training effectiveness evaluations.

Recently, Nintendo explored micro-adaptive assistance in games with a feature called Super Guide [45]. Within some games, such as New Super Mario Bros. Wii, Super Guide offers the opportunity to access assistance after multiple failed attempts at completing a level. The automated guide helps the player in completing the level; however, using the guide prevents the player from obtaining certain achievements related to overall game completion. The logical analog to a Super Guide in adaptive tutoring might be a hinting system, where learners can request or receive a guidance after a number of errors [3]. However, less motivated learners may be inclined to game the system by intentionally making errors in order to get hints and progress through the content more quickly. Research efforts in adaptive assistance evaluate the content and frequency of challenge-based adaptive solutions in interactive learning environments so that they are not abused by unmotivated learners.

5.2 Adaptive State Management

Game-state management is another commonly employed game technology which might be closely coupled with adaptive assistance in intelligent tutoring systems [46]. Many video games, for instance, provide the ability to create multiple save files during play, and then load those game states at some point in the future in order in the event that the player fails, or wishes to explore alternate strategies. With respect to adaptive tutoring, game-state management might be leveraged in teachable moments to repeat key learning situations [47]. Archived save state data might also be used to replay critical events as a part of an after action review (AAR) procedure.

There are a number of research challenges in this area. Adaptive tutors need to be able to manage the data associated with saving and loading world-states, and may also need to maintain a meta-awareness of the current world state, which states were accessed, and how often the feature was used. Further, there are user research questions regarding whether learners should be able to save their own world-states, if the system should automatically save world-states based on some set of criteria, or if a hybrid approach is more appropriate.

5.3 Adaptive Gamification

Gamification is the use of game elements in non-gaming situations. Used in this manner, gamification can direct user behavior toward specific goals or actions via extrinsically motivating features. However, some research suggested that gamification features can be perceived as controlling, which undermines users’ sense of autonomy, therefore negatively impacting intrinsic motivation [37]. With respect to adaptive tutoring systems, an opportunity exists to implement gamification features within an instructional system in situations only in which they are deemed necessary.

Future work should investigate the impact of various game-inspired features among different types of learners. For example, what is the differential impact of badges and achievements on high-interest and low-interest learners, respectively? External incentives might support students with little interest in the domain content [2]. The results of such studies might be used to augment the types of tactics that could be implemented within an adaptive tutoring system based on learner model data, including learners’ interest in the domain content and motivation to learn or interact with the tutoring system.

5.4 Adaptive Team Training

Team training is another area in which game technology can support adaptive tutoring efforts. With the rise of online gaming, particularly in the console gaming space, significant research effort has been placed in matchmaking. Matchmaking technology intelligently assists players in finding teams to join and other teams to complete against in matches that will yield a positive experience. Players are believed to have an enjoyable experience if the skill balance between teams is evenly distributed, and the chance of either team winning an online match is considered somewhat even [48]. Scientists at Microsoft Research developed a system called TrueSkill, which is used in competitive matchmaking games on Microsoft’s online gaming service, Xbox Live. TrueSkill maintains the skill-rating of individual players over time in team-based or free-for-all matches based on the calculation of an average score and an uncertainty modifier [49]. With respect to adaptive tutoring, a similar approach can be leveraged to construct teams and analyze team/individual performance using data collected and stored within long-term learner models.

Matchmaking is not limited to skill-based metrics. The technique is also used to assemble teams based on the player’s team-role preference. Specialized team roles in games are becoming increasingly common. For instance, the military-inspired FPS game, Battlefield (EA Games), allows players to assume squad roles such as recon, engineer, or medic. With respect to adaptive tutoring, matchmaking could be used to assemble teams of distributed learners based on team role, or to pair novice learners with journeymen learners to encourage learner collaboration. Such specialization data might also be found within the learner’s profile, or long-term learner model.

The purpose of the current discussion was twofold. First, a review was provided of relevant, interdisciplinary literature regarding the various means by which games intersect with computer based instructional systems. Games used in learning environments, in part or in whole, can influence users’ mental models and affective states, regarding learning. Instructional designers and developers should seek to understand the learners for which the system is designed in order to achieve the desired outcomes resulting from learner-system interaction.

Secondly, the current effort identified potential opportunities for research in implementing game technology and game elements within adaptive tutoring systems. Game technologies and design elements have been refined over many years through iterative console development cycles and game releases. However, implementing features such as adaptive assistance, gamification, or matchmaking, requires continued research into areas such as information management, learner modeling, and machine learning, respectively. Research is also needed to examine the differential effects of including specific game features, both isolated and in combination, within adaptive tutoring environments across different groups of learners.

Finally, no matter how game design elements are incorporated into instructional systems, games should not be a panacea for sound instructional design. Games are designed to be fun [50], while gamification leverages quantification metrics and social status to incentivize non-game activities and drive user behavior toward specific goals [51]. As such, neither games nor gamification is a substitute for instructional content. As M. David Merrill [52] stated, “a horse led to an empty well will still die of thirst if there is no water in it.” With that in mind, adaptive tutoring systems should continue to evaluate the effectiveness of game-based design approaches to ensure a positive learning experience and that system features align with instructional objectives.