Insights into Applying Artificial Intelligence Methods in Action Video Games for Enhancement of Psychomotor Skills ^†

A new direction has been emerging for experimental psychology that searches for different skills that may be enhanced by playing a video game. Furthermore, questions arise as to how these skills are being improved and what the game features are that are causing such effects. This leads to the requirement of having a specific understanding sof the different genres of video games that may be inducing positive effects. An action video game may be interpreted in different ways, but our team sticks to the understanding that a game that offers challenges that require hand–eye coordination and quick reactions is an action video game (AVG).

This article links studies that point toward the specific AVG features that may be beneficial to the emerging effects of cognitive enhancement. It also presents an idea for personal development that combines specific AVG features with serious game features. A short general direction for game design that avoids addiction is presented. The need for avoidance of specific video game features is described as well. An analysis is presented on the artificial intelligence methods that can be beneficial for personal development.

In the last section, this work describes an experiment with the authors’ method for online machine learning that applies a genetic algorithm for producing different ways of playing with the agents. The latter are player-imitating program entities, the target purpose of which is to be useful to the players for cognitive enhancement. The produced data will be analyzed in the search for a correlation between the game performance results of bots and players.

By providing hand–eye coordination challenges, an action video game (AVG) might be a tool for improving different cognitive skills. A study by renowned researchers Bavelier and Green in this field presents the idea of “learning to learn” [1]. The term is associated with “transfer effects” and describes the effect of faster learning after learning a new task. This may lead to the statement that an AVG that offers different tasks in its gameplay may be beneficial to such transfer effects. However, having a big variety of tasks will most probably dissuade learning [1,2].

A cross-sectional study by Bediou and Rodgers [3] showed that players who play AVGs have better cognitive skills than people who do not play games, and this effect is quite big: g = 0.64. AVGs have been supported by several studies suggesting that they could enhance different cognitive skills and that they can be applied as a training tool [1,2,3,4,5]. A reduction in attentional blink effects as a consequential training effect could be beneficial for different everyday activities. It can be interpreted as an improvement in the skill of being able to process visual information more effectively when there is a rapid change in visuals [4,5,6,7]. In close relatedness, the application of an AVG training program was particularly focused on enhancing the contrast sensitivity function in a study from 2009 [8]. Describing it as “the very act of action video game playing”, the authors pointed to this method as a means of eyesight improvement. The concept of sensorimotor learning should also be tightly related to enhancement via AVGs because of the studies in the sphere of experimental psychology, like the study by Gozli and Bavelier [7]. Their research showed that players who play AVGs, even mainstream ones like Assassin’s Creed or Call of Duty (popular examples of AVGs), have far greater sensorimotor skills compared to non-video-game players. Learning new sensorimotor skills, however, was not shown to result in a great outperformance of the AVG players. If sensorimotor skills are more effective in AVG players, then it might be possible to design a training program with an AVG which might be favorable for the development of skills required in many areas of society. The application of an AVG training program could be beneficial in professions or activities like wood carving, surgery, or others that specifically require sensorimotor skills.

Another term for sensorimotor skills is psychomotor skills, and it is also associated with enhancement via a video game [9]. This study presents a game which is not specifically described as an AVG, but, judging by its gameplay, it is assumed that it follows the requirements of our team’s definition that are stated in Section 1. This study uses a real-life building block task for measuring the effects of playing the “ad hoc game”. The results show that the experimental group was faster in completing the task, and, thus, it is assumed that their eye–hand coordination skills are improved by playing the game.

Even though it is a new concept, the term serious game can describe different genres of video games. Generally speaking, it corresponds to a video game, the main purpose of which is different to entertainment. Most of the video games today are created to be engaging, with a variety of graphics content and game features, which may lead to addiction. The Video Game Addiction organization (video-game-addiction.org) suggests six game elements as “hooks” which may lead to addiction. The “Discovery” and the “Role-Playing” hooks are associated with emotional attachment to the world and the characters of the game. This is related to one critical phenomenon that is described by several studies on understanding the symptoms of video game addiction—the psychological escape [10]. This is tightly associated with preoccupation with gaming and may lead to depression as “… fictional life can become more fulfilling than their real life” [10]. These problems require consideration if a video game would be considered a serious game for cognitive enhancement. A supposition can be made that if high-definition graphics footage and a big variety of character design choices lead to preoccupation with gaming, then having fewer of such game features would be beneficial for avoiding addiction. Another problem that comes with addictive gaming is the aggressive behavior that can be adopted because of too much playing of games with graphic violence [11]. The presented problems were transformed into challenges for our team that led to the search for definition of requirements for the design of an AVG that can be specifically developed for cognitive enhancement. The term Digital Training Game emerged, which is additionally associated with the ideas of the players performing physical exercises mostly with their hands in specific stages in the game. They are not discussed in this work, as the topic is oriented towards the actual gameplay features that can be beneficial for cognitive enhancement. In another work of our team’s second experiment of the first working package, a general conceptual model of attention is presented. Such a conceptual model could be used as an understanding for the players’ subjective experiences happening in specific game events. The latter are viewed as the “intrinsic exercise” that could be beneficial to cognitive enhancement. Therefore, such game event experiences have to be further investigated in order for solid proof to be presented. The intrinsic exercises are considered by our team as very important because they are narrowly related to the understanding of action game elements. They can be viewed as challenges that the players decide to overcome in order to experience satisfaction from an accomplishment in the target game. The players are not pushed to fulfil the task that is demanded by the challenge, as the challenge is not declaratively proclaimed. The game actions that are executed by the players are, rather, learnt in the process of the gameplay, which is why the exercises are named intrinsic. This idea has been implemented in the target game and is described in an article from 2023 [12]. A notable example of an intrinsic exercise is the so-called “relative striving”, which is explained in Section 4 of this work. The exercises can be viewed as “necessary inducement of empathy as the most effective way to understand where a player is going …” [12].

The concept of serious game is regularly related to video games made for learning. As such, the understanding of metacognition is also tightly associated with the genre as being beneficial to learning [13]. The presented book solidifies the genre of digital-learning games that attempt to “target the acquisition of knowledge as its own end and foster habits of mind and understanding that are generally useful or useful within an academic context” [13]. A Digital Training Game can be related to digital-learning games, but as being specifically concerned with “fostering habits of mind” and improving one’s understanding of their own self that could be generally useful for variety of learning activities.

The field of artificial intelligence provides a wide range of tools and methods that can be useful in the design and development of serious games [14]. What is important to pay attention to are the goals of serious games, what results they aim to achieve, and how to achieve them. There is a plethora of research and developments that demonstrate the effectiveness and benefits of using artificial intelligence methods that can aid in personal growth within the realm of serious games.

A paper by M. Frutos-Pascual and B. Garcia Zapirain [15] focused on discovering algorithms from the field of artificial intelligence that are useful in the development of serious games. As mentioned in the review, several examples of algorithms are the following: decision trees, naive Bayes classifier, artificial neural networks, case-based reasoning algorithms, and support vector machine.

A research team consisting of J. Perez, M. Castro, and G. Lopez presented a scientific paper [16], the main goal of which was to demonstrate the synergy between serious games and artificial intelligence as such that opens new horizons for studying human behavior.

In a scientific paper [17], developed by a group of researchers, natural language processing (NLP) and its potential use in the field of serious games are mentioned. The results from the conducted studies indicate that NLP is a good choice for improving the quality of assessment of players’ verbal and written expressions. Some drawbacks are mentioned that may hinder the use of NLP in the field of serious games, such as significant preprocessing or post-processing requirements.

This brief overview of popular methods from the field of artificial intelligence is only part of the wide area of possibilities available to support the development of serious games that aim at personal growth. Additionally, it can be said that another incredible opportunity arises for utilizing gathered data to assess the player’s personality with the help of such methods.

A method for machine learning (ML) has been developed for a multiplayer action video game (AVG) [18]. The genre AVG demands a software architecture that works in a frame loop—it processes each state of the online game environment that corresponds to a frame of the game software. The ML method is based on a genetic algorithm idea, which is implemented as a process that is being executed in the ML system of the software of the target game. Another article describing an application of genetic algorithm (GA) in a multiplayer video game [11] is also focused on the same target game that is used in the experiment presented in Section 5 of this work. The article from 2023 describes the sequential operations that are executed on each processed frame by the game loop of the software. Figure 1 depicts the latest version of the ML frame loop. The consecutive steps represent the ML method itself, while the GA process is executed during the ML system step. A rational agent is represented by a bot that recreates a player in the game [18]. A bot could be described by the three top-level classes (modules):

The previously presented method for ML was designed to be applied in an online environment with a teacher—a bot that does not have its behavior changed (does not learn) and a student that learns. The teacher bot uses a specific ML model, to which different solutions of adversarial behavior are generated by the student. No players were part of the supervised learning process. The models produced by the previous version of the ML method were planned to be used in future ML techniques with actual players. This was fulfilled in the experiment presented in Section 5 of this work. At the current state of the project, a specific game mode exists in which the player’s client activates a bot as an opponent in the opposing team in the game match. The ML method is now applying the initial population of genetic individuals—the initial models as starting ways of playing. Based on the events produced by the world event observing system (Figure 1), the ML system stores required data during the fitness period (Tf). For every Tf a model, a genetic individual (GI) is applied by the bot as a way of playing. In this experiment, Tf was set to 15 s. When the Tf passes, the fitness points are calculated and stored in the model data structure. When all the GIs are evaluated, a new population is generated based on the configuration of the GA. With the new features of the software, the GA algorithm is executed for every bot that is activated by every player. Furthermore, multiple players can join the same server room. This means that a behavior model of a single bot depends on the behavior models of multiple other bots or players that have been met during the evaluation of a population of GIs. The calculation of fitness points of a GI depends on the positive or negative events that have been recorded while playing against one or multiple targets during the Tf. This way, the ML method could be interpreted as an online ML, in which the live data are being produced by one or several unexpected opponents that could be either bots or human players.

The Bot Will module (Figure 1) is responsible for applying the current GI as a way of playing. The behavior of a GI can be interpreted as a reactional behavior that produces a change in the current acting state of the bot when a given type of event occurs. Being part of an online multiplayer environment, a bot cannot react to all of the occurring events. The Bot Will module applies a set of rules that dictate the time for which a specific reaction is executed. Also, producing a reaction to a specific type of event could be more important than another. For example, reacting to movement is less important than reacting to projectile generation. A reaction corresponds to a change in the current acting state of the bot. Such a change, dictated by the synchronizing component of the Bot Will module, results in informing the server via a Transmission Control Protocol (TCP) packet (Figure 1). This way, a GI solution can be viewed as a totality of instruction sets for reacting to types of events.

A GI solution is a totality of four containers, each of which has nine genes. An example of the gene container used for reaction to projectile generation is presented in Figure 2. A gene could be one of the nine possible directions that a game agent (a player or a bot) could have at a given point of time in the two-dimensional game world. The directions, presented clockwise, are up, up-right, right, down-right, down, down-left, left, up-left, and standing still (null direction). Each projectile is generated for a time Tg, after which it is thrown by the controlled characters in the game. The directions of movement of the projectiles are not limited to the nine directions of the genes. A projectile has to be landed on an opponent character in order for that opponent to take damage. After being hit several times by a projectile, a game agent is taken down (falls). Being a critical event, a game agent needs to dodge enemy projectiles so as not to fall. That is why one of the types of world events that are recorded by the observing system is the event of a projectile generation.

An example of a reaction to such event is presented in Figure 3. This example corresponds to the example of reaction to projectile generation in Figure 2. The so-called “strive vector” (Vs) is produced when the event of generation starts. Vs remains immutable and the opponent of the target (in the example case, the bot) does not know towards where Vs is pointing. After Tg passes, the projectile starts moving with a high, but decreasing, speed, on a straight trajectory corresponding to Vs. During Tg, the game agent that generates the projectile can move. That is why in a position like situation 3, depicted in Figure 3, the target would normally start moving with the direction of its opponent (the bot) in order to land its projectile when Tg ends. That is why the term strive vector emerged—the game agent that generates the projectile strives to keep its Vs directed towards its opponent.

The Bot Mind stores the world events that particularly belong to the currently set opponent target. If having multiple opponents in the surrounding area, the bot is only reacting to the events produced only by its target. That is why it is very important for the “target search and set” component of the Bot Mind module (Figure 1) to be setting the best target. The Bot Will module uses the GI, more specifically its four containers of genes, to eventually produce a change in the bot’s acting state. A container of genes is used by the “act upon current acting state” component (Figure 1), when a change in the acting state is going to be made. Each gene container is used for the production of a reaction to a specific type of event. Not all gene containers are presented in this work. In the future, more gene containers are planned to be designed in order for a more complex behavior of the bots to be formed.

For the configured fitness time, a given bot fulfils a behavior based on a specific GI. After recording all the negative and positive events for the fitness time, the ML system switches to the next GI, setting it as behavior of the bot. This means that a bot has a specific playstyle for the period of the evaluation—Tf. After all GIs are evaluated, a new population of GIs is formed. The two best GIs (parents) that had the highest fitness points in the previous population are kept. The third best is also kept, but with 55% of its genes in all of its gene containers mutated. Three children are produced by the parents. Each of them has all of its four gene containers formed based on the three following methods for crossover:

Child 1: Left side of the genes from the first best and right side from the second best.

Child 2: Left side of the genes from the second best and right side from the first best.

Child 3: The genes that the first best has but two of the genes from each container are mutated.

This means that each next population of GIs is presented by the two parents (the two best from the last population), their three children, and the kept third best from the last population with its 55% of genes mutated. The GA process executed by the ML system of the Bot Mind module depends on part of the events produced by the world event observing system (Figure 1). They are divided into positive and negative:

It is important to note that these events do not depend on whether or not they are associated with the current target that the bot has. For example, a given projectile that the bot throws may take down an opponent that is not the current target, but is still recorded as a positive event. The presented events are the only ones that serve as parameters to the fitness function: where

F: fitness points;

Dd: damage done;

T: takedowns;

Dt: damage taken;

F: falls.

The key difference of the new version of the ML method is that other players may join; furthermore, they can join a team of their choice. This allows cooperation to be made between players and bots against other players and bots. This creates an enormous number of combinations of playstyles that can be generated by each bot according to different behaviors. This means that the GA is going to be directed towards finding the best combinative solution of playstyle against different opponents.

The conducted experiment was a sequence of five game matches, one of which was played per day for five consecutive days via the target game that is recognized as a Digital Training Game. The recently developed game mode, played in an Ethernet environment by three players, is expressed by the feature of the game client to activate a rational agent—a bot that is placed as an opponent of the player in the opposing team. A game match is fulfilled by two teams of game agents that can be a combination of bots and human players. Each game agent is represented by a single character that is controlled by a player or a bot. The decisions are made in the real-time game environment by the game agents and are mainly represented by moving in one of the nine directions or by generating and throwing projectiles (described in Section 4). The two teams have a single stationary structure—base structure, near which the ally game agents are spawned in the game world. The area of the game map, where the team characters are spawned, is considered a team base. The goal of each team is to destroy the enemy’s base structure by attacking it with projectiles.

Each game match was considered a daily session of training for bots and players. The game was configured so that each of the game agents were able to use a specific set of four abilities. When the latter is activated by a game agent, a short generation period (Tg = 0.25–0.72 s) passes, after which an event is produced. While generating an ability, the game agents can move but cannot generate other abilities. The duration of Tg depends on the different abilities that were configured, three of which were for projectile throwing and one for character healing.

The duration of a game match depended on the skills of the players and the strategies that they implemented. As for the bots, each training session started with the same initial population of genetic individuals. Their ways of playing depended on the playstyles and strategies that the different players were implementing. Each training session was conducted in an online environment and players were able to talk with each other via the additional communicating software. Each meeting was established in almost the same time period of the day—in the late afternoon around 5:00 p.m. The machine learning (ML) method allows different parameters to be set and applied on each rational agent (bot). All of the bots had the same ML parameter. The first day was dedicated to presenting the game to the players that did not know it and to decide the values of the ML parameters. The most important ML parameters in terms of the genetic algorithm process that are tightly associated with the results presented in Section 6 are described below:

The client (the target game) is run on the personal computer of each player. It instantiates a bot and connects it to the online server room. Each client is responsible for the genetic algorithm process of the bot that has been activated as an opponent for each player. The ML method’s steps of evaluation and population generation were identical—the software versions were the same for all the players in the experiment.

Experimental setup:

Participant’s setup:

Player A has been playing the target game before but has not been playing for a while. Furthermore, player A has not been playing in the specific participants setup, particularly with the configured abilities set. Players B and C are familiar with the game, but do not have that many game hours with it compared to player A. Another important fact is that player B is an action video game player, while player C is not. This was confirmed by the analysis of the results from the game server data. Following these player characteristics, we defined three player types that might be useful for future research:

Other experimental settings include the presented circumstances below:

Game circumstances:

Figure 4 presents the game results that were recorded on the server at the end of each daily game match, with the training day excluded.

Many observations on the provided data point towards the statement that players have learnt to play the target game “Hram Light” improves with the progression of the days. This is going to be referred as the game learning (GL) phenomenon. Probably, the most noticeable measurement supporting this statement is that of the decrease in the match duration. This means that less time was needed for players B and C to take down the enemy base structure and win the game. It is important to note the experimental requirement that the players are not playing any other video games during the 5-day training session. Also, during the four days, they are not playing more than one game match of the target game. The training day was fulfilled by two game matches in which players played cooperatively against three bots.

The GL phenomenon is also supported by the progression of the two teams’ damage done (DD) and damage taken (DT). As a Target Game Player (TGP), player A helped gain the high total DD (positive score) of team 1 in the first day and relatively low DT. With progression of days, the DD/DT ratios tended towards equality. The loss of team 1 in day 1 and, overall, the 1:3 match points in favor of team 2, is explained by the fact that the rational agents (bots) still require improvement. Decision making, like choosing the right enemy and directing towards the best place to heal in the game world, are yet to be developed as higher-order decisions. This can be confirmed by the fact that even the Non-regular Video Game Player (NVGP) fared relatively well against the two bots in team 1. Also, it was concluded that role A, which team 1 played, is not well balanced in terms of benefits compared to role B. The slightly higher distance of throwing did not compensate for the faster movement of role B’s characters. However, bots of team 1 found a way of playing that allowed them to have more takedowns than players from team 1 on several occasions. This is easily noticeable in the match data produced on day 3. The GL phenomenon is further confirmed by the fact that the Action Video Game Player (AVGP)—player B—improved their game score and overall statistics with the progression of days. This can be observed by analyzing the charts presented below (Figure 5, Figure 6, Figure 7 and Figure 8).

An interesting finding was that with the progression of days, the players found a strategy that helped them to take down their opponents’ base structure quicker. This explains the high decrease in the game match duration with each day. This strategy, on the other hand, seems to have helped the bots in their team to find more effective ways of playing. This can be supported by the fact that on day 3 the TGP had a lower takedown/falls (T/F) ratio but bots had higher (see Figure 4). Also, in days 4 and 5, the bot of team 2 had a higher T/F ratio than the NVGP—player C, which might be because of the vivid GL phenomenon observed in the AVGP—player B. This can be noticed after day 3, which was the last day, on which player B had a lower score. On days 4 and 5, player B had both the highest T/F and DD/DT ratios in his team. By observing the gameplay of the participants, it can be concluded that having lower T/F and DD/DT ratios does not mean that the given player performed badly. An important facet of decision making, which led to winning, was to play in an attacking way, sacrificing oneself to do more damage to the enemy’s base structure. Bots were designed towards playing as a defending trainer and were not that self-sacrificial. Also, it has to be considered that only two from the four chosen abilities (see match circumstances in Section 5) could cause damage to the enemy’s structure. Furthermore, they caused less damage on the structure compared to characters (bots and players are playing with the same type of characters). This explains the low T/F and DD/DT ratios of player C in days 4 and 5. A requirement was made for the next survey to provide data on the DD on the enemy’s base structure so that the self-sacrificial playstyle can be observed.

It was thought that evaluating specific playstyles for 15 s (evaluating genetic individuals) may not ensure the effectiveness of the genetic algorithm. However, the bots had relatively similar performances compared to each other. This can be observed by the similarity of DD/DT and T/F ratios of team 1′s bots, shown in Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8. This can support the statement that the developed genetic algorithm and machine learning method is applicable in such types of setup where players play cooperatively with bots.

Another interesting finding was that the hit streak (HS) event decreased in the TGP—player A with the progression of days. An HS event is recorded when a character lands thrown projectiles at enemies three consecutive times. The AVGP had its HS score dramatically increased after day 3. This shows that the player found a beneficial playstyle with the thrown projectiles (the chosen game abilities set). An HS event is not recorded when characters land projectiles on the enemy’s structures. All of this suggests that the AVGP was better at finding beneficial playstyles quicker than the NVGP in terms of hitting moving targets. Even though player A is a TGP, more defending of the team’s base structure was required with progression of the days. This, on the other hand, required throwing more projectiles that were not hitting the incoming attackers but, rather, stopping them from invading the base structure; thus, fewer HS events were produced for player A.

For our team, important directions were found during the search for beneficial artificial intelligent elements in serious games. In striving towards combining action game elements that are related to cognitive enhancement, specific design requirements were presented. An experiment was conducted in which players played cooperatively with other players or with bots. Game match results from short training sessions (5–15 min) in a 4-day sequence were produced. They confirmed that the presented genetic algorithm and the online machine learning method can be applied in a complex participant training setup where bots can produce a variety of playstyles. The latter were generated upon a mix of other playstyles and were more successful in situations of defending the team’s base structure.

Improvements are planned in terms of developing higher-order decision making and more applications of rational behavior based on the direction genes. Our future work will analyze the genes of the successful genetic individuals recorded after each sequential population evaluation. Our team has also set the goal to examine the effects of hand–eye coordination and time perception skills in future experiments on consecutive days of training with many players and bots online.