Game-based learning and Gamification to promote engagement and motivation in medical learning contexts

Smart learning environments can be defined as systems aimed at proposing innovative uses of emerging pedagogical approaches and technologies to support effective learning experiences. In the past years, rather than designing and developing even more advanced technological solutions attention has been focused on defining environments that adopt appropriate strategies to sustain student motivation and engagement. Game-based learning and gamification approaches could be a promising solution, since there is much experimental evidence that proves their effect. In this context, our research aims at defining and developing a Smart Learning Environment able to improve engagement and motivation by means of game-based learning and gamification approaches. In particular, the paper presents two serious games that, using the gamification dimensions, aim at sustaining engagement and motivation in learning processes in medical contexts. In particular, the games involve both the patients, who have to acquire knowledge and skills about their disease, in order to become responsible for their choices, and the medical and paramedical staff, who must acquire knowledge and skills about diagnostic procedures, therapeutic interventions and follow-up of patients. Some results of a user test show that the games enhance student motivation and this means improvement also in knowledge acquisition.

The debate on the meaning of smartness in Smart Learning Environments is still open. Firstly, it was meant as the ability to provide learners with adaptive and personalized pathways, to provide users with the most appropriate content or service (Hwang 2015). Then the focus was shifted to the interplay of technology and pedagogy to define effective and efficient learning environments (Chen et al. 2015; Lee et al. 2013). Now a third dimension is emerging: to allow the learner to become the main actor in her/his own learning process and to maintain a high level of engagement, motivation and involvement during the whole learning process (Frankl and Bitter 2013; Giovannella 2014; Spector 2014). Indeed, the literature has shown that effectiveness and efficiency of training courses are essential but no longer sufficient conditions in traditional learning environments and, even more, in informal learning contexts (Billett 2001; Boud and Middleton 2003; Carini et al. 2006; Dabbagh and Kitsantas 2012). Therefore, it is necessary to implement tools able to improve the engagement of learners (Zyngier 2008).

Traditionally the learner’s engagement problem has been faced from a pedagogical (Smith et al. 2005) and psychological (Finn and Zimmer 2012) point of view, while in the last year, with the spread of technology in education, it has also become a technology-related problem (Chen et al. 2010; O'Brien and Toms 2008). From the technological point of view, we are used to interacting with smartphones and other so called “smart technologies” but, as pointed out by Spector (Spector 2014), the innovative technologies in education have the potential to become smart only if they can be used in a smart way. Every technological tool can add “smartness” to a learning environment, but it has to be grounded on a solid pedagogical basis to be effective.

In our view, a smart learning environment has to combine the smart technologies with the proper pedagogical approaches to promote engagement. A deep understanding of how engagement affects the learning experience is required to reach our goal. Despite the wide agreement on the importance of engagement for successful learning and positive achievements (Finn and Zimmer 2012; Trowler 2010; Zyngier 2008) it is hard to find in the literature as much consensus on its definition (Eccles and Wang 2012). Engagement is often seen as a multidimensional construct and a large body of research is based on the definition of its three components: Behavioral engagement, Emotional engagement and Cognitive engagement (Fredricks et al. 2004).

Furthermore, engagement is often seen as being strictly related to a motivational aspect. Sometimes the terms are used interchangeably, while some authors take them into account as dependent constructs (Reschly and Christenson 2012). Engagement can be seen as the manifestation of student motivation (Schunk and Mullen 2012; Skinner et al. 2009) or as a multidimensional construct that includes both cognitive and motivational aspects (Bangert-Drowns and Pyke 2002).

This point of view is the perspective we adopted in our research: we address this multidimensionality using game-based learning and gamification in the specific context of health education, where engagement and motivation are strictly related (Baranowski et al. 2015; Ruiz et al. 2006) and are both influenced by the goals of different stakeholders. Indeed, education in medical contexts has two different purposes: physicians training and patients empowerment. In the first case, the learners are motivated to acquire skills, knowledge, values and attitudes, in order to be able to take care of their patients. But medical education is a continuous process that also takes place in the workspace and requires high levels of individual engagement (Swanwick 2005). In the latter, patient empowerment is an even more complex process: it requires patients to acquire knowledge and skills regarding their disease, in order to become responsible for their choices. Learning in this context can be painful, aversive or boring, but it is required to guarantee good quality of life, in particular in the context of chronic diseases, where the adoption of a healthy lifestyle and an adequate self-management of the disease are as important as the adherence to the medical therapy (Kato 2010).

Even if patients training is the responsibility of health-care providers, it requires the patient to be highly engaged, in order to become responsible for his/her health status (Aujoulat et al. 2007).

In both cases, motivation and engagement need to be sustained through appropriate strategies, such as clarifying goals, assessing and rewarding achievement, providing regular feedback, and, employing educational methods to actively engage learners (Mann 1999). Videogames offer exciting, innovative and highly effective methods for increasing knowledge, delivering persuasive messages, changing behaviors, and influencing health outcomes (Baranowski et al. 2015). To this end, the game-based learning approach has received great attention during the years, while in the last few years the adoption of the gamification approach has spread.

In this context our research work aims to define and develop Smart Learning Environments as web learning environments able to improve user engagement by means of game-based learning and gamification approaches. In particular, the paper presents two serious games that aim at sustaining the engagement and motivation in learning processes of professional workers, patients and caregivers, using the gamification dimensions.

The paper is organised as follows: Section 2 describes game-based learning and gamification approaches used to design and develop our serious games. Section 3 illustrates different solutions proposed in the literature for game-based learning in e-health contexts. Section 4 describes both the defined serious-games. Section 5 describes a user test to evaluate the impact on both cognitive and motivational aspects. Finally, some conclusions are drawn and future works are discussed.

As widely recognised, the approach based on games makes the attainment of educational objectives and the learning process easier, more student-centred, fun, interesting and more effective (Lepper and Malone 1987; Papastergiou 2009; Prensky 2001; Rieber 1996; Rosas et al. 2003). Several works have investigated the reasons that make games such a powerful learning environment. Papastergiour (Papastergiou 2009) states that games are successful since they support multisensory, active, experiential and problem-based learning. They promote the recovery of prior knowledge, because players must use previously learned information to improve their score in the game, and provide immediate feedback that allows players to test different hypotheses and learn from their actions. Games also provide self-assessment tools such as the mechanisms score and the achievement of different levels, and use the social dimension to involve the community in the learning process. Furthermore, besides the acquisition of knowledge, the game promotes logical-mathematical and critical thinking and the development of personal and social skills, language abilities, communication and collaboration skills, creative, and problem-solving capabilities (McFarlane et al. 2002). In addition, games support pedagogical principles (Oblinger 2004) such as:

• individualization: the level of the game is tailored on the basis of the player’s abilities;

• feedback: immediate and contextualised feedback is supplied during the game session;

• active learning: the game engages the player in active discovery;

• motivation: the players are engaged in pursuing a goal;

• social: the game is often multiplayer or social;

• scaffolding: players are gradually challenged because they cannot move freely among the game levels;

• transfer: the game fosters the ability to transfer learning from the game context to a real context;

• assessment: the player can assess the acquired knowledge or skill with the other players.

In recent years, interest is growing in Gamification, defined as the use of game mechanics and rules in non-gaming contexts (Deterding and Dixon 2011; Zichermann and Cunningham 2011).

In other words, Gamification proposes to use game thinking and dynamics to increase users’ engagement and stimulate their active participation, thus enhancing the outcomes. This approach has been widely used in the e-commerce domain, where a Gamification experience has high impact on the “FOUR Is” (Epps 2009): Involvement, since the game improves the active participation of the players; Interaction, since the game assures a high level of interactivity; Intimacy, because the game stimulates the familiarity with the brand; Influence, because the game allows the spread of the brand and the products.

These outcomes are desirable also in education, where interactivity, active participation and competitiveness are essential, in order to motivate and engage the students. However, in order to fully understand the power of gamification in education, it is necessary to study in depth the game elements that can be used and how they can support the learning process.

The main elements that can be taken from the game and used in the learning context, are (Bunchball Inc. 2010; Simões et al. 2012):

• Status: the acknowledgment of the user reputation leads her/his fame and prestige in the community;

• Recognition of results: the use of points and levels to keep track of achievement and progress is useful to maintain interest and encourage a greater commitment to higher goals;

• Competition: the comparison of the players’ performances is a motivational element that can be achieved for example with charts that allow the player to view the results and the winners to celebrate;

• Ranks: the measurement of all participants’ progress and their achievements can be used to encourage players to do better, driven by the desire to improve their position;

• Social dimension: the activities of friends can influence those of other users, in real life as well as in virtual communities. Gamification initiatives must therefore be able to create a strong sense of community.

• Customization: the game elements and feedback can be customized, thus promoting a sense of belonging to the “game” and to the community.

• Scores and Levels: scores can be used as rewards for users’ progress and for achieving the objectives in various stages. Collecting points will allow access to higher levels, defining the degree of skills achieved by each single player.

• Reward: the reward for obtained results can be real or virtual and be used to increase the player satisfaction for having achieved the desired objective. It triggers the motivational mechanisms necessary to reach new and more ambitious goals. This requires an increased involvement of the users who will be encouraged to offer even better performance.

All these elements can contribute improving the engagement in learning activities and then motivation to acquire new knowledge. However, in order to transform Gamification into an effective learning experience, it is necessary to investigate how it can be combined with the dimensions - cognitive, emotional and social (Illeris et al. 2002; Lee and Hammer 2011) - involved in the learning process.

In the cognitive area, gamification can provide challenges which are perfectly tailored to the player’s skills. In fact, the difficulty of the challenges increases as the skills of the player increase. Moreover, proposing specific problems and encouraging learners to set suitable sub-goals for themselves are features that motivate the players.

In the emotional area, the game, as well as the gamification, involves a wide range of user emotions: curiosity, frustration and joy. It often provides positive emotional experiences and, in the case of negative ones, it encourages the players to use and transform them. For example, usually the game implies repeated failures and often the only way to learn how to play is to repeat the game several times. Each time the player learns something. In order to maintain a positive relationship between the player and his failures, the game provides rapid feedback to encourage the player to try again until he succeeds. In traditional learning feedback is not immediate and the students have few opportunities to try the exercises and when they can, the risks are high because of the teacher monitoring. The result of this process is anxiety and a desire to escape. On the contrary, game-based learning and gamification can create a learning environment in which any attempt is rewarded; in fact, when a failure occurs the player can correct his mistakes over and over again. Thus, the defeat becomes a necessary part of the learning process and students can see the failure as an opportunity, rather than being overwhelmed and feeling helpless and fearful.

In the social area, the players can experience new identities and roles; they have to make decisions according to different points of view, taking on roles that can be imaginary or realistic, according to the situations or environments in which they act. Moreover, the games allow publishing and sharing of the achievements, which otherwise might remain invisible. In traditional learning, a teacher provides the recognition, but in the gamified environments players can reward each other, encouraging participation in the class/community.

In conclusion, it is possible to state that the use of playful elements in learning contexts does not aim to replace textbooks but to promote a powerful learning experience through which to develop knowledge, skills and / or abilities in a learning environment characterized by a high level of active involvement.

This can be particularly useful in medical education contexts, where time and effectiveness of learning are critical, for both the training of professional workers and for the empowerment of patients and caregivers.

As previously mentioned, game-based learning and gamification can be particularly useful in those contexts in which education is perceived as secondary by the final users, as happens in the medical domain. For this reason, different solutions aimed both at professional workers and patients have been developed over the years. The solutions described in the following are only a selection of those available on the Web.

Touch surgery (http://seriousgamesmarket.blogspot.it/2014/06/serious-games-guide-surgeons-through.html) (https://www.touchsurgery.com/) is a serious game on mobile devices designed for medical school students, in particular, that guides the player through a series of surgical procedures, using a process called Cognitive Task Simulation, which fragments each operation in its early stages and key decision points. It is based on the assumption that surgical performance is achieved by the combination of effective decision-making and technical surgical skills. The system keeps track of the scores achieved and allows them to be shared.

Septris (http://med.stanford.edu/septris/) is an educational tool developed by Stanford University that aims at training emergency physicians and nurses to classify the epidemiology of the sepsis syndrome and differentiate between the different forms of the sepsis syndrome. It integrates best practices, clinical expertise and diagnostic test results for early identification and optimal management of septic states, using evidence-based guidelines and clinical decision support tools.

Paper to Patient (http://www.pmlive.com/digital_handbook/best_practice/medical_education/gsk_digital_training_copd) is a training program for the management of Chronic Obstructive Pulmonary Disease (COPD). Paper to Patient is a game that allows health professionals to work through a number of different scenarios and decide which goals must be set for each scenario. After completing a series of training modules, the players have to put into practice their knowledge on the virtual patients. Depending on how well the user plays, the virtual patient acquires different levels of health. This approach has led to significant results in favour of health care workers.

Code Fred (http://www.msichicago.org/play/codefred), developed by The Museum of Science and Industry of Chicago Survival Mode, is a puzzle game that uses the gamification approach to allow the users to understand how information travels from the senses to the brain. Fred, the protagonist, is spending a night in the desert and being chased by ferocious wolves so his goal is to preserve his body from certain death. The player has to keep Fred alive through a series of mini-games sending adrenaline to help Fred run faster. Each mini-game helps Fred run a little farther toward safety.

SisterMatch (https://diabetessisters.org/sistermatch) is the first matchmaking social game aimed at women between the ages of 40 and 60 with type 2 diabetes. The goal is to promote and facilitate relationships and friendship among the users and to create a support group that can help them to live with their disease. The interface of the social network is a patchwork quilt in which each piece of the blanket corresponds to a patient’s story. The other patients can comment on it. Moreover, each piece is placed next to other blocks on the basis of the answers that the users provide to a personality test. In this way, women with similar personalities will have their stories close, in order to create small groups of support staff and to enable them to develop lasting and meaningful relationships.

Jerry the Bear (http://www.jerrythebear.com/) is a game aimed at young patients affected by type 1 diabetes. Children learn how to take care of themselves by taking care of Jerry the Bear with diabetes, giving insulin shots, monitoring his diet and measuring glucose levels. Children should help Jerry to train for the All Star Games through a sequence of animated storybooks that play on his touchscreen. When a storybook is completed, new tasks are assigned.

Pain Squad (http://www.sickkids.ca/AboutSickKids/sickkids-mobile-apps.html) is an iPhone app that helps kids and teens with cancer to keep track of how intense their pain is, how long it lasts, and where it hurts, which is important information to calibrate the best medical treatment. In the game, the patient plays the role of a special unit of the police with missions and awards, in order to encourage the child to provide data on his/her pain twice a day for 2 weeks. The game-based learning approach offers a virtual world where the disease becomes secondary and transforms the traditional questionnaire on the detection of pain into an amusing activity.

Re-Mission (http://www.re-mission2.org/) is a game aimed at adolescents and young adults with cancer. The mission is to improve treatment showing how chemotherapy and radiation could destroy cancer cells. In the game, players control a nanobot, which flies through the body of different cancer patients to destroy cancer cells and tumors. The nanobot also combats side effects of treatment. Experimental studies (Kato 2010) showed that patients increase their knowledge about cancer and self-efficacy. Moreover, an impact on important health behaviors that are related to survival outcomes was registered.

Following the examples described, in the context of our research two serious games have been designed and developed. The games offer solutions both for the training of professional staff and for the empowerment of the patients and their caregivers. In particular, one of the proposed solutions uses the same approach as “Septris”, where the international best practices in the specific context of chronic heart failure and peritoneal dialysis are applied. The game allows the player to use a diagnostic test, actions and therapy for optimal management of the patient, using evidence-based guidelines and clinical decision support tools. Moreover, an avatar is used to allow the player to follow the evolution of the patient’s health status, like in Septris. The main innovation with respect to Septris, is that the proposed solution uses real patient’s record data in order to allow more realistic training.

In the other solution defined, aimed at patients and caregivers, the user is involved in an interactive and fun interface following the pattern of “Pain Squad mobile”, with the aim of overshadowing the problem disease. Finally, from “Paper to Patient” the definition of points and specific learning goals, depending on how the user plays, has been used.

In addition, for both defined solutions an author component has been developed, in order to allow new games and clinical cases to be built by the trainers to keep the games up-to-date according to the specific learning objectives to reach.

The proposed games are designed to support the training of professional workers and the empowerment of patients.

They are aimed at patients suffering from chronic heart failure and peritoneal dialysis. The first condition occurs when the heart is unable to pump sufficiently to maintain blood flow to meet the body’s needs and it is becoming more common with the increase in the average age of the population. The latter is a treatment for patients with severe chronic kidney disease. Both are chronic diseases that require in depth education of the patients to speed up de-hospitalisation and allow them to self-manage their health status, not to change their daily habits, to keep working or studying, and to maintain a social life. They also require a great effort for professional workers who are responsible for the patient training and follow up. These workers need specific training and continuous updating on international guidelines and protocols concerning therapy and treatments, in order to be able to adequately treat the patient (even at home).

In order to guarantee a high level of engagement and motivation, game-based learning and gamification approaches were used to develop two kinds of games: Simulation of Clinical Cases and Edugame. The former, is aimed at professional training and aims at enabling users to take care of patients according to standard protocols and to manage any critical state of health. The second kind of game, named Edugame, is mainly addressed to patients and caregivers. The main learning objective is to empower the users to manage their diseases and to adopt a healthy life style, to minimize critical situations.

Since the contexts in which the game should be used were two kinds of chronic diseases very delicate, a participatory design has been adopted. During the design and development of the games, two physician and two nurses have been involved. They act as domain expert thanks to their expertise in educational training. The analysis phase was conducted using a collaborative process to achieve a good educational game design, as stated by (Torrente et al. 2014). This allows designer and physician to share a common vocabulary for describing the game and its elements. The game design phase has been formalized using flowcharts to represent the gameplay and the interaction. Then, a set of prototypes has been developed and the experts have selected those more suitable for the specific educational goals. Then the games have been developed and refined through a cyclic process of revisions and improvements.

Simulation of Clinical Cases

Simulation of Clinical Cases has been designed for the training of professional workers on international standard protocols and procedures for chronic heart failure and peritoneal dialysis. The game is a single player simulation game (Apperley, 2006) which uses the problem-solving approach. It has been developed as a multiplatform game with client–server architecture [Author citation]. The server stores the knowledge base of the best practices to be acquired and the game sessions, while web and mobile interfaces allow users to access from different devices.

The game mission is to save the patient who gives his symptoms to ask to the player a diagnosis. Since the game is aimed at different kinds of professionals (specialist physicians, general practitioners, nurses), different scenarios have been designed with specific goals; each player will be required to intervene according to his/her professional role and to the specific knowledge and skills he/she should acquire (e.g. the physician will be asked for a diagnosis or a therapy, while the nurse will be asked for a treatment or a medical test).

When the game starts, it supplies the player with basic information: patient general information (such as name, age, sex, etc.) and clinical record data (such as medical history, clinical examinations, diagnostic tests and results). The main innovation of the game is that data provided are real clinical data related to patients involved in the project. Attention was paid to make data anonymous.

Then the simulated patient asks the player for help for a specific problem (Fig. 1a – Description of the problem – Summary: exit site infection resolved - On the basis of the patient’s clinical record what would you suggest?). The player has to suggest the right therapy, action and/or examination to solve the problem posed by the patient (Fig. 1b – List of possible Actions – Therapy control; Contact the doctor; Infection resolved; Catheter removal and simultaneous reinsertion of a new catheter). The list of possible suggestions varies according to the specific scenario and to the competences required (e.g. a nurse cannot suggest a therapy but suggest to contact a general practitioner).

Simulation of clinical cases: a Description of the problem; b List of possible Actions; c Patient avatar; d Time counter

Full size image

Gamification has been implemented using different dynamics and mechanisms:

• Score and Level: the player can earn points by picking the right action and giving the answer within a predefined time slot, in order to prevent the patient from getting worse. Both a counter and an avatar indicate the worsening of the patient (Fig. 1c – The patient avatar - Fig. 1d - the time counter).

• Ranks: the score will be used to rank the player and to measure the participant’s progress.

• Rewards: the player’s answer could be completely wrong, correct or partially correct. In the first two cases, the game ends and the score is calculated, while in the latter case the player will be rewarded with a time bonus that allows him to provide the correct answer (Fig. 2a – Bonus is given when the answer is partially correct).

  

  The bonus and the feedback: a Bonus; b List of given answers and related feedbacks

  Full size image

• Customization: When the simulation ends, a summary supplies detailed information on the specific scenario. In particular, feedback is supplied for and against each possible answer (Fig. 2b – List of given answers and related feedback that explains why the answer is wrong or correct). The feedback is customised on the basis of the player’s role.

When the game ends, on the summary screen the player can:

• read the details of the game (Fig. 3a - Score and time spent);

  

  The summary screen shows when the game ends: a Score and time spent; b Game details and feedbacks; (C) New game; d Top ranks; e Tips

  Full size image

• read the feedbacks on the game to improve the knowledge of the specific clinical case (Fig. 3b);

• start another game session (Fig. 3c);

• look at the highest score to compare his results with the community scores (Fig. 3d);

• send tips to the staff of the game to improve the scenario of the game (this is particularly useful when players are specialist physicians) (Fig. 3e).

One of the main problems in using games in learning contexts is the scalability of the didactic content. From this point of view, it is important to involve the player in the game updating process. This is the reason why a trainer profile has been defined. The trainer can create new simulation sessions and modify the existing ones. Updating actions can be required because players send suggestions (Fig. 3e) about the quality of the simulation game and to suggest improvements or because of updates of the international guidelines and protocols. For both medical contexts, cardiology and nephrology, the trainer can select a real patient profile among a set of real patient records stored. On the basis of the selected data the trainer could define which kind of problem should be included in the simulation. Then, the trainer selects a list of possible answers that could be suggested to the patient. As previously mentioned, the actions are classified as: Therapies, Actions, and Examinations. The game offers a set of predefined answers on the basis of the international best practices, but if necessary the trainer can insert new ones that will be available in the next interactions. In this way, according to his/her own experience, a trainer can customize the general best practices to solve specific clinical cases.

In order to improve the user experience, Simulation of Clinical Cases is a cross-platform game. Both playing from the web or from mobile devices, all game elements (interface, rules, medical records, monitoring data and possible answers) are built in real time by a web service in order to create game interactions always new. Moreover, the web service stores all the games played by each player and the related statistics.

Edugame

The other game designed and developed, named Edugame, is mainly aimed at patients and caregivers. Its learning objective is to empower patients and caregivers to manage the disease and to promote the adoption of a healthy life style, to minimize critical situations and to avoid hospitalization. Also in this case different scenarios are proposed for each medical context: Cardiology and Nephrology. The game is a role-play game (Apperley 2006) for a single player and was developed as web-based game. Like in the previous game, Edugame uses the problem-solving approach. The user mission is to give the right answers to solve the problem posed by the patient (the avatar in the game).

As depicted in Fig. 4, the game starts with a personal assistant who explains the rules of the game and the interaction instructions (Fig. 4b). The game interactions and the game screens have been carefully designed in order to provide continuous support to users (Fig. 4a: the picture reminds the player the ward chosen, Cardiology in this example).

Edugame welcome screen: a Chosen Ward; b Game instructions

Full size image

Different scenarios have been designed to represent a large number of problems in order to cover the majority of common problems that the patient might face. For each game interaction, a random scenario is picked: the avatar in the game describes his/her problem to the player. For example, in the street scenario (Fig. 5a) the patient tells his sister that over the past few days he has gained 3 kilograms and that his feet are swollen (Fig. 5b). Then, the player can visualize the medical record data (Fig. 6a - hospitalization for chronic heart failure one month ago, weight, blood pressure, etc.), useful to understand the patient’s clinical profile. In the same screen, the three categories (Actions, Examinations and Therapy) of possible answers are shown (Fig. 6c). Then the player has to select a category of answers. For each category he/she has to choose the correct answer from a list of possible options. The available answers are defined according to the competences and skills that the patients and caregivers should acquire. In the specific case represented in Fig. 5, for example, in the Therapy category there will be “Decrease the diuretic” that is a decision on therapy that can be taken by the patients without asking their practitioner.

A clinical case: a Street scenario; b Problem description

Full size image

The clinical cases: a Patient record data; b Lives available in the game; c Possible answers

Full size image

The right answers allow the players to earn points and the score (Score and Levels) is used to rank the players according to their total scores (Ranks). Edugame also uses the gamification mechanics of “life” (Fig. 6b – The hearts represent the lives available for the patient). When wrong answers are supplied the player loses a life. When all three lives are lost, the player can no longer play the game until the trainer “unlocks” him. This strategy has been implemented in order to give the trainer (usually the nurse who is responsible for the patient’s training) the chance to monitor and measure the patient learning progress. In this way, the player and the trainer are encouraged to get in touch: the trainer will investigate the learning gap and could define suitable training initiatives to support the player. This means that the patient will receive a personalised feedback on the basis of his learning gap (Customization). For this reason, also in this case, as in Simulation of Clinical Case, an author component has been implemented to allow the trainer to create new games for specific target audience learning needs.

When the game ends a summary of the game shows the final score. The player can share the score in order to compete with other players (Competition). A leader board is built and shown (Ranks). This mechanism engages players in challenges with other players (Social Dimension).

Both proposed solutions were evaluated by means of a user test aimed at measuring the user satisfaction, motivation and learning gain within a real context of use. Due to time constraints, the project had a strict deadline, and because of the delicate context, the patients suffering from chronic disease and peritoneal dialysis had to be treated with care: only one user test was performed with a very small sample.

Goals

In order to evaluate Simulation of Clinical Case and Edugame components three different goals were defined:

• Goal 1: to evaluate the user satisfaction of the proposed solutions;

• Goal 2: to evaluate the users’ motivation in acquiring knowledge;

• Goal 3: to evaluate the users’ knowledge gain after the use of the games.

Finally, in order to evaluate the general effectiveness of the games, the results of Goal 2 and Goal 3 were combined.

Pre-experimental design

To evaluate the user satisfaction of the proposed solutions in Goal 1 a questionnaire composed of 20 items was used. The items, through a likert scale, measure: the system interaction, the learning content, the terminology and the appearance. To measure the motivation (Goal 2) and the knowledge gain (Goal 3), the test was conducted using the “one group pre-test post-test study” (Heffner 2004) that defines a baseline score using a pre-test and measures the outcomes of the treatment using a post-test. Questionnaires of 20 items were defined for both pre-test and post-test: 8 items were devoted to measure the motivation (using a likert scale), 12 items were multiple choice questions focused on the specific domain knowledge (Nephrology and Cardiology). The questions related to the knowledge were slightly different between the two tests to avoid the recall effect.

The treatments were the Simulation of Clinical Cases and the Edugame components described in the paper. The users were asked to use the games at least for one hour. The expected outcome for the Simulation of Clinical Cases was the skill acquisition about the application of international standard protocols and procedures on the chronic heart failure and peritoneal dialysis. For Edugame, instead, the expected outcomes were the enhancement of users’ knowledge about the disease and the right behaviour to adopt when symptoms appear or critical events occur and the improvement in users’ motivation to acquire knowledge. In context of chronic diseases engaging patient’s motivation is necessary. Often, in fact, people suffering of chronic disease are required to undergo procedures or engage in behaviours that are painful, boring and mundane, thus it is necessary to use learning approach that are able to improve their motivation during the knowledge acquisition process (Kato 2010). The comparison of the level of motivation before and after the treatment allows us to understand if the games have impact on motivation.

Participants and Procedure

As previously mentioned, the users test was carried out in the context of chronic heart failure and peritoneal dialysis. Since Simulations on Clinical Cases and Edugame were aimed at both patients and professional figures, for both games the test procedure was divided into two distinct phases.

The first phase was called hospital phase, since it was carried out at the Departments of Nephrology and Cardiology at the General Hospital in Bari and involved only professional figures: 7 users in the Nephrology Department (3 specialist physicians, 1 specialist and trainer, 3 nurses); and 6 users in the Cardiology Department (2 specialist physicians, 1 specialist and trainer 3 Nurses).

The aim of this phase was to verify if games were appropriate for the training needs of the different profiles involved in the project. Furthermore, this phase allowed trainers to prepare new case studies for the Simulation of Clinical Cases and new scenarios in Edugame to be submitted to the users in the second phase.

The second phase, called home phase, aimed at assessing the users’ satisfaction, the users’ motivation and the users’ knowledge gain using both solutions. The participants in this phase were both professional figures and patients. In particular, in the Department of Nephrology were involved 15 users (3 specialist physicians, 2 general practitioners, 5 patients, 5 caregivers); in the Department of Cardiology, 13 users were involved (2 specialist physicians, 1 general practitioner, 5 patients, 5 caregivers).

This phase was divided into different sub-phases:

• administration of a pre-test to measure the users’ knowledge and motivation before the system interaction;

• user training to use the games;

• interaction with the games;

• administration of both a post-test to assess the knowledge gain and the motivation, and a questionnaire for the usability assessment.

Despite the low number of users involved in the test, the sample was representative of all the different categories of actors involved in the care and management of the patient.

In the following sections a discussion of the obtained results are presented.

Goal 1: discussion of users’ satisfaction evaluation results

The components of Edugame and Simulation of Clinical Cases have been analysed from the points of view of the final user, as a game environment, and of the trainer, as an author environment.

For the Edugame component, the analysis of the results revealed a substantial uniformity of satisfaction among the different categories of users. The practitioners, in particular, highlighted a limited number of game scenarios on which to practice. The problem was not observed by the patients and caregivers, because more games were available for them, prepared by the practitioners in the hospital phase. Also for the Simulation component a positive judgment was recorded both for the web application and the mobile version.

From the trainer point of view the satisfaction assessment was focused on the creation and updating procedures. For Edugame more than 80 % of the sample group stated that the creation procedure was easy but too long. For the Simulation of Clinical Cases about 60 % of the sample group assessed both the procedures as slightly complicated. The main problem was in the creation of new clinical cases, which requires real medical records that should be picked from the database that, at this early stage, was still lacking in information.

Goal 2: discussion of the user motivation results

pt?>Goal 2 aims at measuring the users motivation to acquire knowledge about the disease in a real context of use before and after the interaction with the system. As widely recognised in literature, the motivation is a key factor in the learning process. The empowerment of patient suffering of chronic disease requires that both patient and caregivers would be involved in a learning process to acquire knowledge and skills about the management of disease and the critical situations. This process is fundamental to avoid hospitalization and to guarantee a good quality of life in the long term. From the point of view of professional workers, they need to align their previous knowledge with the international protocols, also in this case the motivation is important as they has to revise their beliefs and behaviours.

The one group pre-test and post-test was used in order to analyse the results better. In particular, the measurement of the user motivation allows us to understand how much attention was paid by the user during the system interaction and how much interest he had in acquiring knowledge about peritoneal dialysis and chronic heart failure. To measure this, 4 levels were defined:

• No motivation (score 0): the user has no interest in the topic;

• Minimum motivation (score 1): the user is interested in acquiring knowledge about basic information on the disease;

• Average motivation (score 2): the user is interested in acquiring knowledge about the disease and its symptoms;

• High motivation (score 3): the user is interested in acquiring in depth knowledge about the disease and the right behaviour to adopt;

The pre-test results analysis revealed that patients and caregivers had a minimum level of motivation: the mean score was 1.3. An increment has been noted in the post-test results. About 65 % of the sample group showed a medium level of motivation (score 2) and about 10 % of the sample group showed a high level of motivation (score 3) (Fig. 7). Among the professional workers, in the pre-test, about 50 % of the sample group showed a high motivation (score 3) and the other 50 % showed a medium level of motivation (score 2). The post-test pointed out a discrete increase of motivation, about 82 % of the sample group showed high motivation (score 3) and the other 18 % showed medium level of motivation (score 2) (Fig. 8).

Distribution of the sample of patients and caregivers before and after the use of the system

Full size image

Distribution of the sample of professional workers before and after the use of the system

Full size image

Goal 3: discussion of users’ knowledge gain results

Similar to the user motivation, to measure the knowledge gain 4 levels of knowledge acquisition were defined:

• No knowledge (score 0): the user does not have any information on the disease;

• Basic knowledge (score 1): the user has generic information on the disease and knows the fundamental terms of the disease;

• Average knowledge (score 2): the user knows the fundamental terms of the disease, he recognizes the main symptoms and the right behaviour to adopt when they occur;

• Advanced knowledge (score 3): the user has an in-depth knowledge of both the disease and the behaviour to adopt when critical episodes occur.

The analysis of the pre-test results pointed out that patients and caregivers had a basic level (1) of prior knowledge, whereas for the professional figures 20 % of them had a medium level of previous knowledge (2) and 80 % had a high level (3). This is due to the distribution of the sample group that involved many specialist physicians and less general physicians, who are usually less expert in these specific diseases.

The post-test results of the patient and caregiver sample show a slight improvement of the knowledge; only 5 subjects out 20 had a higher score in the post-test (Fig. 9). This outcome should be combined with the sample’s motivation results, in order to be better evaluated. Since, the sample’s initial motivation was minimum, probably the majority of the subjects used the games and did the tests with a low level of attention.

Knowledge gain of patient and caregiver sample

Full size image

Also in the professional worker sample group (Fig. 10), the improvement was minimum, but it is clear that the professional workers already had an in-depth knowledge of the topic.

Knowledge gain of professional worker sample

Full size image

Effectiveness of the games

Finally, in order to evaluate the general effectiveness of the games, the mean value of the motivation gain and knowledge gain for both sample groups were calculated. In Table 1, the thresholds and the classification of the level of effectiveness are presented. In the Motivation gain (M) for patients and caregivers the mean value was 0.55. This confirms that the motivation of the patients and caregivers towards knowledge acquisition improved. For the professional workers the value of 0.3 was obtained, since they already have a good level of motivation. From the knowledge gain point of view, however, for both sample groups the effectiveness of the system was Minimum: K = 0.25 for patients and caregivers and K = 0.15 for professional workers.

The negative results on knowledge gain were expected because of the low level of the initial user motivation towards the system. Moreover, the user test took little time to appreciate how the interest increased in the system and, therefore, the motivation to use the system, will have positive effects on the acquisition of new knowledge. In any case, the increase in user motivation of both sample groups allows us to state that the games could be promising tools to improve the motivation in using the system and then to acquire knowledge. Long-term evaluations are needed in order to prove and state that the games are really effective to improve motivation and knowledge acquisition.

Over the years, there has been experimental evidence to document the effectiveness of game-based learning. In recent years, the gamification approach has revealed the value of game-based mechanics to create meaningful learning experiences. For these reasons, in the UBICARE (UBIquitous knowledge-oriented HealthCARE) project, which aims at designing and building a system that favours the de-hospitalization of patients suffering from peritoneal dialysis and chronic heart failure, different kinds of learning services have been designed and implemented using the gamification approach. In particular, two different games have been defined: Simulation of Clinical Cases starts from real medical records and utilizes the user profile to build a realistic problem, in order to allow the acquisition of specific skills regarding treatment protocols and diagnostic procedure. Edugame that proposes different game scenarios aiming at empowering the patients and caregivers to manage their diseases and adopt a healthy life style, to minimize critical situation and to manage crises at home.

One of the main limitations of this work was the small sample in the user test. This was due to the critical domain: patients suffering from chronic diseases are extremely sensitive to stress, thus their inclusion in stressful circumstances, such as a user test, should be carefully considered.

The data collected in the first user test conducted during the project pointed out a discrete appreciation in terms of user’ satisfaction and user motivation and knowledge gain from the different categories of users who interacted with the system. The first results are promising: game-based learning and gamification approaches in the medical education can improve user engagement even when the context and the learning contents are unpleasant. However, they are not sufficient to affirm the games effectiveness. An experiment with a larger sample groups would be necessary to collect more data, in order to provide further evidence. Nevertheless, the promising preliminary results are useful to encourage the authors to continue their research in this field.

This work was supported in part by the Project UBICARE (UBIquitous knowledge-oriented HealthCARE) - EU-FESR P.O. Puglia Region 2007–2013 Grant in Support of Regional Partnerships for Innovation - Investing in your future (UE-FESR P.O. Regione Puglia 2007–2013 – Asse I – Linea 1.2 - Azione 1.2.4 - Bando Aiuti a Sostegno dei Partenariati Regionali per l’Innovazione - Investiamo nel vostro futuro).

Authors and Affiliations

1. Department of Computer Science, University of Bari, via Orabona, 4, 70125, Bari, Italy

   Enrica Pesare, Teresa Roselli, Nicola Corriero & Veronica Rossano

Corresponding author

Correspondence to Veronica Rossano.

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

All the authors where involved in the project. TR was the scientific responsible of the project for the University of Bari and, as responsible, was involved in all the design phase of the proposed solutions and in the definition of the scientific issues, VR is an Assistant professor of University of Bari she contributed to the design and coordination of the development of the proposed solutions, EP (currently PhD student) was owner of a project contract and she designed and developed the edugame and NC (PhD) was owner of a project contract and he designed and developed the Simulation of Clinical cases. All authors participated in the writing of the paper according with their specific skills. All authors read and approved the final manuscript.

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions