Serious Game For Promoting Health Data Literacy Using The Inven!RA Analytics Architecture

FACULDADE DE E NGENHARIA DA U NIVERSIDADE DO P ORTO
Serious game for promoting health data

literacy using the Inven!RA analytics
architecture
José Ricardo Vieira Gomes
Mestrado em Engenharia Informática e Computação
Supervisor: Fernando José Cassola Marques, Ph.D.

Second Supervisor: Maria van Zeller, Ph.D.
July 27, 2023

Serious game for promoting health data literacy using the
Inven!RA analytics architecture
Mestrado em Engenharia Informática e Computação
July 27, 2023

Abstract
Currently, professors increasingly have the ambition to obtain a deeper, critical, and reflective un-
derstanding of their students’ progress in activities, where the methodologies adopted are capable
of constructing knowledge with the mediation of the professor to the student, in several different
areas. However, these methodologies generate new challenges and difficulties, especially in the
awareness of what happens during the activity. Despite the wide range of educational games cur-
rently available, a game-based learning application where teachers can actively intervene has not
been yet explored.
To address this problem, the use of video games has been one of the methods increasingly
used to make student learning more immersive. However, this method poses a challenge for the
teacher to be able to properly monitor the student’s progress in didactic activities.
The objective of this dissertation is to develop an educational game that serves as a proof of
concept for the use of learning analytics in serious games to help teachers to monitor better and
in a more granular way the progress of their students. The game was developed using the Unreal
game engine and is a ‘branching story’ game for digital and health literacy. An activity provider
was also developed to start the game and communicate with the Inven!RA platform, which allows
real-time monitoring of students’ progress in activities orchestrated by the teacher in immersive
environments. This ‘activity provider’ runs on a XAMPP server and was developed using PHP:
Hypertext Preprocessor (PHP) and Javascript.
This work demonstrates that it is possible to use serious games in real-time monitoring by the
teacher of the student’s progress throughout the didactic activities, which gives the teacher the
benefits of being more aware of the progress of his students and of allowing him to intervene so
that the students can meet learning objectives.
Keywords: game-based learning, serious games, immersive learning, learning analytics, learning
management systems
i
ii
Resumo
Atualmente, os docentes têm cada vez mais a ambição em obter uma compreensão mais aprofun-
dada, crítica e reflexiva sobre o progresso dos seus estudantes nas atividades, onde as metodolo-
gias adotadas sejam capazes de transferir conhecimento do professor para o aluno, em varias áreas
diferentes. Contudo, estas metodologias geram novos desafios e dificuldades, especialmente na
consciência do que ocorre durante a atividade. Apesar da grande variedade de jogos educativos
disponíveis nos dias de hoje, não foi explorado uma aplicação baseada em game-based learning
onde os docentes possam intervir ativamente.
Para abordar este problema, o uso de vídeojogos tem sido um dos métodos cada vez mais
usados para tornar a aprendizagem dos estudantes mais imersiva. Contudo, este método traz o
desafio do docente conseguir devidamente acompanhar o progresso dos estudantes nas atividades
didáticas.
O objetivo desta dissertação é desenvolver um jogo educativo que sirva de prova de conceito
ao uso de learning analytics em jogos sérios para ajudar os docentes a monitorizar melhor e de
forma mais granular os progressos dos seus estudantes. O jogo foi desenvolvido usando o motor
de jogo Unreal and consiste num jogo do tipo ‘história ramificada’ para a literacia nas área digital
e da saúde. Um activity provider foi também desenvolvido para arrancar o jogo e comunicar com a
plataforma Inven!RA, que permite o acompanhamento em tempo real do progresso dos estudantes
em atividades orquestradas pelo docente em ambientes imersivos. Este ‘activity provider’ corre
num servidor XAMPP e foi desenvolvido usando PHP: Hypertext Preprocessor (PHP) e Javascript.
Este trabalho demonstra que é possível usar jogos sérios no acompanhamento em tempo real
pelo docente do progresso dos estudantes ao longo das atividades didáticas, o que confere ao
docente os benefícios de ter maior consciência do progresso dos seus estudantes e de lhe permitir
intervir para que os estudantes consigam cumprir com objetivos de aprendizagem.
Keywords: gamificação, aprendizagem imersiva, análise de aprendizagem, sistemas de gestão de

aprendizagem
iii
iv
Acknowledgements
I would like to start by thanking my supervisor, Ph.D. José Cassola Marques, for the support and
advice in preparation of my dissertation.
I also want to express my gratitude to my co-supervisor Ph.D. Maria van Zeller, and Ph.D.
Leonel Morgado, who were always very helpful with their suggestions and contribution in this
dissertation, guiding me through the literature, and motivating to work with great suggestions in
the area of learning management systems in scenarios where video games are used for educational
purposes, where the work for this dissertation can contribute.
I also like to thank my best friend Gonçalo Leão for all the support on my journey in FEUP. He
was always there when I needed, with opinions and suggestions through the courses, and, wouldn’t
be able to get this far without him. I thank as well my friends Ricardo Lopez, Victor Magalhães
and João Santos.
I also like to thank all the Teachers from FEUP, for the support given during classes and outside
the classroom.
Finally, to would like to thank my family for supporting me throughout this journey.
v
vi
“Until I began to learn to draw,
I was never much interested in looking at art.”
Richard P. Feynman
vii
viii
Contents
1 Introduction 1
1.1 Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 Research questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.5 Document structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 State-of-the-art on educational games and teacher awareness 5

2.1 Game-Based Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Learning analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3 Serious Games Learning Analytics . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4 Existing Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.5 Examined Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.6 Inven!RA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.7 Moodle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3 System conceptualization 19
3.1 Main Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2 Research Methodology: Design Science Research . . . . . . . . . . . . . . . . . 19
3.3 System architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.4 System requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.4.1 Requirements elicitation . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.4.2 Requirements for the serious game . . . . . . . . . . . . . . . . . . . . . 24
3.4.3 Requirements for the activity provider . . . . . . . . . . . . . . . . . . . 25
3.5 System workflows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.5.1 Workflow for IAP creation . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.5.2 Workflow for IAP deployment . . . . . . . . . . . . . . . . . . . . . . . 27
3.5.3 Workflow for the serious game’s gameplay . . . . . . . . . . . . . . . . 30
3.5.4 Workflow for activity creation . . . . . . . . . . . . . . . . . . . . . . . 31
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4 System implementation 33
4.1 System description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.1.1 Database structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.1.2 Description of the activity provider . . . . . . . . . . . . . . . . . . . . 34
4.1.3 Description of the serious game . . . . . . . . . . . . . . . . . . . . . . 38
4.1.4 Description of the learning analytics . . . . . . . . . . . . . . . . . . . . 39
ix
x CONTENTS
4.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.2.1 Iterations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.2.2 Inven!RA integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.3 Prototype Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.3.1 Evaluation of the serious game . . . . . . . . . . . . . . . . . . . . . . . 51
4.3.2 Evaluation of the activity provider . . . . . . . . . . . . . . . . . . . . . 51
4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5 Conclusions and Future Work 53
A Interviews with Professor Maria Van Zeller 55
References 61
List of Figures
2.1 Gameplay of “Where in the world is Carmen Sandiego?” 1 . . . . . . . . . . . . 6

2.2 Gameplay of "ARTé: Mecenas"2 . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3 Gameplay of "Variant: Limits"3 . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4 Gameplay of "Full Spectrum Warrior"4 . . . . . . . . . . . . . . . . . . . . . . 9
2.5 Existing Technologies Examples. . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.6 Inven!RA architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.7 A LA an Inven!RA qualitative analytics report. [17] . . . . . . . . . . . . . . . . 17
2.8 A LA example of dashboard from a website [16] . . . . . . . . . . . . . . . . . 18
3.1 Design Science Research model proposed by Hevner et al. [29] . . . . . . . . . . 20

3.2 Design Science Research model process proposed by Peffers et al. [40] . . . . . 21
3.3 UML component diagram of the architecture overview . . . . . . . . . . . . . . 22
3.4 Sequence diagram for IAP creation in Inven!RA, adapted from Morgado [38] . . 27
3.5 Sequence diagram for IAP deployment in Inven!RA, adapted from Morgado [38] 28
3.6 Example of a JSON with analytics . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.7 Sequence Diagram for the serious game’s gameplay . . . . . . . . . . . . . . . . 30
3.8 Sequence diagram for activity creation in the activity provider . . . . . . . . . . 31
4.1 Class UML diagram for the activity provider’s database . . . . . . . . . . . . . . 33

4.2 Create Activity, List of available activities . . . . . . . . . . . . . . . . . . . . . 34
4.3 Question List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.4 Activity Edit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.5 Question edit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.6 Answer Edit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.7 Platform Analytics Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.8 Game Flow Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.9 Game Initial Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.10 Game Question. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.11 Inven!RA Dashbaord Example 1. . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.12 Inven!RA Dashbaord Example 2. . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.13 Platform prototype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.14 Serious game prototype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.15 Activity Provider Configuration JSON File . . . . . . . . . . . . . . . . . . . . . 45
4.16 Creating IAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.17 IAP Deployed Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.18 Creating IAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.19 Moodle Course . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.20 Inven!RA Link Moodle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
xi
xii LIST OF FIGURES
4.21 Creating Moodle Activity External (URL) . . . . . . . . . . . . . . . . . . . . . 48

4.22 Setting User ID in Moodle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.23 Inven!RA Link Moodle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.24 Student Access Code for the Activity . . . . . . . . . . . . . . . . . . . . . . . . 49
4.25 Student Insert Access Code in the Game Activity . . . . . . . . . . . . . . . . . 50
List of Tables
3.1 Description of the actors of the system . . . . . . . . . . . . . . . . . . . . . . . 23

3.2 Description of the user stories for the serious game . . . . . . . . . . . . . . . . 24
3.3 Description of the non-functional requirements for the serious game . . . . . . . 25
3.4 Description of the user stories for the activity provider . . . . . . . . . . . . . . 25
3.5 Description of the non-functional requirements for the activity provider . . . . . 26
4.1 Feedback of the user stories for the serious game . . . . . . . . . . . . . . . . . 51

4.2 Feedback of the user stories for the activity provider . . . . . . . . . . . . . . . . 51
xiii
xiv LIST OF TABLES
Abbreviations
AA Academic Analytics
AJAX Asynchronous JavaScript and XML
API Application Programming Interface
BEACONING Breaking Educational Barriers with Contextualised, Pervasive and Gameful
Learning
aDoS Denial of Service
DSR Design Science Research
EDM Educational Data Mining
EG Edutainment games
EngAGe Engine for Assessment in Games
EU European Union
GBL Game-Based Learning
H5P HTML5 Package
HTTP Hypertext Transfer Protocol
IAP Inventive Activity Plan
INESC-TEC Institute for Systems and Computer Engineering, Technology and Science
JSON JavaScript Object Notation
LA Learning Analytics
LMS Learning Management Systems
LRS Learning Record Store
LTI Learning Tools Interoperability
PBL Problem Based Learning
PHP PHP: Hypertext Preprocessor
RPG Role-Playing Game
SG Serious Game
SCORM Sharable Content Object Reference Model
UML Unified Modeling Language
URL Uniform Resource Locator
xAPI Experience API library
XSRF Cross-Site Request Forgery
xv
Chapter 1
Introduction
Chapter 1 presents the dissertation context, motivation, objectives, and research questions.
1.1 Context
The digital progress in technologies in the last two years due to the COVID-19 pandemic has
accelerated, especially in the area of education, due to the need for teachers to give classes in a
remote environment. In this new teaching configuration, it is more challenging for teachers to
effectively monitor and have awareness of their students’ progress during class activities. As such,
new learning tools and technologies are needed.
Many studies conducted throughout the years have shown that the integration of serious games
in the classroom as a learning tool has great potential in the learning process. This tool can improve
the obtainment and retention of knowledge [15, 27]. However, games alone are not enough for
teachers to monitor and be aware of their student’s progress, as implementing serious games in the
classroom is still limited and does not guarantee success on a pedagogical and motivational level
[54, 27, 3].
Inven!RA is an architecture developed in cooperation with the Institute for Systems and Com-
puter Engineering, Technology and Science (INESC-TEC) and by the UNISINOS Project CAPES/
PRINT "Digital Transformation and Humanities" (Brazil). It allows for online pedagogical inter-
vention with third parties by tracking and managing distributed learning activities [18]. Inven!RA
is based on the Breaking Educational Barriers with Contextualised, Pervasive, and Gameful Learn-
ing (BEACONING) project1 , which began in 2016 and is also developed at INESC-TEC [8]. In-
ven!RA adopted a broker-style pattern [50], similar to a ‘middleman’ approach between teachers,
students, and the activity provider, which provides the activities for the learning sessions.
Inven!RA’s main purpose is to allow third-party pedagogical activities to design activity plans
without the need for a particular game narrative or having to be restricted to an approach such as
1 https://www.inesctec.pt/en/projects/beaconing
1
2 Introduction
Game-Based Learning (GBL) [18]. One of the functions of Inven!RA is allowing education pro-
fessionals to design activities with specify the learning objectives and track the planned activities,
known as Inventive Activity Plans (IAPs) [17].
TRIO2 is an active INESC-TEC project that started in February 2022. It provides relevance
and rigorous requirements for digital health data literacy. It will incorporate a serious game in
health education.
1.2 Motivation
Currently, teachers’ awareness of their student’s progress during learning activities using tradi-
tional learning methods is insufficient. The issue is exacerbated when the classes have a large
number of students [6], especially during the last two years with the global pandemic, where
teachers were forced to give their classes remotely, which made monitoring students very difficult
for teachers to make a proper pedagogical intervention [11]. This means teachers are only aware
of their student’s progress at the beginning and end of the learning activities [19].
Examples of dimensions awareness by the teacher [19] include:
• student motivation throughout the learning activity process;
• sub-activity completion status;
• student satisfaction.
1.3 Objectives
The objective of this dissertation is to develop a proof-of-concept of how Inven!RA can be used
to collect analytics from a serious game during learning activities. These analytics allow teachers
to have a proper assessment and awareness of their students’ progress during teaching activities.
This awareness allows for an effective orchestration of better pedagogical interventions.
First, the relevant data analytics extracted from the serious game need to be identified. These
analytics will be useful for monitoring the learning outcomes. In addition, the method used to col-
lect this data from the serious game using Inven!RA architecture needs to be defined. Inven!RA
provides a means of visualizing the data collected in order for the teacher to have a better aware-
ness of where their students are in relation to the learning outcomes.
The main steps of this dissertation’s approach will follow the Design Science research method-
ology:
• Design: Develop a serious game suitable for relevant data collection and integration with
Inven!RA architecture for Learning Analysis.
2 https://www.inesctec.pt/en/projects/trio
1.4 Research questions 3
• Demonstration: Extract the data from serious games for real-world scenarios to test the
suitability of game design and the relevancy of the data analytics by using Inven!RA archi-
tecture.
• Evaluation: Measuring the effectiveness of the monitoring that allows teachers to have
awareness of their students’ progression and if they are in the right direction towards learn-
ing outcomes.
1.4 Research questions

In order to guide this dissertation work, two research questions were posed:
• RQ1: What relevant analytic data can be extracted from the game that is useful for teacher
awareness?
• RQ2: How can this relevant data be extracted using Inven!RA architecture to perform the
analytic analyzes?
1.5 Document structure

The rest of this report is organized as follows. Chapter 2 presents the state-of-the-art on edu-
cational games and teacher awareness. Chapter 3 present how the proof-of-concept system was
conceptualized, while Chapter 4 details how it was implemented. Finally, Chapter 5 presents the
conclusions and future work.
4 Introduction
Chapter 2
State-of-the-art on educational games

and teacher awareness
Chapter 2 presents the state-of-the-art educational games and learning analytics.
2.1 Game-Based Learning
In the current times, there is growing evidence that traditional teaching methods are not enough to
engage and motivate students in learning activities. Recent research has shown that in the Euro-
pean Union (EU) eight out of ten students that range from first grade up to university play video
games [24]. The enormous success of games that are made for pure entertainment has increased
the interest of educators and researchers in the use Game-Based Learning (GBL) methods with
video games as educational learning tools.
Qian and Clark [42] stated that GBL "describes an environment where game content and
gameplay enhance knowledge and skills development, and where game activities involve problem-
solving spaces and challenges that provide players/learners with a sense of achievement”. GBL
is considered a learning method where students discover portions of the game, further developing
abilities while accomplishing explicit learning outcomes [4].
Using video games as a learning tool has been studied since the 80s, with these games being
known as educational video games [26]. Many studies suggest that educational video games are
effective in obtaining knowledge and retention while keeping students interested during the activ-
ity, as they provide the development of skills and knowledge by completing the game’s challenges
[51] [37] [31].GBL may also be viewed as Problem-Based Learning (PBL), on a few specific
situations [20].
Educators considered the use of computer games as a highly valuable means to address many
problems in the classroom [39]. In fact, a few educators took notice of the success of the game
5
6 State-of-the-art on educational games and teacher awareness
called "PacMan", and were left wondering if “the magic of Pac-Man cannot be bottled and un-
leashed in the classroom to enhance student involvement, enjoyment, and commitment” [9]. For
this reason, teachers believed that if they integrated video games into the classroom as a learning
activity, the students would be more engaged in the lesson. Unfortunately, the opposite occurred,
causing teachers to be unfavorable to the use of video games as a learning tool, with negative
feedback over the effectiveness of this learning approach [39].
This sort of game uses the characteristics of entertainment video games, to build a motivational
and engaging learning experience in the classroom while approaching students toward specific
learning outcomes that are set by teachers in a practical educational setting. They are designed to
foment cognitive skills or practice the use of such skills in a simulation environment [22]. These
include edutainment games (EG) and serious games (EG).
EGs combine education with entertainment [30]. An example of an EG is “Where in the world
is Carmen Sandiego?” (Figure 2.1), from 1985, which teaches the player geographical facts. This
type of game marked the beginning of the first age of digital computer games in the classroom
in the mid-80s to mid-90s [55]. But, by the end of the 90s, ‘edutainment games’ was used as a
buzzword in the development of educational games [10].
Figure 2.1: Gameplay of “Where in the world is Carmen Sandiego?” 1
Clark C. Abt was the first to define the concept of SG in his book written in 1975 called
’Serious Games’ [1, 10]. However, the term SG for educational video games would only be coined
in the late 90s by Ben Sawyer. This marks the beginning of the second age of video games in the
classroom [7].
1 https://www.youtube.com/watch?v=ScvM9pecFOo
2.1 Game-Based Learning 7
A more recent definition of SG that gathers the most consensus is “digital games created not
with the primary purpose of pure entertainment, but with the intention of serious use as in training,
education, and health care”, as stated by Loh et al. [35].
SG and EG are used as synonyms in educational game development, with game developers
agreeing that SGs go a step further than EGs [39], in other words, SGs are not designed with
focus on entertainment [33]. SGs are also used to train high-level skills, such as training for
practicing laser therapy, how to prepare for a catastrophic disaster, or practicing laparoscopic
surgical procedures, then EG [13]. Design attributes desirable in Serious Games SG should have
the following desirable design attributes [5]:
• Clear objectives;
• Repeatable assignments for information solidification;
• Follow students’ progression;
• Stimulate more time spent on a task;
• Capable of adapting the learning level of difficulty.
With the use of SGs, students are able to develop competencies that were difficult to train with
traditional methods. Studies have shown that this method improves students’ skills such as critical
thinking or solving complex problems [35] [25]. However, the use of SGs as a learning strategy in
the classroom brings the need for a deep understanding of the effectiveness of the learning process.
In other words. there is a necessity to evaluate if the SG is performing as intended as a learning
tool.
Recent examples of successful SG include:
• "ARTé: Mecenas" (Figure 2.2), from 2016, a Role-Playing Game (RPG) that aims to teach
historical facts about the Italian Renaissance. The game follows the historical footsteps of
the Medici family and features various influential artworks of the 15th and 16th centuries,
including over 130 pieces of art by more than 70 artists [52].
• "Variant: Limits" (Figure 2.3), from 2017, a 3D platformer that teaches students about
limits, a fundamental concept in calculus (mathematics). The virtual world the player must
traverse is divided into four zones, each one associated with a part of the game’s plot and a
set of topics/learning outcomes [53].
Figure 2.2: Gameplay of "ARTé: Mecenas"2
Figure 2.3: Gameplay of "Variant: Limits"3
2 https://store.steampowered.com/app/934950/ART_Mecenas
2.1 Game-Based Learning 9
However, in spite of the growing acceptance and interest in using SGs as a learning tool, they
still have limitations that are preventing their integration in the classroom, such as the cost of
developing and maintaining the game. One example of the cost of developing and maintaining
limitations is the game "Full Spectrum Warrior", 2004, which was developed for the US Army.
This game aimed to train soldiers in the army’s doctrine. However, the doctrine was changed,
making the game unsuitable and the cost of implementing the new doctrine was deemed unfeasible
for the time it takes to implement 2.4.
Figure 2.4: Gameplay of "Full Spectrum Warrior"4
The largest obstacle to the use of SGs is the teachers’ hesitancy in their use as a learning
tool in their classroom [24]. This hesitancy is namely caused by their misconceptions when they
have little knowledge about this type of game, as teachers need to feel they are in control of the
classroom, in order to help their students overcome their difficulties during the learning process,
and to get the desirable learning outcomes [5].
It is important for teachers to understand if their students are learning while playing. However,
SGs are not an evaluation tool by themselves. Also, it is difficult to make measurements of the
students’ interaction with the SGs, as well as the learning progress during this interaction (and
3 https://jakelenzen.com/portfolio/variant-limits/
4 https://coop-land.ru/allgames/strategy/509-full-spectrum-warrior-ten-hammers/
tech-faq.html
not only after the interaction ends). In fact, teachers need to have awareness of their students’
interaction during a SG session, so that they can have better pedagogical interventions if and when
is necessary. This awareness can be attained using a method called Learning Analytics LA, which
enables teachers to measure the students’ progression, detect their difficulties and become more
aware of the current situation of the class [14].
2.2 Learning analytics
SGs generate an enormous amount of data when students interact with the game. This data goes
through established learning ecosystems, such as the eLearning platform Moodle. These plat-
forms are also known as Learning Management Systems (LMS) [5]. They help teachers detect if
students interpreted the lesson’s content correctly or if they misinterpreted it. These interactions
are behaviors and decisions made by the students during gameplay.
These platforms can use the LA method to analyze the data generated by the students’ in-
teraction while playing the game. This method is intended to assist teachers in understanding
if students are completing the exercises. It is critical for the teacher to have the knowledge of
whether a student is falling behind, in other words, of whether the student is not interpreting the
content correctly or they have problems overcoming the obstacles [32].
For this reason, since teachers are concerned when it comes to planning their educational
curricula, it is desirable to incite as much interaction as reasonably possible, to not make the
lesson boring and uninteresting, which may cause students to not engage in the learning process.
LA offer new tools to uncover digital information from various sources, such as log records
[36]. A widely accepted definition of LA, was stated by Siemens and Long [46], as it follows
“the measurement, collection, analysis, and reporting of data about learners and their contexts,
for purposes of understanding and optimizing learning and the environments in which it occurs”.
Most researchers believe that LA is a chance for improving the quality of the learning process.
Ferguson et al. [23] characterized three factors that drove the emergence of LA: big data,
online learning and political concerns.
Data analysis and visualization are the most important aspects for the teacher, as they give
insight into students’ interactions during the gameplay session, as well as valuable assistance
for a better pedagogical intervention when is necessary. Until recently, traditional methods for
measuring and assessing student learning outcomes were time-consuming. They lack detailed
information for teachers to assess if the lesson was going according to the expectations [21].
There are other fields associated with the processing of this data besides LA, such as:
• educational data mining (EDM), which depends mostly on automation and needs to manage
the whole process of finding knowledge gathered from enormous and complex data sets
from an educational ecosystem;
2.3 Serious Games Learning Analytics 11
• academic analytics (AA), which mirrors the job of information analysis at an institutional
level, recognizes patterns that will notify issues, and give noteworthy choices for academic
administration [35].
2.3 Serious Games Learning Analytics
As previously discussed, the primary goal of SGs is to involve computer games in education, rather
than leisure [14]. They are successful in enhancing interest in learning, and enabling the practice
and assessment of student performance [47]. The main role of LA in SG is to obtain a significant
understanding to perfect the game and its educational design, as well as to refine the player´s
abilities and execution in order to demonstrate game adequacy. By integrating LA with SGs, it is
now possible to measure how effective SGs are in the classroom during a gameplay session.
However, student interaction with a game creates a tremendous quantity of information, even
in small sessions. LA are useful for filtering and aggregating the data, so that the processed data
can be visualized through the customized dashboard, adapted to the teacher’s needs [2]. As such,
the requirements set by the teachers define how the collected data is to be processed.
To obtain this information, it is necessary to use data collection methods. These methods can
be classified into two main families: ex situ and in situ. On the one hand, ex situ is the collection
of information outside of the game session, such as surveys or questionnaires. On the other hand,
with in situ methods, information is collected during the students’ interaction with the game. This
gameplay data is acquired with methods such as log files [28] or game telemetry [35], which
allows the detection of flaws in the game design [44]. Log files are mapped according to a set of
rules that are standard for game events or states [45].
After the process of collection of the relevant data for the teacher, the information collected
is preserved in a Learning Resource Store (LRS). Afterward, the data is processed and cleaned
to make the information reliable and readable, which includes eliminating information that is
duplicated and irrelevant information such as computer crashes [35]. After the cleaning process is
completed, the information is converted to a suitable format, usually, JavaScript Object Notation
(JSON), so that platforms, such as H5P, can perform the analysis of one or more students. This
way, teachers have more explicit awareness of where their students are regarding the learning
outcomes, as well as their mistakes.
However, the collection of this gameplay data used to be challenging, due to concerns about
data security and privacy. The reason for these concerns was that LAs integrated into SGs can
gather any type of information. Fortunately, in recent years, there has been a positive development
in terms of data sharing with a growing market of free applications [5].
As Petrov et al. [41] stated, the interaction between SG and LA occurs for two primary rea-
sons: analysis and visualization. Analysis helps to make the collected information understandable
when is visualized using graphical portrayals such as tables, diagrams [35] or dashboards, in order
to display the information in real-time [12]. A dashboard should be intuitive and versatile, to dif-
ferentiate the results of each student playing the game [2]. Dashboards are typically customizable,
for optimal comprehension of whether their students are acquiring knowledge as expected [34].
It is critical to consider at the initial phase of the design all the important information so that
the representation of the results is evident for the teachers to make a proper evaluation of their
students [5].
2.4 Existing Technologies

Some solutions were already developed to assist teachers in gaining a better awareness of a game
environment during a game session, how are students progressing, and comprehending how stu-
dents interact during gameplay, this solution provides valuable data about various students per-
spectives in the form of reports (log files), they aim to improve teacher performance of pedagogi-
cal intervention when it is necessary, being aware of where the student is in relation to the learning
outcomes set by the teacher, in the learning activities.
• The Experience API library (xAPI)5 (Figure 2.5a) is an eLearning standard for communica-
tion between the Learning Record Store (LRS) and learning content [43]. It aims to facilitate
communication between learning technologies [3];
• Learning Tools Interoperability (LTI)6 is an education technology developed to specify a

method for learning systems to invoke and communicate with external and/or third-party
systems. LTI can be used to host content and tools provided by external and third-party
systems, with information about the student in the educational learning environment with
outside systems.
• Sharable Content Object Reference Model (SCORM)7 is a group of standards and specifica-
tions for eLearning systems, it also defines how content may be packaged into a transferable
ZIP file, and it sets the rules that specify the order in which the student will use the activate.
It constrains the student to follow the desired path through the activity content.
• Engine for Assessment in Games (EngAGe) (Figure 2.5b) permits designers to save a lot of
time and cost, not just executing the evaluation process into the game, but as well consid-
ering it. One of the objectives of this project is to assist game designers and teachers with
an instrument for evaluation as well as directives on the evaluation characteristics that must
be included in the game. Importantly, EngAGe facilitate splitting of the SG mechanics and
logical evaluation, making the entire framework more adaptable [14];
5 https://adlnet.gov/projects/xapi/
6 https://www.imsglobal.org/activity/learning-tools-interoperability
7 https://scorm.com/
2.5 Examined Platforms 13
(a) Architecture of Experience API li- (b) Architecture of Engine for Assess-
brary (xAPI)8 ment in Games (EngAGe) [14]
Figure 2.5: Existing Technologies Examples.
2.5 Examined Platforms

To achieve the objective of this dissertation a number of platforms were selected for review that
could be activity providers (in this case educational games). First, analyze what could be extracted
for Inven!RA to use, in order show to the teachers the students’ learning outcomes, and second, if
the data extracted is relevant for data analytics.
• WordWall9 is interactive and collaborative. It is a student-centered learning tool in a class-

room. WordWall is designed to be an interactive tool for students or others to use and
contains an array of activities that can be used during writing and/or reading;
• Genially10 is an online platform that converts presentations into interactive and animated
content activities. With thousands of customized templates, it is an interactive presentation
tool that presents its content dynamically and in a highly interactive manner;
• GoMeta11 is a browser-based low-code development platform for creating web applications.

Independent developers publish templates for apps and games to create new applications.
Users have access to tools that allow easy manipulation of 3D models, sounds, and images.
These visual abstractions are defined by the template creator in the original application
bundle;
• AhaSlides12 is an interactive classroom, meeting, and workshop platform, with thousands

of templates of polls, quizzes, word clouds, etc. The data analyses are displayed in real-time
in customized formats;
9 https://wordwall.net
10 https://genial.ly
11 https://studio.gometa.io
12 https://ahaslides.com
• HTML5 Package13 (H5P) is a free and open-source content collaboration framework based
on JavaScript. It aims to make it easy for everyone to create, share and reuse interactive
content, such as Interactive videos, interactive presentations, quizzes, interactive timelines,
etc. The framework consists of a web-based content editor, a website for sharing content
types, plugins for existing content management systems, and a file format for bundling
together HTML5 resources. It uses the xAPI library method for tracking data and it is
accessible to third parties.
Therefore, the problem presented is one of interoperability in learning environments with plat-
forms such as H5P. At the present time, most of the activities used for GBL are not integrated
inside of the LMS ecosystem, forcing teachers to redirect their students to platforms outside of
LMS platform were teachers lecture their class´s, also forcing the student to create a profile ac-
counts to participate in the activity, such as Kahoot!.
And so, to facilitate the use of these platforms in a LMS ecosystem there is a need to incorpo-
rate interoperability between them, Inven!RA is a solution to this interoperability problem.
2.6 Inven!RA
As mentioned, the problem at hand of interoperability for teachers and trainers during the learn-
ing game session, Inven!RA is one approach to this problem, which begs the question of "What
is the role of Inven!RA in an ecosystem where applications for activity integration in Learning
Management Systems, such as H5P, LTI, or SCORM exist?"
Inven!RA (Figure 3.3) is an architecture that aims to provide online monitoring and pedagogi-
cal assistance in the learning process by numerous activity providers [18]. It enables the planning
of the appropriate learning activities known as IAP, and monitoring progress during the activity,
while other ecosystems such as H5P do not, it was inspired by outside modules that give every ac-
tivity and register the LA generated. Inven!RA architecture approach is derivative from the project
(BEACONING), Inven!RA is mainly used for educational environments such as LMS platforms.
the approach is to be the "middleman" that enables activity providers (third-party platforms) for
pedagogical and monitoring activities with a IAP. The activity providers are responsible for host-
ing the activities and data collection of these activities in order to be able to perform data analyses
from the interactions that students with these activities are performing, the activity providers also
provide a dashboard (Figure 2.8) to display the process data that was collected from the activity.
Inven!RA can also display the data analysis through the dashboard (Figure 2.7), this dashboard
gives teachers the ability to better monitor their student’s progress and difficulties without the
need to use traditional methods of evaluation [17].
However Inven!RA imposes some constraints regarding its workflow:
• IAP must be used, with activities which come from the Activity Providers.
13 https://h5p.org
2.7 Moodle 15
• The Activity Providers must make a set of calls to Web services available to Inven!RA.
• Inven!RA allows for distinct front-ends but each front-end must connect with the Inven!RA
backend.
Figure 2.6 depicts Inven!RA’s architecture. There are three main components:
• Inven!RA;
• the Activity Provider, which corresponds to the companies that host and collect data from
the activities;
• the LMS, is a web-based application to plan educational courses, training programs, mate-
rials, or learning programs, and implement and assess a specific learning process.
The diagram shows four types of users:
• the creator, who designs, configures, and sets the learning goals for each activity set by the
teacher/trainer for the IAP plan. This role can be performed by the teacher.
• the deployer, who designs the course in the LMS system, deploys the activities to the course,
and decides who can access these activities This role can also be performed by the teacher.
• the active agent, who accesses the activities through the LMS system that is granted by
their teachers/trainer, and is redirected to the activity by Inven!RA. This role is typically
performed by the student.
• the awareness agent, who monitors the students’ progress status through the Inven!RA IAP
system during the performed activities to keep his awareness of the goal status. This role
can be performed by the teacher or student.
2.7 Moodle
Moodle is an acronym for Modular Object Oriented Developmental Learning Environment, it is
the most used learning management system in the world. It is free and distributed under an Open
Source license. The MySQL database management system, and PHP programming language are
used to create this architecture.
All those who are participating in the teaching and learning process, offering online courses
(e-learning), or enhancing traditional classroom instruction are the target audience for Moodle.
The system’s primary target group is the Teachers, who may control the platform as needed. They
can decide afterward on the learning environment, its specific goals, the information to be made
available and in what formats, the student-accessible communication channels, and the follow-up
procedures.
13 https://info.library.okstate.edu/c.php?g=741859&p=5306929
Figure 2.6: Inven!RA architecture overview
2.8 Summary
This chapter began by explaining the concept of Game-Based Learning (GBL). Then, educational
video games were presented: the older edutainment games (EGs) and the more recent serious
games (SGs). Afterward, Learning Analytics (LA) were presented in a general setting, before
focusing on their integration with SGs and presenting some existing projects on LA integrated
into SGs. The chapter concludes with the description of Inven!RA and Moodle.
2.8 Summary 17
Figure 2.7: A LA an Inven!RA qualitative analytics report. [17]

Figure 2.8: A LA example of dashboard from a website [16]

Chapter 3
System conceptualization
This chapter presents the conceptualization of the system developed in this thesis. After presenting
the main goal of this work and the research methodology, the system’s requirements, architecture,
and workflows are presented.
3.1 Main Goal
The primary objective of this dissertation is to develop a prototype that serves as a proof-of-concept
of how Inven!RA can be used to extract Learning Analytics (LA) from a serious game. These LA
provide teachers with real-time monitoring and awareness of their students. In fact, as discussed
in Chapter 1, this dissertation addresses the significant challenge of the absence of suitable tools to
accurately assess students’ progress during an activity, thereby emphasizing the need for effective
monitoring and awareness mechanisms.
To achieve this goal, the research conducted follows the Design Science Research (DSR)
methodology, as presented in the next section.
3.2 Research Methodology: Design Science Research
The Design Science Research (DSR) methodology consists of the development of an artifact from
a scientific standpoint. According to Hevner et al. [29], DSR has several advantages when it
comes to helping businesses with real-world problems due to its pragmatism.
Figure 3.1 presents the model proposed by Hevner et al. [29]. This model consists of three
cycles:
• the rigor cycle, which makes sure that the research is in accordance with the required theo-
retical and methodological standards.
19
20 System conceptualization
Figure 3.1: Design Science Research model proposed by Hevner et al. [29]
• the design cycle, which produces an artifact that will address a real-world problem for a
specific situation, where this artifact will be put into practice.
• the relevance cycle: which establishes the requirements for the analysis conducted by the
use of this artifact.
Peffers et al. [40] described the DSR process in detail, as depicted in Figure 3.2. The process
consists of six activities. As shown in figure 3.2 activities two to six are done in a cyclic fashion.
The six activities consist of:
1. Identify the problem and motivate: defines the particular research that breaks down the
problem so that the solution may adequately reflect its complexity. The problem’s descrip-
tion will be used for the development of an artifact that can provide an appropriate solution
successfully. This activity is helpful for understanding the current status and the significance
of the solution to the problem.
2. Define the objectives of a solution: establishes the goals for a potential solution, based
on the problem description and specifications of what is practical and attainable by the re-
searcher’s knowledge. A description of how a new artifact is intended to provide answers to
problems that have not yet been addressed is one example of a qualitative objective. Objec-
tives can also be quantitative, where a desirable solution is preferable to the existing ones.
So the goals should be logically deduced from the problem description and specifications.
3.2 Research Methodology: Design Science Research 21
Figure 3.2: Design Science Research model process proposed by Peffers et al. [40]
3. Design and development: consists of the creation of an artifact, which can consist of con-
structs, models, methods or instantiations. This artifact might theoretically be any created
object in which a research contribution is included. This activity entails deciding on the ar-
chitecture and expected use of the artifact before its creation. The ability to apply theory to a
problem is one of the resources needed to go from the objectives to design and development.
4. Demonstration: shows how the artifact may be used to solve one or more instances of the
problem. This can entail using the artifact for an experiment, a simulation, a case study, a
demonstration or another applicable task. It is crucial to have an effective understanding
of the use of this artifact in order to solve the problem demonstrated. This step is pivotal
for developing a serious game that is not only adept at gathering relevant data but also
seamlessly integrating into the Inven!ra Learning Analytics architecture. In this step, we
will extract game data from real scenarios and thoroughly assess the suitability of the game
design, as well as the relevance of data analysis through Inven!ra.
5. Evaluation: assess the artifact’s ability to aid in finding a solution to the problem. This task
entails contrasting a solution’s goals with the outcomes that were actually obtained when
the artifact was used in the demonstration. It is necessary to have familiarity with pertinent
measurements and analytic methods. The evaluation can take many different shapes de-
pending on the artifact and the problem venue. It could consist of things like a comparison
of the artifact’s functionality with the solution goals from the second activity (i.e. defini-
tion of the objectives). Conceptually, such an assessment may contain any pertinent logical
or empirical support. At the conclusion of this activity, the researchers can choose to go
on to communication and leave further development to the following projects, or they can
go back and try to increase the efficacy of the artifact in the third activity (i.e. design and
development of the solution). Whether or not such repetition is possible may depend on
the nature of the study environment. Measure the efficacy of the monitoring process, which
enables teachers to stay informed about the progress of students and their alignment with
the learning objectives.
6. Communication: explain the problem and its significance to the pertinent audiences, as
well as the artifact’s utility, uniqueness, and efficacy. Researchers may use a similar nominal
structure of an empirical research process, which is a typical structure for empirical research
papers, to structure the paper in scholarly publications.
3.3 System architecture
A general view of the architecture of the system to be implemented is depicted as a UML com-
ponent diagram in Figure 3.3. This diagram is based on an architectural diagram of Inven!RA by
Morgado et al. [38]. The elements developed in this thesis are highlighted in blue.
Figure 3.3: UML component diagram of the architecture overview
With respect to the original architecture presented in Figure 2.6, the system developed in this
thesis was simplified to only encompass two actors, presented in Table 3.1.
Teachers will create the desired IAPs, which are then stored at an activity provider using
Inven!RA. Inven!RA functions as the broker between the teacher and the activity provider, by
creating an Uniform Resource Locator (URL) for each IAP stored in the activity provider.
3.4 System requirements 23
Table 3.1: Description of the actors of the system
Name Description
The user that is teaching the class.
Teacher
They configure the learning activities and monitor the analytics.
The user that is enrolled in the class.
Student
They partake in the learning activities in order to accomplish the learning outcomes
Next, the teacher deploys the IAPs and assigns it to the students that are participating in the
activity, which access it via the LMS and request to start the activity session. Afterward, Inven!RA
will request the IAP deployment’s URL to the activity provider, which sends the requested IAP to
Inven!RA. In turn, Inven!RA sends the activity to the LMS. After receiving the requested activity,
the LMS proceeds to start the activity session.
During the activity session, the activity provider will be collecting the data of the captured
events and performing the LA process. Teachers can monitor their students during the activity
session by using the dashboard generated by Inven!RA. To generate this dashboard, Inven!RA
requests LA data to the activity provider, which after receiving the request, searches for the activity
data and sends it to Inven!RA, which in turn processes the data to generate the dashboard for the
teacher.
3.4 System requirements

This section will detail the requirements that both components developed in this thesis (the activity
provider and the serious game) must satisfy. The requirements can be divided into functional and
non-functional requirements.
Functional requirements declare services that an application must provide and how it should
react given an input or situation [48]. This type of requirement will be presented as user stories in
the following section. A user story consists of a sentence with the following format "As a <role>,
I want to <capability>, so that <goal>." and with three components:
• Role: the name of the actor that has access to the feature of the user story.
• Capability: the feature of the user story.
• Goal: what the feature enables the user to accomplish.
During the development of this thesis work, each user story was assigned a priority level, reflect-
ing the importance of fulfilling specific requirements for the correct functioning of the system.
Non-functional requirements, encompassing aspects like usability and scalability [48], serve as
constraints on the application’s services and features. Before delving into the requirements of
both components, a subsection will be provided to explain the requirements elicitation process,
shedding light on how the requirements were determined.
3.4.1 Requirements elicitation
The functional and non-functional requirements of both components were obtained through a se-
ries of interviews conducted with Professor Maria Van Zeller, as coordinator of the TRIO project,
a project from INESC-TEC that will incorporate a serious game in health education, as mentioned
in Chapter 2. These interviews are presented in Appendix A. Each interview was conducted face-
to-face and lasted between 30 minutes and 1 hour.
For each interview, a set of questions were posed. These questions were categorized according
to the interrogative word (how, where, when, who, what).
In order to properly document how the interview connects with the system’s requirements, an
ID is given to each question-answer pair and will be associated with its respective functional and
non-functional requirements in the upcoming sections.
3.4.2 Requirements for the serious game
Summarizing the interview with Professor Maria van Zeller, the serious game was designed to
improve digital health and data literacy for adults in a fun and interactive way. The game should
have various multiple-choice questions in a branching scenario. Participants will access the game
through a browser, and the interface should be user-friendly for people with less experience with
technology. The game should provide feedback on correct answers, and participants can review
their results after completing the game. Learning analytics to be collected include answers, times-
tamps, and media usage.
Only the student actor will interact with the serious game that is hosted by the activity provider.
Table 3.2 presents the functional requirements as user stories for this component.
Table 3.2: Description of the user stories for the serious game
Identifier Name Priority Description Question

from inter-
view
US1-1 Register High As a student, I want to register in the 25
game, so that I can play the game.
US1-2 Group High As a student, I want to register the group 24
that I am a part of, so that it can be regis-
tered before the game starts.
US1-3 View current ques- High As a student, I want to view the current 27
tion question I am on in the branching scenar-
ios game, so that I can select an answer
and advance in the game.
US1-4 View question me- High As a student, I want to view the image or 31
dia video associated with the current ques-
tion, so that I view the audiovisual con-
tent of the question.
3.4 System requirements 25
The non-functional requirements are presented in Table 3.3.
Table 3.3: Description of the non-functional requirements for the serious game

from inter-
view
NF1-1 Compatibility High The serious game must be compatible with the most 20
commonly-used Web browsers: Google Chrome,
Mozilla Firefox, Microsoft Edge and Opera.
NF1-2 Usability High The game must be playable by people who have little to 25
no digital literacy. Therefore, the game’s user interface
must be intuitive, i.e. the users must be able to interact
with the game with no external help.
NF1-3 Scalability High The system must be able to handle between 30 to 40 22
users playing the game simultaneously without slowing
down.
3.4.3 Requirements for the activity provider

According to the interview, the activity provider platform aims to create and manage serious game
activities. The questions and answers can be edited, and the number of questions and answers can
be customized. Media like images and videos can be used in questions.
Only the teacher actor will interact with the activity provider. Table 3.4 presents the functional
requirements as user stories for this component.
Table 3.4: Description of the user stories for the activity provider

from inter-
view
US2-1 Create activity High As a teacher, I want to create activities, so that students 32
can access them to learn.
US2-2 Edit activity High As a teacher, I want to edit activities (namely their 33
name and the first question), so that I can fix incorrect
details or update them.
US2-3 Delete activity Medium As a teacher, I want to remove activities, so that they 34
are no longer accessible by students.
US2-4 Add question High As a teacher, I want to add questions, so that they are 35
accessible in the branching scenario of an activity.
US2-5 Edit question High As a teacher, I want to edit questions (namely the 33
text of the question), so that they are accessible in the
branching scenario of an activity.
US2-6 Delete question High As a teacher, I want to delete questions that are not 35
necessary in the branching scenario of an activity.
US2-7 Add extra answer High As a teacher, I want to add an extra answer to a ques- 37
tion, to the branching scenario of an activity.
US2-8 Edit answer High As a teacher, I want to edit the answers (namely the 33
text of an answer), so that I can fix incorrect details or
update them.
US2-9 Delete extra an- High As a teacher, I want to delete the extra answer, that is 37
swer not necessary in the branching scenario of an activity.
US2-10 Add/Edit media to High As a teacher, I want to add an image or video to a ques- 39
question tion, so that students have access to visual information
that is important for them to select their answer.
US2-11 Choose answer High As a teacher, I want to choose the next question that 38
destination an answer will connect after this question is selected as
the answer to the current question.
US2-12 View analytics High As a teacher, I want to view all the analytics collected 41
during an activity, so that I can monitor the student’s
progress.
US2-13 Filter analytics High As a teacher, I want to filter the analytics of an activ- 42
ity (namely by the student’s identifier), so that I can
more easily access specific information of the student’s
progress.
The non-functional requirements are presented in Table 3.5.
Table 3.5: Description of the non-functional requirements for the activity provider

from inter-
view
NF2-1 Compatibility High The activity provider pages must be able to function 43
with the most commonly-used Web browsers: Google
Chrome, Mozilla Firefox, Microsoft Edge and Opera.
NF2-2 Usability High The activity provider must have an intuitive and simple 38
interface for the creation and editing of activities. It
must also provide an easily-readable interface for dis-
playing analytic data that was collected in the activi-
ties.
3.5 System workflows 27
NF2-3 Scalability High The system must be able to support a large number of 41
activities, a large number of users accessing the activity
as well as their respective data analytics collected by
the users participating in the event.
3.5 System workflows
This section will detail the system’s workflows using sequence diagrams. These diagrams are used
to validate and document the flow of data in an information system, allowing for a careful analysis
of the progression of data inside a framework [49]. The diagrams that will be presented next are
based on Morgado et al. [38] and adapted to both components: the serious game and the activity
provider, each one accessed by a distinct kind of user.
3.5.1 Workflow for IAP creation
Figure 3.4 shows the workflow for a IAP creation in the Inven!RA side.
Figure 3.4: Sequence diagram for IAP creation in Inven!RA, adapted from Morgado [38]
Once the activities are created in the activity provider, they become available to be included in
the IAP. The teacher sets up the IAP in Inven!RA and includes the desired activities in the IAP.
A configuration window is displayed after an activity from the activity provider is included
in the IAP. When the configuration is completed with the desired activities, the IAP is stored in
Inven!RA, making it accessible for use.
3.5.2 Workflow for IAP deployment
Figure 3.5 depicts the workflow for IAP deployment to a Moodle system.
Figure 3.5: Sequence diagram for IAP deployment in Inven!RA, adapted from Morgado [38]
An IAP can be deployed by a teacher via Inven!RA’s interface. The deployment produces an
URL with the format http://<domain_name>/deploy-activity-activityID. This
URL can then be configured on Moodle by the teacher, so that students have access to a personal-
ized URL that includes their unique identifier on Moodle.
When a student accesses their unique URL through Moodle, Inven!RA receives the student’s
Moodle identifier, creates a unique internal identifier on Inven!RA and sends a POST request to
the activity provider to add the new student to the activity. The student is then redirected to the
activity’s page, where they receive a code that grants them access to the branching scenarios via
the serious game’s mobile app. It should be noted that Inven!RA uses an internal identifier for
privacy reasons.
The teacher can then keep track of students’ progress on the learning objectives during and
after the activity. These objectives’ data are collected by the activity provider during the learn-
ing activity. The teacher can access this data through the Inven!RA dashboard. Accessing this
dashboard triggers a GET request from Inven!RA to the activity provider, which returns a JSON
document with the analytics. An example of this JSON is depicted in Figure 3.6. The URL to
perform these requests has the format
http://<domain_name>/analytics-activity-<activityID> .
Figure 3.6: Example of a JSON with analytics

3.5.3 Workflow for the serious game’s gameplay

Figure 3.7 portrays the workflow of the students’ interaction with the serious game.
Figure 3.7: Sequence Diagram for the serious game’s gameplay
The student registers their participation in the activity that was assigned to them and selects
their age group (18 to 25, 26 to 50, or 51+). Then, the student begins the game activity and each
of their interactions with the game (frontend) are recorded in the backend.
3.5.4 Workflow for activity creation
Figure 3.8 presents the workflow for activity creation in the activity provider.
Figure 3.8: Sequence diagram for activity creation in the activity provider
The teacher begins the activity creation process in the frontend, which issues a command to
the backend. The backend then notifies the teacher whether the activity was created successfully
or not. The teacher then proceeds to the configuration page of the activity, where they can edit
questions. After editing each question, the modifications are stored in the backend’s database.
The teacher can access the activity provider to access their students’ analytics via a dashboard.
3.6 Summary
This chapter began by presenting the main goal of this dissertation, followed by the research
methodology based on the Design Science Research (DSR). The chapter then detailed the system’s
requirements, architecture and workflows.
Chapter 4
System implementation
This chapter presents how the proof-of-concept’s system was developed. First the system is thor-
oughly described, then the implementation steps are detailed. Finally, an evaluation of the proto-
type using a survey is presented.
4.1 System description
4.1.1 Database structure
The interview with Professor Maria Van Zeller aided in the definition of the database used to
support the activity provider. Figure 4.1 depicts the database’s structure using a class Unified
Modeling Language (UML) diagram.
Figure 4.1: Class UML diagram for the activity provider’s database
The database is in the Third Normal Form and in the Boyce-Codd Normal form to avoid
redundant or repeated information.
33
34 System implementation
The data related to each question (namely the title, text and media) is stored in the Question
table. Each question has a set of associated answers, each representing an instance of the Question
table). Each user (represented by an instance of ActivityUser) is connected to multiple analytics
(associated with the Analytic class). Derived classes were used to allow the analytics to have
multiple subtypes, each with their own fields, but with all instances having two common fields:
the time instant when the analytic was recorded and the associated user identifier. Finally, the
Activity table is used to store the created activities which in turn enables Inven!RA to deploy
the activities. Each activity contains a set of parameters that the third-party provider selected for
Inven!RA has access to, via the teacher, when creating a IAP.
4.1.2 Description of the activity provider
The activity provider that was developed is a part of the proof of concept. The activity creation
page of the platform is shown in figure 4.2. This page allows teachers to create and edit activities.
Figure 4.2: Create Activity, List of available activities
After clicking on the edit button on the activity that the teacher wants to modify, they are
directed to an edition page where the list of questions can be found, as shown in figure 4.3. This
page allows the teacher to create new questions and change their order by dragging and dropping
them. It should be noted that changing this order only affects the visual presentation on the website
(to help the teachers organize their activities better). The actual sequence of questions the students
will have access to is defined by editing the questions, given that this sequence is non-linear:
different questions are accessed according to the answers given to the previous questions, which
is the main mechanism of a branching scenario game.
The top of the activity edition page contains a form to allow changing the activity’s name and
selecting the first question, as shown in figure 4.4.
4.1 System description 35
Figure 4.3: Question List
Figure 4.4: Activity Edit
For each question, teachers can edit the question’s title and text as well as upload media, which
can either be an image or a link to a YouTube video, as shown in figure 4.5.
Each question can have between 1 and 3 answers. 1-answer questions are used to pass addi-
tional information to students and to segment activities into parts, with comments or guidelines
for the students delimiting each segment. The form also allows the teacher to define which is the
Figure 4.5: Question edit
next question after the student selects a specific answer, as portrayed in figure 4.6. Any answer
can also be set to lead to the conclusion of the activity by selecting the "End" option.
Figure 4.6: Answer Edit
The platform also has an analytic dashboard to show the teacher their students’ progress after
an activity game session is finished, as shown in Figure 4.7. This dashboard has a separate tab
for each type of analytic. It is possible to filter the analytics by the student’s name and/or a time
interval for when the analytic was collected. Given that hundreds or even thousands of analytics
may be collected, the analytics are divided into pages and teachers can set how many analytics
should be shown per page. All the results are shown in decreasing order of the timestamp field.
Figure 4.7: Platform Analytics Dashboard

4.1.3 Description of the serious game
The developed game is also part of the proof of concept. It consists of an interactive form branch-
ing scenario-type game. The game revolves around a set of questions, each containing a set of
answers that vary from one to three that will make the player follow one of the paths that were
determined by the teacher. The answer given in each question determines which path the player
will follow as is shown in Figure 4.8.
Figure 4.8: Game Flow Diagram
To begin playing the game, players must insert their name and choose the age group they
belong to, as depicted in figure 4.9.
Figure 4.9: Game Initial Screen
When the game starts, the player will view the activity’s first question, including the question’s
text, any media attached to it (image or video) and the possible answers. Figures 4.10a and 4.10b
provide an example of a question with an image and a Youtube video, respectively.
(a) With Image Screen (b) With Youtube Video Embedded Screen
Figure 4.10: Game Question.
4.1.4 Description of the learning analytics

The activity provider collects several game data analytics that are provided for the teacher. The
analytics can be divided into two main types:
• Start activity analytics: a single analytic that is created for each student when they start an
activity. It records the timestamp for when the student initiated the activity.
• Answer analytic: an analytic that is created when the student answers each question. It
records the selected answer and the timestamp.
The analytics are sent by the activity provider to Inven!RA via a HTTP. Inven!RA’s dashboard
presents not only the elementary information of both types of analytics, but also the following
derived data (as presented in Figure 4.11 and 4.12):
• total time elapsed on the activity;
• average time per question;
• question that took the most time to be answered;
• question that took the least time to be answered.

Figure 4.11: Inven!RA Dashbaord Example 1.

Figure 4.12: Inven!RA Dashbaord Example 2.

4.2 Implementation
4.2.1 Iterations
4.2.1.1 First iteration: November 2022 - December 2022
The "Design and development" step of DSR process was performed with the collaboration of
Ph.D. Maria Van Zeller. This step consisted of the development of a serious game and a third-
party activity provider platform to create and modify activities related to health education. These
activities are used in a serious game developed specifically for these activities. The game has
a data collection, to be integrated with Inven!RA to perform adequate learning analyses for the
teacher.
Using the DSR methodology, an initial interview was conducted with Ph.D. Maria Van Zeller
on November 8, 2022. On November 14, 2022, a demonstration of this nonfunctional prototype
for both platform and serious game was presented, as shown in Figure 4.13 and Figure 4.14. This
prototype was evaluated by Ph.D. Maria Van Zeller, who suggested improvements to the activity
provider and serious game, namely:
• on avoiding reloading the page after modifying the information associated with a question,
to make the interface more user-friendly;
• allowing for the user to drag and drop the elements of the question list to change the or-
der in which they are presented, to facilitate the construction of more complex branching
scenarios;
• introducing images and videos on the serious game, to allow the creation of more engaging
questions.
Figure 4.13: Platform prototype

4.2 Implementation 43
Figure 4.14: Serious game prototype
4.2.1.2 Second iteration: January 2023 - February 2023
In the second iteration, the platform was developed in PHP, a scripting language geared towards
Web development. This language was chosen since it is user-friendly and flexible, which allows
developers to have more control over the project compared to other programming languages that
are bogged down by long, complicated scripts. Asynchronous JavaScript and XML (AJAX) was
used on the activity provider to allow users to perform actions on the platform without a compul-
sory page reload.
The serious game was initially developed with a well-known game engine called Unity, be-
cause of its user-friendly, its effectiveness in rendering 2D scenes for game development, and its
built-in analytics, which helps to collect data necessary for our proof concept. It was also detected
that the game can be accessible with a device with Android/iOS system after the student clicks
on the link of the activity that is deployed, the platform will check if the game is installed on the
device, if the game is installed the game will startup, if not the platform will redirect to the link to
the game to install (ex: Google Play Store). During the evaluation of this second iteration, it was
noticed that it is not possible to embed Youtube videos in Unity games without buying plug-ins
that did not offer any guarantees of being compatible with the serious game’s requirements. This
motivated a change in the framework used to develop the game.
4.2.1.3 Third iteration: March 2023 - July 2023
In the third iteration, the game had to be remade in a different game engine that allowed to use of
Youtube video links. The engine that was chosen for this requisite was Unreal Engine 4, this engine
editor is a bit more complex to use than Unity engine but it provides more features and plugins
such as the Web browser that allows using both images and Youtube video links without the need
of third-party programs as it is shown in Figures 4.10a and 4.10b, during the demonstration and
evaluation, it was made a few suggestions such as change question order by drag and drop.
In this final iteration, the activity provider was also integrated with Inven!RA. This proved to
be time-consuming as the version that was worked on for Inven!RA had several issues that needed
to be overcome:
• The frontend was not connected properly to the backend. While the backend uses a Mon-
goDB non-relational database stored on disk, the frontend was using a SQLite3 relation
database that was stored in memory. Thus, the frontend code had to be adapted to access
the backend and use the MongoDB database.
• Understanding Inven!RA code structure was not trivial at certain times as different libraries
were being used to accomplish similar tasks. While Javascript was used as the programming
language, Node.js as the runtime environment, and React as the underlying framework,
different libraries were being used to perform (HTTP) requests: Axios1 , Node Fetch2 and
React Request3 .
The integration is further detailed in subsection 4.2.2.
4.2.2 Inven!RA integration
Inven!RA is integrated with the platform using a service that performs all interactions between the
activity platform and Inven!RA, this service provides the deployment and analysis operations.
The activity provider is one of the main components of Inven!RA. It is responsible for the cre-
ation and supply of the activities, it is also responsible for storing the analytics for those activities
and feeding these analytics to Inven!RA that is not an internal system to Inven!RA architecture.
According to Cota et al. [17], in order to register an activity in Inven!RA, the architecture
must receive a configuration file. As seen in Figure 4.15, this file is a JSON type file that will need
to follow the following configuration parameters:
• name: It is the name of the activity provided by the activity provider.
• con f ig_url: URL of the Activity Provider website where the activity can be configured.
• json_params_url: URL for a Web service that produces a list of items in JSON format
listing the configuration page’s components where Inven!RA may gather the configuration
settings specified there.
• user_url: URL for a Web service that permits activity deployment and takes as a parameter
an Inven!RA instance identity.
• analytics_url: URL for a Web service that accepts a parameter and provides the analytics
values for each user of an activity instance in Inven!RA.
• analyticsl ist_url: URL for a Web service that gives a list of the analytics that the Activity
Provider will gather.
1 https://www.npmjs.com/package/axios
2 https://www.npmjs.com/package/node-fetch
3 https://www.npmjs.com/package/react-request
Figure 4.15: Activity Provider Configuration JSON File
4.2.2.1 IAP creation
The Teacher can begin developing a (IAP) after all activities are ready. Every time the learning
designer adds an activity, a window with the corresponding configuration page is presented since
an IAP is made up of one or more activities as it is shown in Figure 4.16.
Figure 4.16: Creating IAP
Inven!RA gathers the filled-in data after filling out and storing the configuration data. The
setup JSON for the Health Literacy Quiz activity is shown in Figure 4.15. The configuration
JSON file provided in the json p aramsu rl field is the foundation for this recording. Reading this
JSON identifies the fields, and in this example, the summary and instructions fields’ text entries
are gathered.
4.2.2.2 IAP deployment
Activity plans are ready for deployment after being created by the teacher. Inven!RA gives an
URL that should be published on the LMS platform when the instructor publishes an activity as it
is shown in Figure 4.17.
Figure 4.17: IAP Deployed Screen
The last stage in the creation of an activity plan is to obtain the deployed IAP. As it is shown
in Figure 4.18 demonstrates how the deployed link for the IAP was created by the teacher, the
teacher proceeds to copy this link to a LMS platform, where the student will use it to access the
activity of the IAP.
Figure 4.18: Creating IAP

4.2.2.3 Moodle installation and configuration for Inven!RA
The generation of the deployment URL is the next step in the execution of integrated tests. To do
this, Moodle was locally installed together with XAMPP, a free and open-source cross-platform
Web server solution that incorporates the Apache Hypertext Transfer Protocol (HTTP) Server.
The supporting database was developed in MySQL initially it was developed using MariaDB, but
was later changed due to better support for JSON by the former database system). The XAMPP
server also includes interpreters for scripts written in the Hypertext Preprocessor (PHP) and Perl
programming languages.
Four users with student profiles and one with a teacher profile were created to manage the
integrated assessments. There is now a course on Digital Health offered as is shown in Figure
4.19.
Figure 4.19: Moodle Course
One practical Moodle activity with a Digital Health theme was created in the respective course,
where the teacher sets the activity using the external (URL) option where the (URL) provided by
Inven!RA is inserted to be available to the students that are enrolled in the course, as it is shown
in Figure 4.21 and Figure 4.20.
The teacher then setups the userId in the (URL) variables option, as shown in Figure 4.22. The
deployment URL plus userId variable will be generated by Inven!RA was made available in the
activity.
This completes the setting of the activity for the students, URL that is shown in the Moodle
activity shows that the userId variable is generated and filled with the student’s ID and added to the
URL provided by Inven!RA. The students have access to the URL http://localhost:3000/
invenira/deploy/activity?id=64a5fe2128185006687c38aa&userId=5 in the Moo-
dle activity page as it is shown in Figure 4.23 when the student clicks on the link.
Figure 4.20: Inven!RA Link Moodle
Figure 4.21: Creating Moodle Activity External (URL)
The student is redirected to the page that was provided by the activity provider to Inven!RA in
Figure 4.22: Setting User ID in Moodle
Figure 4.23: Inven!RA Link Moodle
the configuration file in the useru rl field (Figure 4.15). This page shows the student their unique
code that is exclusive to the student to access the activity game as it is shown in Figure 4.24.
Figure 4.24: Student Access Code for the Activity
The student then proceeds to install the game, then the student proceeds to copy the activity
access code and insert it in the game’s initial screen to proceed to the activity as shown in Figure
4.25.
Figure 4.25: Student Insert Access Code in the Game Activity
4.3 Prototype Evaluation

To validate the prototype, a narrated video4 was prepared to showcase all of the functional re-
quirements presented in section 3.4 for both the activity provider and the serious game. The video
presents all of the system’s workflows, including the corresponding steps in Moodle and Inven!RA.
A survey5 on Google Forms was prepared to assess whether the prototype fulfills all of the
functional requirements. It is divided into two parts, one for each main component: activity
provider and serious game. In each of these parts, two sections were included in the survey:
• a set of questions asking the user to rank from 1 to 5 whether each of the functional re-
quirements (presented as user stories) were present in the prototype. The lowest score (1)
represents that the user story was not implemented at all, while the highest score (5) signifies
that the user story was completely and correctly implemented.
• an optional free-text question asking for additional feedback about the component.
Due to the limited amount of time during the thesis work, it was only possible for one person
to answer this survey: Ph.D. Maria Van Zeller, who plans to use this work in workshops in the
future, according to the first interview in Appendix A.
The following subsections detail the feedback for each of the components.
4 https://drive.google.com/file/d/1vyv2A5hXLvyAhHRC9dsannmDiqtjw4O-/view?usp=
drive_link
5 https://drive.google.com/file/d/1AAbdxkRnQEUq14IamHcucz_Ufpxb59Pl/view?usp=
drive_link
4.3 Prototype Evaluation 51
4.3.1 Evaluation of the serious game

Table 4.1 presents the scoring by Ph.D. Maria Van Zeller for the functional requirements of the
serious game. All of the functional requirements achieved the maximum score of 5.
Table 4.1: Feedback of the user stories for the serious game
Identifier Name Score (out of

5)
US2-1 Register 5
US2-2 Select age group 5
US2-3 View current question 5
US2-4 View question media 5
In the free-text question, additional feedback was provided:
• Ph.D. Maria Van Zeller commented that she would like to view the progression of the game
in a more visually appealing manner. One idea that the author of this thesis suggests is
adding a progress bar.
• For future work, it was suggested to thoroughly experiment with playing the game on mobile
devices with screens of different sizes, to confirm that the game’s interface is responsive.
• It was advised to add options to make the app more accessible, such as increasing the font
size or changing the colors (for color-blind people).
From these results, it can be concluded that the serious game completely fulfilled all of the
basic requirements identified during the requirements elicitation phase, with some ideas of how
the game be improved in the future.
4.3.2 Evaluation of the activity provider

Table 4.2 presents the scoring by Ph.D. Maria Van Zeller for the functional requirements of the
activity provider. Most (11) of the functional requirements attained the maximum score of 5, with
only a few of them (2) obtaining a score of 4.
Table 4.2: Feedback of the user stories for the activity provider
Identifier Name Score (out of

5)
US1-1 Create activity 5
US1-2 Edit activity 5
US1-3 Delete activity 5
US1-4 Add question 5
US1-5 Edit question 5
US1-6 Delete question 5
US1-7 Add extra answer 5

US1-8 Edit answer 5
US1-9 Delete extra answer 5
US1-10 Add/Edit media to question 5
US1-11 Choose answer destination 5
US1-12 View analytics 4
US1-13 Filter analytics 4
In the free-text question, additional feedback was provided:
• Ph.D. Maria Van Zeller commented that she did not find it very clear how to read the col-
lected information in Inven!RA’s dashboard and suggested adding a progression line (such
as a chart) to make the progression of each student more explicit. This feedback is the likely
cause as to why the two user stories related to the analytics obtained the sub-optimal score
of 4.
• It was suggested to add the age group of each student to the dashboard.
• Some usability improvements were detailed:
– Improving the clarity of the different stages of the creation of an activity.

– Saving the progress during activity creation in a more user-friendly manner than hav-
ing to click to save the fields.
– Adding a hierarchical view (for instance with a graph) of the paths formed by the
questions within an activity.
– Adding a button to show a summary of the whole activity.
• On the video, it was suggested that it was not clear how a teacher creates a JSON to set up
an activity.
From these results, it can be concluded that the activity provider is suitable for the previously
defined objectives and requirements, with some clear lines of how to improve it for future work.
4.4 Summary
This chapter began by describing in detail the proof-of-concept’s system, followed by the imple-
mentation’s iterations and the integration with Inven!RA. The chapter concludes by explaining
how the prototype was evaluated with the aid of a survey. The results show that the developed
prototype respects the previously-defined requirements.
Chapter 5
Conclusions and Future Work
Conclusions
After examining the state-of-the-art, one can conclude that there has been a significant increase in
the past couple of years in using games for education, namely GBL. However, there is still no clear
answer to how LA can be applied effectively to a serious game to extract useful data to increase
teaching awareness.
The contributions of this dissertation include:
• A serious game based on branching scenarios on a mobile app. The game was developed
using the Unreal game engine.
• A server that functions as an activity provider for Inven!RA. The server includes an inter-
face that allows teachers to define their own instances of the serious game by configuring
the activity’s questions. The interface includes its own dashboard to view the students’ ana-
lytics throughout the learning activity. The activity provider was developed using PHP and
Javascript and the server runs using XAMPP.
• Improvements to the Inven!RA framework, so that its frontend and backend communicate
correctly and so that the developed activity provider can communicate with it.
A brief answer can now be provided to the research questions posed during this dissertation:
• RQ1: What relevant analytic data can be extracted from the game that is useful for teacher
awareness?
The serious game that was developed is able to register which answer was selected by the
students for each question on the branching scenario, as well as a timestamp for when the
answer was given. In addition, the game records the timestamp from when students begin
the activity. This data, though elementary, allows more complex information to be extracted
from the activity, such as the average time taken per question and which questions require
the least and most time to be answered, which can reflect which parts of the activity require
more thought.
53
54 Conclusions and Future Work
• RQ2: How can this relevant data be extracted using Inven!RA architecture to perform the
analytic analyzes?
Chapter 4 detailed how Inven!RA was integrated with the serious game, to serve as a proof-
of-concept. This integration shows how Inven!RA can obtain the learning analytics pro-
duced by the serious game using HTTP requests. The positive results of the survey in
section 4.3 highlight that the serious game and its activity provider have enough quality to
serve as a suitable proof-of-concept.
In a nutshell, this dissertation work provides a contribution to the state-of-the-art regarding the
use of educational games as a learning tool to provide greater awareness for teachers in monitoring
their students during the learning process.
Future Work
This work leads to several lines of future research.
First of all, all of the improvements enumerated by Ph.D. Maria Van Zeller for the serious game
(section 4.3.1 and activity provider (section 4.3.2 should be incorporated in the next iteration of
the system’s prototype. These suggestions will improve the accessibility and usability for both the
students and the teachers.
Secondly, the author strongly believes that conducting usability studies with actual students
and teachers would have greatly benefited the project, regardless of whether the prototype ful-
fills its original objectives. By engaging real users, valuable insights could have been obtained
regarding their engagement with the prototype and its ability to meet their requirements and ex-
pectations. These studies would have provided the opportunity to gather user input, identify areas
for improvement, and refine the prototype based on real-world usage scenarios.
Lastly, it will be interesting to develop other types of serious games to integrate with Inven!RA.
Despite the game developed in this thesis serving as a proof-of-concept that learning analytics
can be collected in real-time, it will be very enriching to compare how the learning analytics
of different activities interact in Inven!RA, and to identify which analytics are common to both
games.
Appendix A
Interviews with Professor Maria Van

Zeller
First interview
The first interview was conducted in order to determine the requirements for the serious game and
activity provider. The questions were categorized according to their type.
How?
Q 1 How will the serious game be used?
A: It will be used in workshops. It will be composed as a set of questions and answers about
digital literacy.
Where?
Q 2 Where will the users be able to have access to the serious game?
A: The user will have access to the serious game through TRIO’s Moodle, which belongs to
INESC-TEC.
Q 3 Where will the results be stored and they will be visible to the user?
A: The results may be accessible to the user, and the data will be stored in a game server.
When?
Q 4 When does the data collect from the serious game need to be available to the activity creator?
A: The data needs to be available during and after the event.
Who?
Q 5 Who will be creating the serious game activities?
A: The presenter of the event.
55
56 Interviews with Professor Maria Van Zeller
Q 6 Who will do the serious game activity?

A: The users that will participate in the event, this users will be divided into groups of 18-25,
26-50, and 51+ groups.
Q 7 Who will have access to the serious game analytics of the activities during the event?
A: Only the presenter of the activity and administrator should have access to the analytics data
of the activities.
What?
Q 8 What physical location is the user in to use the serious game?

A: To play the serious game the user will need to be in the event that he will participate.
Q 9 What will be the phases in using the serious game?

A: The serious game will be used in three phases, each phase will have 2 questions and will
be about different digital literacy areas.
Q 10 What is the use of the serious game in the environment where it is inserted?
A: The serious game will be used to promote digital illiteracy in digital environments.
Q 11 What is TRIO’s objective?

A: Is a project that aims to develop ways to improve digital health and digital data literacy for
adults of all ages.
Q 12 What are the expectations?

A: It is to develop a model approach with a complete curriculum and toolset for educators,
supporting a teaching platform that ensures the flexibility to adapt to users.
Q 13 What options will the end user have at their disposal?

A: The options will be, enter the game with your user profile, start the game, answer the
questions and see the results.
Q 14 What data should be extracted for analysis?

A: The data to be extracted will be the answers that the player will give to the questions, the
current level if it reached the end and the result of the game.
Q 15 What infrastructures are intended to be used?

A: The infrastructures will be those of INESC-TEC
Q 16 What tools can be used in the development of Serious Game?

A: The developer is free to choose what tools are appropriate for the development of the
Serious Game.
Interviews with Professor Maria Van Zeller 57
Second interview
This second interview was conducted in order to consolidate and rectify some of the requirements
missed in the first interview, this interview was divided to better comprehend the division between
the serious game and the activity provider platform.
Questions about the serious game
Q 17 Which purpose will the serious game serve?

A: The serious game will serve as a vehicle to improve digital health and digital data literacy
for adults of all ages, in a fun and interactive way.
Q 18 How should the serious game be structured? Should the serious game be a normal quiz or
be more complex like a branching scenario?
A: The serious game structure should be more complex than a normal quiz, it can take the
approach like a branching scenario, with a number of questions, and each answer can proceed to a
different question, for example, question 1-answer1 proceed to question 2 and question-answer2
proceed to question 3.
Q 19 Should the questions be divided into phases?

A: The serious game should be divided into three phases where each phase will have a mini-
mum of two questions for each phase for the participant to answer.
Q 20 Will the serious game be used in an online or offline environment? If it is online, should it
be accessible via a browser? If so, which browsers should it be compatible with?
A: The serious game is expected to be used in an online environment, the serious game should
be able to be played with the most used browsers (Google Chrome, Mozilla Firefox, Microsoft
Edge, and Opera).
Q 21 Who will be participating in the activity?

A: The participants are people that have low literacy in the topics of digital health and digital
data literacy, they are 18+ years old.
Q 22 How many participants are expected to participate in a single activity?

A: The number of participants that are expected to participate in an activity in an event that
will perform an activity is 40 people.
Q 23 Will the participants be divided into groups? If so, according to which criteria should the
users be divided?
A: The participants are to be divided into 3 different types of groups (example: group 1 =
18-25, group 2 = 26-50, and group 3 51+),
Q 24 Which options should be available for the participant to interact with the serious game?
A: The participant should register his/her participation, choose the activity that it is supposed
to participate and start the serious game.
Q 25 How should the game’s interface look like so it is intuitive for people with less experience
with technology?
A: The interface should be the most user-friendly possible, taking into account universal design
concerns, for example, devices that have enabled the increased size of the character type.
Q 26 Should the game have a feedback system for the participant? If so, how should it work?
A: The most suitable feedback that can be given to the participant should be to show which
answer is the correct answer.
Q 27 Should the results of the activity be shared with the participant? If so, which results should
be shared?
A: Yes, the participant should be able to see the question and respective answers at the end of
the serious game.
Q 28 What type of information should be extracted from the serious game, which may be consid-
ered useful for the teacher?
A: The information that should be recorded is the activity name, the timestamp of every inter-
action, the question, the answer selected by the participant, and if it starts and finishes watching a
media file.
Q 29 Can a user select a specific language for themselves or will all participants play the game in
the same pre-determined language?
A: The user should not be able to select other languages that are not predefined by the activity
file.
Q 30 Which security threats should the game be robust against? (examples: prevent the unautho-
rized participation, multiple open sessions from the same participant . . . )
A: The game must not allow for the participants to open more than one window session, it also
should prevent cheating, for example, running third-party software to now the correct answer.
Q 31 How should a question’s media be shown to a user? For videos, can the media be replayed?
Does the video start once the new question is shown or only after pressing a play button?
A: If the question has a video to be displayed, the video should start playing before it displays
the question to the participant, it also should able to be replayed.
Questions about the activity provider

Q 32 Who will create the serious game activities?
A: The activity will be created by the host of the event/workshop, it should be also able to
delete the activity.
Interviews with Professor Maria Van Zeller 59
Q 33 Can the user edit the text of a question and its answers?
A: Yes the user that creates the activity should be able to edit the text of both questions and
answers.
Q 34 Can the user create more than one activity? Or can they only create one instance of the
serious game?
A: The user should be able to create, and delete as many activities as it deems necessary.
Q 35 Should the number of questions in an activity be customizable by the user? If so, what is
the minimum and maximum amount of questions?
A: Yes, the minimum number of questions for an activity should be six, and should give the
user the ability to increase or decrease the number of questions if it deems necessary.
Q 36 What will be the appropriated number of answers for a question by default?

A: Each question should have at least two answers.
Q 37 Should the number of answers be fixed or should the user have the option to add/delete
answers from a question? If so, what is the minimum and maximum amount of answers per
question?
A: If the should be able to add at most one more answer to a question, it should also be able to
delete this extra answer if deems necessary.
Q 38 How should the user configure the question a participant goes to next? What should the in-
terface look like? Is the next question fixed and based only on the answer to the previous question?
A: The user should have a user-friendly interface that should be able to easily choose what
question should be connected to the answer (excluding the question that the answer belongs to).
Q 39 Will the user be using media for the activity questions? If so what type of media should be
expected? (examples: images, videos. . . ) If videos are used, what is the maximum duration (in
seconds) and size (in MB) for a video?
A: Yes, the user should be able to use media such as images, videos, or gifs, the videos should
be at most three minutes long, and should be embedded in the game like a video stream.
Q 40 What type of analytics data should the system collects? What fields should each instance of
an analytic have?
A: The information that should be recorded is the activity name, the timestamp of every inter-
action, the question, the answer selected by the participant, and if it starts and finishes watching a
media file.
Q 41 When should the analytics data be available to the user? During the activity (when partici-
pants are still playing the game) or only after the activity ends?
A: The information should be available to the user when the activity starts, during the activity,
and after the activity is performed.
Q 42 Which criteria should be used to filter the analytics data? (examples: timestamp, user ID
...)
A: The information should have the following filters, activity data and time, activity name, and
participant name.
Q 43 Which browsers should the activity provider be compatible with?

A: The activity provider should be compatible with the most used browsers (Google Chrome,
Mozilla Firefox, Microsoft Edge, and Opera).
Q 44 Which security threats should the activity provider be robust against? (examples: prevent
the unauthorized participation, SQL injection. . . )
A: The activity provider platform must be robust against attacks such as SQL Injection, Cross-
Site Request Forgery (XSRF), Denial of Service (DoS) and unauthorized user access.
References
[1] Clark C. Abt. Serious Game. Viking Press, 1970.
[2] Cristina Alonso-Fernandez, Antonio Calvo, Manuel Freire, Ivan Martinez-Ortiz, and Bal-
tasar Fernandez-Manjon. Systematizing game learning analytics for serious games. In IEEE
Global Engineering Education Conference, EDUCON, pages 1111–1118, Athens, jun 2017.
IEEE Computer Society.
[3] Cristina Alonso-Fernández, Antonio Calvo-Morata, Manuel Freire, Iván Martínez-Ortiz, and
Baltasar Fernández-Manjón. Applications of data science to game learning analytics data: A
systematic literature review. Computers and Education, 141:103612, nov 2019.
[4] Theofylaktos Anastasiadis, Georgios Lampropoulos, and Kerstin Siakas. Digital Game-
based Learning and Serious Games in Education. International Journal of Advances in
Scientific Research and Engineering, 4(12):139–144, dec 2018.
[5] Papadopoulou Athena. A systematic literature review on learning analytics for serious
games. Master thesis, UNIVERSITY OF MACEDONIA, 2020.
[6] Paulo Eduardo Battistella and Christiane Gresse von Wangenheim. Games for Teaching
Computing in Higher Education-A Systematic Review Reuse of Experiential Knowledge
in Software Measurement View project Educational games for teaching Software Project
Management View project. IEEE Technology and Engineering Education (ITEE), 1(3):8–
30, 2016.
[7] Katrin Becker. Choosing and Using Digital Games in the Classroom. Springer International
Publishing, Calgary, 2017.
[8] Aikaterini Bourazeri, Sylvester Arnab, Olivier Heidmann, Antonio Coelho, and Luca Morini.
Taxonomy of a gamified lesson path for STEM education: The beaconing approach. In
Proceedings of the 11th European Conference on Games Based Learning, ECGBL 2017,
pages 29–37. Zenodo, oct 2017.
[9] Richard F Bowman. A ’Pac-Man’ theory of motivation: Tactical implications for classroom
instruction. Educational Technology, 22(9):14–16, 1982.
[10] Johannes S. Breuer and Gary Bente. Why so serious? On the relation of serious games and
learning. Eludamos: Journal for Computer Game Culture, 4(1):7–24, 2010.
[11] David M. Broussard, Yitoshee Rahman, Arun K. Kulshreshth, and Christoph W. Borst. An
interface for enhanced teacher awareness of student actions and attention in a VR classroom.
In IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops
(VRW), pages 284–290. IEEE, mar 2021.
61
62 REFERENCES
[12] Ada Angeli Cariaga and Rommel Feria. Learning analytics through a digital game-based
learning environment. In IISA 2015 - 6th International Conference on Information, Intelli-
gence, Systems and Applications. Institute of Electrical and Electronics Engineers Inc., jan
2016.
[13] Dennis Charsky. From edutainment to serious games: A change in the use of game charac-
teristics. Games and Culture, 5(2):177–198, 2010.
[14] Yaelle Chaudy, Thomas M. Connolly, and Thomas Hainey. EngAGe: A link between edu-
cational games developers and educators. In 2014 6th International Conference on Games
and Virtual Worlds for Serious Applications, VS-GAMES 2014. Institute of Electrical and
Electronics Engineers Inc., jan 2014.
[15] Meng-Tzu Cheng, Jhih-Hao Chen, Sheng-Ju Chu, and Shin-Yen Chen. The use of serious
games in science education: a review of selected empirical research from 2002 to 2013.
Journal of Computers in Education, 2(3):353–375, jul 2015.
[16] Ricardo Jorge Correia and Jesus Grilo. Laboratórios remotos: interação remota com braços
robóticos e integração com plataformas de e-learning. PhD thesis, UNIVERSIDADE
ABERTA, nov 2022.
[17] Duarte Cota, Tiago Cruzeiro, Dennis Beck, António Coelho, and Leonel Morgado. Inven-
tiveTr@ining – Inven!RA architecture Activity Provider modules for online tracking of mi-
croelectronics student projects. Revista de Ciências da Computação, 16:113–136, 2021.
[18] Tiago Cruzeiro. Inven!RA – Platform for authoring and tracking of Inventive Activity Plans.
PhD thesis, Universidade do Porto, 2020.
[19] Claudio Cleverson de Lima. A jornada dos híbridos: acompanhamento dos percursos de
aprendizagem em movimento no contexto da Internet das Coisas. Unisinos, São Leopoldo,
2, 2021.
[20] Martin Ebner and Andreas Holzinger. Successful implementation of user-centered game
based learning in higher education: An example from civil engineering. Computers and
Education, 49(3):873–890, nov 2007.
[21] Tanya Elias. Learning Analytics : Definitions , Processes and Potential. page 23, 2011.
[22] S. Erhel and E. Jamet. Digital game-based learning: Impact of instructions and feedback on
motivation and learning effectiveness. Computers and Education, 67:156–167, sep 2013.
[23] Rebecca Ferguson. Learning analytics: Drivers, developments and challenges. International
Journal of Technology Enhanced Learning, 4(5-6):304–317, 2012.
[24] Manuel Freire, Ángel Serrano-Laguna, Borja Manero Iglesias, Iván Martínez-Ortiz, Pablo
Moreno-Ger, and Baltasar Fernández-Manjón. Game Learning Analytics: Learning Ana-
lytics for Serious Games. In Marcus D. Childress Michael J. Spector, Barbara B. Lockee,
editor, Learning, Design, and Technology, pages 1–29. Springer Nature Switzerland AG, apr
2016.
[25] Ricardo L. Gómez and Ana María Suárez. Gaming to succeed in college: Protocol for a
scoping review of quantitative studies on the design and use of serious games for enhancing
teaching and learning in higher education. International Journal of Educational Research
Open, 2:100021, jan 2021.
REFERENCES 63
[26] Thomas Grace. Digital game-based learning: effects on students’ perceptions and achieve-
ments in a business process management course. PhD thesis, School of Economic and Busi-
ness Science, 2016.
[27] Thomas Hainey, Thomas M. Connolly, Elizabeth A. Boyle, Amanda Wilson, and Aisya
Razak. A systematic literature review of games-based learning empirical evidence in pri-
mary education. Computers and Education, 102:202–223, nov 2016.
[28] Jannicke Baalsrud Hauge, Riccardo Berta, Giusy Fiucci, Baltasar Fernandez Manjon, Car-
men Padron-Napoles, Wim Westra, and Rob Nadolski. Implications of learning analytics for
serious game design. In Proceedings - IEEE 14th International Conference on Advanced
Learning Technologies, ICALT 2014, pages 230–232, Athens, sep 2014. Institute of Electri-
cal and Electronics Engineers Inc.
[29] R Hevner Alan. A Three Cycle View of Design Science Research. Scandinavian Journal of
Information Systems, 19(2):87–92, 2007.
[30] Xiaoshuang Jia, Yue Tan, Qianyun Xiong, Yifang Li, and Wei Hai. Designing edutainment
games for intangible cultural heritage: The BNREG project. In International Conference on
Digital Presentation and Preservation of Cultural and Scientific Heritage (DiPP), volume 9,
pages 281–286, 2019.
[31] Cheryl I. Johnson and Richard E. Mayer. Applying the self-explanation principle to multi-
media learning in a computer-based game-like environment. Computers in Human Behavior,
26(6):1246–1252, 2010.
[32] Kyungrog Kim and Nammee Moon. A model for collecting and analyzing action data in a
learning process based on activity theory. Soft Computing, 22(20):6671–6681, oct 2018.
[33] Fedwa Laamarti, Mohamad Eid, and Abdulmotaleb El Saddik. An overview of serious
games. International Journal of Computer Games Technology, 2014:15, 2014.
[34] Richard L. Lamb, Leonard Annetta, Jonah Firestone, and Elisabeth Etopio. A meta-analysis
with examination of moderators of student cognition, affect, and learning outcomes while
using serious educational games, serious games, and simulations. Computers in Human
Behavior, 80:158–167, mar 2018.
[35] Christian Sebastian Loh, Yanyan Sheng, and Dirk Ifenthaler. Serious games analytics:
Methodologies for performance measurement, assessment, and improvement. Springer, New
York, serious ga edition, 2015.
[36] Stella Maris Massa and Franco D. Kuhn. Learning Analytics in Serious Games: A systematic
review of literature. In 2018 IEEE Biennial Congress of Argentina, ARGENCON 2018, San
Miguel de Tucuman, feb 2019. Institute of Electrical and Electronics Engineers Inc.
[37] Richard Mayer and Cheryl Johnson. Adding instructional features that promote learning in a
game-like environment. Journal of Educational Computing Research, 42(3):241–265, 2010.
[38] Leonel Morgado, António Coelho, Dennis Beck, Christian Gütl, Fernando Cassola, Ricardo
Baptista, Maria van Zeller, Daniela Pedrosa, Tiago Cruzeiro, Duarte Cota, Ricardo Grilo,
and Eliane Schlemmer. Inven!RA Architecture for Sustainable Deployment of Immersive
Learning Environments. Sustainability, 15(1):1–20, 2023.
64 REFERENCES
[39] Stamatios J. Papadakis. The use of computer games in classroom environment. International
Journal of Teaching and Case Studies, 9(1), 2018.
[40] Ken Peffers, Tuure Tuunanen, Marcus A. Rothenberger, and Samir Chatterjee. A design
science research methodology for information systems research. Journal of Management
Information Systems, 24(3):45–77, dec 2007.
[41] Egor V. Petrov, Jamila Mustafina, Mohamed Alloghani, Lenar Galiullin, and Sin Ying Tan.
Learning analytics and serious games: Analysis of interrelation. In Proceedings - Interna-
tional Conference on Developments in eSystems Engineering, DeSE, volume 2018-Septe,
pages 153–156. Institute of Electrical and Electronics Engineers Inc., feb 2019.
[42] Meihua Qian and Karen R. Clark. Game-based Learning and 21st century skills: A review
of recent research. Computers in Human Behavior, 63:50–58, oct 2016.
[43] Ángel Serrano-Laguna, Iván Martínez-Ortiz, Jason Haag, Damon Regan, Andy Johnson, and
Baltasar Fernández-Manjón. Applying standards to systematize learning analytics in serious
games. Computer Standards and Interfaces, 50:116–123, feb 2017.
[44] Ángel Serrano-Laguna, Javier Torrente, Pablo Moreno-Ger, and Baltasar Fernández Manjón.
Tracing a little for big improvements: Application of learning analytics and videogames for
student assessment. In Procedia Computer Science, volume 15, pages 203–209. Elsevier, jan
2012.
[45] Laila Shoukry, Stefan Göbel, and Ralf Steinmetz. Learning analytics and serious games:
Trends and considerations. In SeriousGames 2014 - Proceedings of the 2014 ACM Interna-
tional Workshop on Serious Games, Workshop of MM 2014, pages 21–26, New York, New
York, USA, 2014. ACM Press.
[46] George Siemens and Phil Long. Penetrating the Fog: Analytics in Learning and Education.
46(5):7, 2011.
[47] Abdelali Slimani, Fatiha Elouaai, Lotfi Elaachak, Othman Bakkali Yedri, and Mohammed
Bouhorma. Learning analytics through serious games: Data mining algorithms for perfor-
mance measurement and improvement purposes. International Journal of Emerging Tech-
nologies in Learning, 13(1):46–64, 2018.
[48] Ian Sommerville. Software Engineering. Addison-Wesley, 2011.
[49] Il-Yeol Song, Ritu Khare, Yuan An, and Margaret Hilsbos. A multi-level methodology for
developing UML sequence diagrams. In International Conference on Conceptual Modeling
(ER), volume 5231 LNCS, pages 114–127, Barcelona, Spain, 2008. Springer.
[50] Michael Stal. The Broker Architectural Framework. Workshop on Concurrent, Parallel and
Distributed Patterns of Objects Oriented Programming, held at OOPSLA, 95, 1995.
[51] Han Yu Sung and Gwo Jen Hwang. A collaborative game-based learning approach to improv-
ing students’ learning performance in science courses. Computers and Education, 63:43–51,
apr 2013.
[52] Andre Thomas, Anatol Bologan, Steve Carruthers, Sherman Finch, Susan Pedersen, Janet L.
Spurgeon, and Livia Stoenescu. ARTé: Mecenas, 2016.
REFERENCES 65
[53] Andre Thomas, Paulo Lima-Filho, Susan Pederson, Hadeel Ramadan, and Anatol Bologan.
Variant: Limits, 2017.
[54] Emmanuel Tsekleves, John Cosmas, and Amar Aggoun. Benefits, barriers and guideline
recommendations for the implementation of serious games in education for stakeholders and
policymakers. British Journal of Educational Technology, 47(1):164–183, jan 2016.
[55] Richard Van Eck. 1. EDUCAUSE Review, 41(2):16, 2006.

Serious Game For Promoting Health Data Literacy Using The Inven!RA Analytics Architecture

Uploaded by

Copyright:

Available Formats

Serious Game For Promoting Health Data Literacy Using The Inven!RA Analytics Architecture

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Serious Game For Promoting Health Data Literacy Using The Inven!RA Analytics Architecture

Uploaded by

Copyright:

Available Formats

FACULDADE DE E NGENHARIA DA U NIVERSIDADE DO P ORTO

Serious game for promoting health data

José Ricardo Vieira Gomes

Mestrado em Engenharia Informática e Computação

Supervisor: Fernando José Cassola Marques, Ph.D.

July 27, 2023

José Ricardo Vieira Gomes

Mestrado em Engenharia Informática e Computação

July 27, 2023

Keywords: gamificação, aprendizagem imersiva, análise de aprendizagem, sistemas de gestão de

José Ricardo Vieira Gomes

2 State-of-the-art on educational games and teacher awareness 5

5 Conclusions and Future Work 53

A Interviews with Professor Maria Van Zeller 55

2.1 Gameplay of “Where in the world is Carmen Sandiego?” 1 . . . . . . . . . . . . 6

3.1 Design Science Research model proposed by Hevner et al. [29] . . . . . . . . . . 20

4.1 Class UML diagram for the activity provider’s database . . . . . . . . . . . . . . 33

4.21 Creating Moodle Activity External (URL) . . . . . . . . . . . . . . . . . . . . . 48

3.1 Description of the actors of the system . . . . . . . . . . . . . . . . . . . . . . . 23

4.1 Feedback of the user stories for the serious game . . . . . . . . . . . . . . . . . 51

• student motivation throughout the learning activity process;

• sub-activity completion status;

1.4 Research questions

1.5 Document structure

State-of-the-art on educational games

Chapter 2 presents the state-of-the-art educational games and learning analytics.

2.1 Game-Based Learning

Figure 2.1: Gameplay of “Where in the world is Carmen Sandiego?” 1

• Repeatable assignments for information solidification;

• Follow students’ progression;

• Stimulate more time spent on a task;

• Capable of adapting the learning level of difficulty.

Figure 2.2: Gameplay of "ARTé: Mecenas"2

Figure 2.3: Gameplay of "Variant: Limits"3

Figure 2.4: Gameplay of "Full Spectrum Warrior"4

2.2 Learning analytics

2.3 Serious Games Learning Analytics

2.4 Existing Technologies

• Learning Tools Interoperability (LTI)6 is an education technology developed to specify a

Figure 2.5: Existing Technologies Examples.

2.5 Examined Platforms

• WordWall9 is interactive and collaborative. It is a student-centered learning tool in a class-

• GoMeta11 is a browser-based low-code development platform for creating web applications.

• AhaSlides12 is an interactive classroom, meeting, and workshop platform, with thousands

The diagram shows four types of users:

Figure 2.6: Inven!RA architecture overview

Figure 2.7: A LA an Inven!RA qualitative analytics report. [17]

Figure 2.8: A LA example of dashboard from a website [16]

3.1 Main Goal

3.2 Research Methodology: Design Science Research

3.3 System architecture

Figure 3.3: UML component diagram of the architecture overview

Table 3.1: Description of the actors of the system

3.4 System requirements

• Capability: the feature of the user story.

• Goal: what the feature enables the user to accomplish.

3.4.1 Requirements elicitation

3.4.2 Requirements for the serious game