Open AccessArticle

Training and Certification of Competences through Serious Games

Ricardo Baptista

^1,2,3,*

António Coelho

^1,2

and

Carlos Vaz de Carvalho

⁴

Department of Informatics Engineering, Faculty of Engineering, University of Porto, 4200-465 Porto, Portugal

INESC TEC—INESC Technology and Science, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal

Instituto Politécnico da Maia, Avenida Carlos de Oliveira Campos—Castêlo da Maia, 4475-690 Maia, Portugal

⁴

GILT (Games, Interaction, Learning Technologies), Instituto Superior de Engenharia do Porto, 4200-072 Porto, Portugal

Author to whom correspondence should be addressed.

Computers 2024, 13(8), 201; https://doi.org/10.3390/computers13080201

Submission received: 8 July 2024 / Revised: 6 August 2024 / Accepted: 8 August 2024 / Published: 15 August 2024

(This article belongs to the Special Issue Game-Based Learning, Gamification in Education and Serious Games 2023)

Download

Browse Figures

Review Reports Versions Notes

Abstract

The potential of digital games, when transformed into Serious Games (SGs), Games for Learning (GLs), or game-based learning (GBL), is truly inspiring. These forms of games hold immense potential as effective learning tools as they have a unique ability to provide challenges that align with learning objectives and adapt to the learner’s level. This adaptability empowers educators to create a flexible and customizable learning experience, crucial in acquiring knowledge, experience, and professional skills. However, the lack of a standardised design methodology for challenges that promote skill acquisition often hampers the effectiveness of games-based training. The four-step Triadic Certification Method directly responds to this challenge, although implementing it may require significant resources and expertise and adapting it to different training contexts may be challenging. This method, built on a triadic of components: competencies, mechanics, and training levels, offers a new approach for game designers to create games with embedded in-game assessment towards the certification of competencies. The model combines the competencies defined for each training plan with the challenges designed for the game on a matrix that aligns needs and levels, ensuring a comprehensive and practical learning experience. The practicality of the model is evident in its ability to balance the various components of a certification process. To validate this method, a case study was developed in the context of learning how to drive, supported by a game coupled with a realistic driving simulator. The real time collection of game and training data and its processing, based on predefined settings, learning metrics (performance) and game elements (mechanics and parameterisations), defined by both experts and game designers, makes it possible to visualise the progression of learning and to give visual and auditory feedback to the student on their behaviour. The results demonstrate that it is possible use the data generated by the player and his/her interaction with the game to certify the competencies acquired.

Keywords:

serious games; competencies and skills; “in-game” assessment

1. Introduction

The global computer games industry holds immense learning potential. Games, now marketed to all ages and genders, are increasingly being recognised as powerful tools for learning (GBLs). They have been shown to motivate players, enhance problem-solving, and facilitate collaboration and competition. These benefits are not limited to a specific discipline or education level but apply across the board, from formal classrooms to non-formal learning processes. This recognition of the potential of game-based learning is a cause for optimism about the future of education.

There are already many studies about the effectiveness of Serious Games, both in training and other activities [1,2]. Sousa and other authors [3] consider measuring their actual effects on learning as one of the biggest challenges for accepting Serious Games as an effective educational method. Mayer [4] highlighted that few existing evidence-based approaches to assessing their contribution to learning are still available. Various frameworks have been developed relating to theories of instruction and learning to the mechanics and elements of games, one of two distinct domains: one with the design and development of Serious Games on distinct approaches like describing game-based learning scenarios [5], a scenario generation framework for mission-based virtual training from both the trainer and trainee’s perspective [6], a conceptual framework to design Serious Games that have empathy as part of the learning outcomes [7], considering the flow framework that uses the dimensions of flow experience to analyse the quality of educational games [8], a framework of an evaluation-driven design which offers guidance in the evaluation process [9], a Serious Games design framework in cultural heritage with steps to follow during the whole process [10], and a conceptual framework based upon a systematic literature review of developments in student-centred digital learning [11]. The other domain is evaluation, which uses other approaches, such as the serious game design assessment framework, as a constructive structure to examine purpose-based games [12]; a holistic approach to serious game evaluation with four key areas: theoretical, technical, empirical, and external [13]; an interpretive evaluation framework that can identify the educational value in COTS games [14]; and the dimensionalisation of game- based -learning and further decomposition into factor/sub-factors based on theoretical constructs [15].

Yet, the current state of Serious Games and game-based assessments does not translate into valid certifications. Students still need to prove their knowledge through traditional evaluations. Despite games supporting learning, assessing the knowledge and skills acquired this way needs significant improvement. This research aims to develop a design and development process for Serious Games, considering that they integrate training, evaluation, and skills certification. The focus is on the potential to incorporate the evaluation process and the consequent certification of competencies within the game context, governed by specific norms that systematise student performance measurement.

Four key questions guide this research, each exploring a crucial aspect of the intersection between gaming and skills training:

To determine whether a significant relationship exists between game genres and the training of specific skills. By analysing different game types, we aim to uncover which genres are most effective in cultivating particular competencies, thereby enhancing the educational potential of Serious Games. Another proposition is to identify practical elements of serious game design for evaluating learning and training player skills. As such, the research also explores the possibility of a competency certification method using Serious Games, with game design considering learning objectives and certification performance metrics.
To investigate the elements of serious game design that best support the evaluation of learning and training players’ skills based on structured competency frameworks. This proposal examines how game mechanics, narratives, and feedback systems can be optimised to engage players and assess and develop their abilities.
To identify a robust method for certifying competencies using games. This proposal entails creating a game design that aligns with learning objectives and incorporates performance metrics that can reliably measure and validate players’ skills. By establishing clear criteria and standards, we can ensure the certification process is rigorous and credible, reassuring us about its reliability.
To understand the integration of Learning Analytics into the in-game assessment system to precisely measure player performance in skills training. This proposal involves leveraging data analytics to track and analyse player behaviour and outcomes, providing detailed insights into their learning progress. By doing so, we can enhance the accuracy and effectiveness of skill assessments, ensuring that players receive meaningful and actionable feedback.

This study was able to answer the various questions posed, the answers to which allowed the four-step Triadic Certification Method to be structured and supported by new tools. With the successful integration of the evaluation process into Serious Games, the results obtained made it possible to check the players’ performance in the various training contexts, attesting to the validity of the training design, as well as validating the skills acquired by visualising the progression of skill acquisition through the Triadic Certification Method. This component of the method is particularly comprehensive and well-rounded, as it conventionally designs SGs for the acquisition of competencies based on a balance between the three elements: essential competencies (skills), mechanics (game), and reality (training).

This paper has four sections. The first section presents a literature review on digital games, genre taxonomies of games, Serious Games and their taxonomies, game-based learning, and learning design assessment. In the second section, we discuss the training and certification of competencies. The subsequent section focuses on the Triadic Certification Method, a comprehensive approach developed to address the identified problem, integrating training, evaluation, and skills certification within the game context. The final section describes a case study on the training and certification of competencies in teaching automobile driving and the results obtained. It serves as a proof of concept for the developed method, demonstrating its practical application and effectiveness.

2. Digital Games and Game Based-Learning (GBL)

In this part, we endeavour to elucidate the significance of games within educational contexts and their consequential impact on human behaviour, drawing from constructivist perspectives. We commence by delineating fundamental concepts such as “game” and “play”. A game is a purposeful competition governed by rules wherein players strive for victory. Conversely, play encompasses many intentional activities, often undertaken for recreational or leisurely purposes.

Johan Huizinga’s conceptualisation of the “magic circle” accentuates the discrete space wherein game-related activities manifest devoid of real-life repercussions, underscoring the immersive nature intrinsic to gameplay. This concept has proved pivotal in elucidating the symbiotic relationship between players and games, shaping discourse across digital and traditional gaming paradigms [16].

Scholars including Huizinga [16], Caillois [17], Juul [18], and Salen and Zimmerman [19] underscore the inherent allure of games to players, influenced by variables such as age, cognitive aptitude, and individual personality traits. Game designers leverage these factors to augment player engagement by iteratively adjusting goals, rules, challenges, and participant dynamics.

Contrary to prevalent misconceptions, games are structured environments imbued with clearly delineated objectives, adversaries, and regulatory frameworks. They thereby afford players opportunities for cognitive stimulation and skill refinement. Nonetheless, unlike real-world scenarios, games have distinct consequences for successes and failures, contributing to their intrinsic allure and divergence from reality [20].

Games epitomise rule-bound activities characterised by delimited beginnings, middles, and denouements. They present players with cognitive challenges necessitating proactive engagement. While games ostensibly simulate real-world scenarios, they deviate in outcome predictability and repercussions, thereby furnishing distinct pedagogical and experiential paradigms for players [21].

2.1. Game Genre Taxonomies

The defining characteristic of computer games lies in the interactive pattern established between the player and the game environment. Video games are categorised into genres primarily based on their patterns of gameplay interaction rather than their visual or narrative elements [21]. However, the taxonomy of game genres has been a contention, with numerous proposals in the field [22,23]. These genre classifications organise games into distinct categories defined by their underlying gameplay mechanics.

Understanding game genres is crucial for game designers as it enables them to align additional content, such as new levels or characters, with established gameplay mechanics like jumping or shooting in a platformer game. It also allows them to innovate existing ones, for instance, by introducing a new gameplay mechanic like time manipulation in a puzzle game [24].

Game genres are more than just categories—they are a unifying force. The primary factor uniting games within a genre is the similarity in the interactions facilitated between the player and the game environment. These interactions manifest through various gameplay mechanics, encompassing the actions of in-game objects and players throughout gameplay [25]. It is these recurrent actions or challenges that ultimately define the genre of a game, creating a sense of belonging and connection among players and designers alike.

While there is no universally standardised taxonomy of video game genres, the industry’s recognition is a validation of their importance. The industry commonly recognises several overarching categories. These typically include action, strategy, role-playing, sports, management simulation, adventure, puzzle, and quiz genres, acknowledging the diversity and significance of each [24,26,27,28] (Table 1).

However, it is essential to note that this classification is not exhaustive, and numerous hybrid genres exist that blend elements from multiple established categories. Moreover, the continuous technological advancements within the gaming industry constantly give rise to new genres, particularly with the introduction of novel platforms or input devices. For instance, the advent of Nintendo’s Wii console spurred the emergence of “physical” games like Wii Fit, exemplifying the industry’s ongoing evolution and diversification of genres.

2.2. Serious Games

Serious Games (SGs) are not just a trend in corporate settings and research communities; they are a transformative tool. This game definition harnesses the engaging features of video games to make learning processes not just bearable but exciting [17,18]. With SGs, players can learn while playing, a concept revolutionising education and training. These games are designed to engage players with specific topics, effectively teaching educational content or training workers to perform particular tasks. This transformative power of Serious Games inspires educators and researchers in their work.

SGs refer to digital games used for purposes other than entertainment, such as training, advertising, simulation, or education. Intentionally, these games help learning, skills acquisition, and behaviour change through a game design process that focuses on achieving learning outcomes through gameplay [4,21,29,30]. Clark C. Abt [31], a pioneer in the field, introduced the concept of Serious Games in his book. This marked a shift in the perception of games, extending their meaning beyond mere entertainment when used or embedded in a specific context. Serious Games, as he defined them, are not primarily for amusement but have an explicit and carefully thought-out educational purpose. Since the first Serious Games initiative sponsored by Woodrow Wilson in 2002 and the Serious Games Summit in 2004, there has been significant growth in game-based learning. Serious Games are interactive computer applications with a challenging goal, are fun to play and engage, incorporate some concept of scoring, and impart skills, knowledge, or attitudes that can be applied in the real world.

The concept of Serious Games still lacks a precise definition, with some authors using other terms like immersive learning simulations, digital game-based learning, gaming simulations, and “games you have to play” [32,33]. The main goal of Serious Games is to provide an interactive means for the transference of knowledge to the player. One main goal of Serious Games is to provide an interactive means for transferring knowledge to the player.

SGs educate or train the player, contain a direct means of assessing a skill or learning, and employ a game interface that provides these features. They are considered a new tool within the active learning paradigm. If game design focuses on learning outcomes, learning becomes a natural consequence of playing.

Serious Games have many applications, from government and corporate training to health, public policy, and strategic communication [34]. Despite their diverse uses, the primary focus of SGs remains their educational purpose [29,35]. They are designed to facilitate learning and training and to apply new pedagogies. Research has shown that SGs can accelerate learning, increase motivation, and support the development of higher-order cognitive thinking skills [5]. This diverse range of applications intrigues game developers and professionals in the field.

The key to the success of these games is motivation. It is a psychological process that stimulates an individual to act upon something to attain a desired effect or goal. Motivation in learning can be affected by intrinsic motivation [36], extrinsic motivation [37,38], and emotional stability. These motivational factors should be considered when designing and developing Serious Games. However, due to the broad range of individuals’ emotional stability, it may take much work to address it.

Serious Games (SGs) are not limited to a single field; they have distinct classifications that cater to various purposes. These games are used in government, defence, education, corporate, and industry settings, highlighting their versatility. They address aspects such as occupational safety, skills, communications, and orientation, proving that Serious Games can be applied in various scenarios. Examples of SGs include Alcoa SafeDock, Rosser Surgery Skills w/Games, Shield of Freedom, America’s Army, Darfur is Dying, and Tactical Language & Culture.

Several attempts have been made to classify Serious Games into genres or similar typologies. The criteria used to classify the games vary greatly, with the most commonly used being the educational content and field of application of Serious Games. Michael and Chen [29] name military, government, educational, corporate, healthcare, political, religious, and art games. This typology solely bases itself on the application areas of the games, showcasing the diverse range of Serious Games. For instance, in the health games, Susi et al. [39] list the subgroups of exergaming [40], health education [41], biofeedback [42] and therapy [43].

Sawyer and Smith [44] suggest a higher-resolution taxonomy that crosses game and learning types with application areas. They list advergames, games for work, or games for health as game types. The core innovation of Sawyer and Smith was to separate the designed purpose from actual application areas. Based on the promising work of Ratan and Rittefeld [45], we would propose the following label categories, which allow for the inclusion of specifically designed Serious Games as well as COTS games for “serious” purposes, as the following Table 2 presents:

2.3. Game Assessment

Game assessment is a critical component of the learning process. It involves comparing the expected learning goals with the evidence obtained from the learning actor. In the context of digital game-based learning (DGBL), this evaluation is typically conducted through traditional methods such as questionnaires, interviews, log file analysis, or observation of experience [46]. However, new technologies and media are emerging to support more advanced evaluation tools, addressing the current gaps in evaluation.

Over the past decades, research has aimed to develop new approaches that support the evaluation paradigm in game-based learning. This analysis is divided into two contexts: measuring learning and incorporating the evaluation component in the game development.

2.3.1. Assessment of Digital Learning

The assessment of the game’s learning experience is primarily carried out outside the game’s context, using external tools such as Acumen Team Skills Assessment and Profiles Team Analysis [47]. These tools attest to the results obtained in higher education, management, and personal and team skills development.

The literature refers to three robust theoretical frameworks that underpin the evaluation of learning in Serious Games: RETAIN (Relevance, Embedding, Transfer, Adaptation, Immersion, and Naturalization) [48], Kirkpatrick’s levels of evaluation [49], and the CRESST learning model [50]. These frameworks provide a solid foundation for developing and evaluating learning games, ensuring they effectively incorporate educational content. This thorough evaluation process of Serious Games gives educators and researchers confidence in the effectiveness of this tool.

First, the RETAIN framework supports learning games’ development and evaluates how they contain and incorporate educational content. It aims to identify the best combination between the various game elements, associating them with the genre taxonomy. This framework’s relevance to this work is related to the analysis of the conceptualisation of the relationship between the components of games—the elements and genres of games—and the objectives and levels of learning competencies. The educational potential of games depends on the coherence between different elements using distinct levels of learning conceptualisation and assessment. These models join the curriculum and motivational aspects of their design. The models of learning conceptualisation include Bloom’s taxonomy, which corresponds to a taxonomy of educational objectives, dividing the learning into three main objectives: to generate skills, to develop competencies, and to transfer knowledge in three distinct domains: cognitive, behaviour, and aptitude; and Gagné’s Events of Instruction [33,51], which propose motivational events as positively influencing the achievement of the expected results. Involvement with experience does not derive from a hierarchy of events but rather from the assumption that what goes on inside and outside the game learning experience are lines of and many elements in a single event can be combined or interconnected. This approach, in particular, is a structuring example in terms of the design of the structure of events in the gaming experience for this study, which focuses on skills learning. The closer it is to reality, the more involvement it brings to the transfer of knowledge after learning. Finally, Keller’s ARCS Model [52] states that motivation in student learning corresponds to a systematic process represented by four steps, within which motivation can be achieved or promoted. These models aim to maximise the potential of educational situations by choosing the most appropriate combination of factors to incorporate into the game’s development.

The second model for conceptualising evaluation is Kirkpatrick’s four levels of assessment: Reaction, Learning, Behaviour, and Results. This model presents a hierarchy of levels for evaluating learning or training programmes. The transition between levels plays a vital role in the evaluation process, adding value to the information collected. However, the process becomes more complex and time-consuming. Each evaluation level produces expected results, but this methodology must be applied for a correct analysis [49]. This framework is widely used in training evaluation due to its structure, aligning with the learning aptitudes and competencies cycle.

The CRESST framework by Baker and Mayer [50] comprises five fundamental cognitive requirements for learning: content comprehension, problem-solving, self-regulation, communication, and collaboration/teamwork. This model focuses on the actions considered for each group, which will be tested to validate the expected results.

In summary, the three frameworks presented reflect the domains of learning evaluation but still need to improve their convergence with technologies. Evaluation in game-based learning (GBL) must consider a hierarchical set of needs (Bloom Taxonomy), expected results (the four levels of Kirkpatrick’s assessment), and cognitive requirements (CRESST model). The learning environment must always be considered, as it must promote student-friendly involvement. For successful learning, the experience should increase students’ motivation in the game process through tactics (Keller’s ARCS model) and/or events (Gagné’s Nine Events of Instruction model), focusing on results.

2.3.2. Evaluation Design in Serious Games

The evaluation design in Serious Games involves several conceptual frameworks that apply various evaluation models integrated into the game development process.

The first is the Evidence-Centred Assessment Design (ECD) framework [53,54,55,56], which posits that evaluation results from evidence analysis functioning in a triangular interaction. The keys to a balanced triangular interaction are cognition (theory and information about how students learn), observation (student task performance can demonstrate their learning), and interpretation (used to draw inferences from observations).

Another evaluation framework is the structure of De Freitas and Oliver’s four dimensions [4,57,58]: context, student specification, modes of representation, and pedagogical principles. These dimensions allow for the evaluation of game-based learning and simulations and should be considered interactive, each containing key characteristics. Harteveld and other authors [59,60,61] propose another model that contends that the design fundamentals of any game have a serious purpose.

This model is in continuity with the structures presented previously, where the balance between the forces exerted by the three intervening areas (pedagogy, game elements, and reality) in the design or use of educational games is key. This model has three nuclear pillars: Play Space, Meaning, and Reality. Another structure developed for designing and developing educational games is the “Design, Play, and Experience Framework” (DPE) [62,63,64]. Its main objective is to describe the relationship between the designer and the player as a mediated experience to achieve the expected results through the game. This model is supported by three pillars, each of which contributes to the game’s design according to the phase/level of the game and the type of player.

When the “DPE Framework” is applied to Serious Games, it expands with another set of layers related to the specificity of the design of these types of games (learning, storytelling, gameplay, and user experience), which are transversal to the three components of the structure (Design, Play, and Experience).

Two more models in the context of Serious Games design are the “Experimental Gaming Model” and the EFM Model (effective learning environment, flow experience, and motivation). These models emphasise the experience in the game context, focusing on goals such as motivation, the player’s learning experience, and other emotional or affective aspects.

According to several researchers [65,66,67], the “Experimental Gaming Model” presents the learning process as circular, based on constructing cognitive schemes through activities within the game environment. The direct interactions between players and their experiences with the environment create a circular learning mechanism that includes all the necessary steps to ensure the success and achievement of the objectives. The principal elements of an educational game should be contained in the scenario that will define the learning objectives. Feedback is crucial in providing insight into the acquired knowledge and evaluating the player’s performance. According to Song and Zhang [68] and Hussein [69], the EFM model suggests clever design practises to inspire motivation and help learners genuinely learn from the game. It proposes ideas for developing games with effective learning environments where students develop increased motivation during the experience flow. An effective learning environment supports seven basic requirements by presenting specific tasks with clear objectives and appropriate challenges while achieving a high degree of interaction and feedback. The model includes two distinct levels: a group of nine components of the flow of experience, subdivided into three categories (conditional, experience, and results), and another group of strategies with four essential components (relevance, trust, satisfaction, and attention) to stimulate motivation.

2.3.3. Final Considerations on the Design and Evaluation of Learning with Games

Despite the existing research and diverse approaches to applying evaluation in game design and development, the use of assessment in game-based learning has yet to gain full recognition for its role in the success of this learning approach.

The referenced models aim to associate the assessment process with students’ gains in game-based learning. They highlight how various elements of games contribute to practical evaluation within the games themselves. Key aspects of the game, such as the interface, play environment, narrative, mechanics, and student motivation and involvement, are crucial in this process.

Another critical aspect of the evaluation process is its stakeholders’ respective roles. Evaluators face new challenges as they are expected to collaborate and assist in the evaluation process and in understanding and identifying the entertainment elements of the game.

Evaluating knowledge acquisition and transfer through games focuses on the need to hierarchise learning to achieve expected results. However, learning is not seen merely as a sequence of goals achieved through a gaming experience. Though still performed traditionally (i.e., summative tests), the evaluation now considers convergence models. Motivation, a key element in the decision to carry out activities and corresponding knowledge acquisition, is highlighted in various models.

According to Conole [70,71] and Wills and others researchers [72], there is a need for greater convergence between the role of technology and its impact on evaluation. In game-based learning, new models must be developed that allow us to explore and take advantage of the success of the gaming experience. These issues are very important for future research, especially in the methodologies for designing Serious Games.

3. Training, Competences, and Certification

Training and education are the same in definition but slightly different in context. Both are actions associated with acquiring competencies (knowledge, skills, and attitudes). However, education plays a crucial role in workplace learning, where learning is developed, preferably in the workplace, to improve the performance of employees. They gain practise with tools, equipment, and other elements that can be used daily. Complementing this is another concept: certification. An initial definition of this concept is the validation of the skills that the individual has achieved after the training programme, and this issue will be discussed later in this chapter.

Training is associated with many contexts and is a planned learning experience that ensures permanent change in individual knowledge, attitudes, or skills. The meaning of this learning corresponds to improving the individual’s performance to achieve a certain level of knowledge or skill through the organised transmission of information and/or guided instructions.

Author Michael Armstrong [73] reinforces the idea of performance associated with training by stating that it is a systematic development of the knowledge, skills, and attitudes required of an individual to perform a particular task or job. Similarly, Edwin Flippo [74] states that training increases an employee’s knowledge and skills to perform a specific or particular job. Finally, the author Aswathappa [75] defines the concept as improving skills and attitudes, where training contributes to updating old skills and developing new ones.

Training, as a systematic process, must be directed in such a way as to achieve the expected benefits. We can characterise a training system (programme) in four phases: (1) assessment of training needs; (2) design of training programmes; (3) implementation of the training programme; and (4) evaluation of the training programme.

We can conclude that skills training reflects a programme whose structured approach corresponds to an individual’s training needs to achieve specific results. When the training is completed, the evaluation is carried out on-site, at work, or in the context in which the task is carried out to verify whether or not the acquisition of desired knowledge, skills, or attitudes is necessary.

Building on this understanding, certification is a voluntary process that precedes the on-site verification of competencies (assessment). It is a powerful tool for professionally recognising knowledge, skills, and other practises [76]. According to the authors Byrne, Valentine, and Carter [77], the act of certification, when aimed at validating a more advanced level of knowledge and practise, is a formal procedure that allows an individual or an accredited/authorised entity to assess, verify, and attest, in writing and by issuing a certificate, to the attributes, characteristics, quality, and/or other aspects related to the status of individuals or organisations, procedures, or pre-processes, which are following established requirements or standards [78,79].

Acquiring a certification signifies that the individual’s competencies and attitudes gained endure. This enduring nature, coupled with benefits such as personal development, career progression, financial reward, professional recognition, and perceived empowerment [80], underscores the value and security of the investment in training and certification.

Finally, training is an organised activity that imparts information or instructions to improve performance or help individuals attain the required knowledge and skills. In training, it is essential to distinguish between competency and competence. The former refers to an individual’s ability to make deliberate choices from a repertoire of behaviours in specific professional contexts. The latter is context-dependent and involves integrating knowledge, skills, judgement, and attitudes. Competences are “domain-specific cognitive dispositions that are required to cope with certain situations or tasks successfully and acquired by learning processes” [81].

Different types of competencies can be considered in organisations or specific fields. First, we have personal competencies, representing the core knowledge, skills, and attitudes each person should have for superior performance. Next, functional competencies are related to specific technical knowledge within a particular area or profession. Finally, task competencies are implicit and associated with specific role functions.

Regarding skills, “skill” is often preferred over “competence” in the training and work environment. Hard skills encompass specific technical abilities or solid factual knowledge required for a job, such as machine operation, programming languages, and safety standards. These skills are typically trainable and easy to observe, quantify, and measure. On the other hand, soft skills (also known as “people skills”) are more subjective and are associated with personal attributes and character. Soft skills are essential for applying technical skills in the workplace, including communication, teamwork, problem-solving, and time management. These skills are more challenging to observe and quantify.

3.1. Reference Structures for Skills and Competences

This section aims to present various frameworks used to prepare and recognise the skills that are being learned. The skills matrix is a tool for assessing the skills needed to achieve maximum impact and locating where these skills can be found. In this way, the skills framework is a structure that establishes and defines each skill (such as problem-solving or people management) required of people who work in or are part of an organization.

This matrix/framework can take several approaches, most notably when recruiting employees by aptitude standard and performance appraisal or identifying the aptitudes required to perform an activity in any given role. A matrix can, therefore, be considered an inventory of skills categorised by level, with a given required/chosen level of skills. This matrix results in what can be learned (skills) and the quantification (points) required to acquire and improve a skill.

3.1.1. European Qualifications Framework

The recognition of qualifications in Europe, more specifically in the European Union, is carried out through the European Qualifications Framework (EQFs) (https://wwwcdn.dges.gov.pt/sites/default/files/brochure_eqf_en.pdf (accessed on 7 July 2024)), which acts as a standard reference system to link all national qualifications systems. For each of the eight levels defined, there is a set of indicators that specify the expected learning outcomes corresponding to the qualifications of that level in any qualifications system, covering several education levels (primary, secondary and higher education, vocational training) [82] as well as the processes of recognition, validation, and certification of competences obtained, whether by non-formal or informal means [83].

This approach is based on learning outcomes, with eight reference levels defining what is necessary and sufficient for each student to know, understand, and be able to achieve after completing the learning process. These criteria are defined in terms of knowledge as the result of assimilating information during learning; aptitude, as the ability to apply knowledge and know-how to complete tasks and solve problems; and attitude, as the proven ability to use knowledge; skills; and personal, social, and methodological competences in a work or study context and professional and personal development [84].

3.1.2. Lominger’s Competency Models and Education Competencies: A Comprehensive Approach

A competency model is a comprehensive framework that outlines the behaviours employees must exhibit to achieve success in their roles or perform specific tasks effectively. Unlike job descriptions, which enumerate the tasks and responsibilities associated with a particular position, competency models delve deeper by elucidating how employees should perform their duties. While job descriptions provide a list of tasks and functions required for a role, competency models identify the requisite behaviours, skills, and knowledge essential for executing those tasks proficiently.

Lominger’s sixty-seven competencies have emerged as a universal standard for achieving task success. Known as the Leadership Architect Competencies, this assessment tool enables us to compile a comprehensive list of competencies by combining existing models. The goal is to encapsulate the essential skills for success in various contexts [85]. This competency model represents “a collection of competencies associated with successful performance” [86]. To apply it specifically to education and training, the same authors collaborated with Microsoft to create a similar approach known as Education Competencies or the Educational Competency Wheel [87]. This tool encompasses various attributes, behaviours, knowledge areas, and abilities for effective job performance.

The competency table, as depicted in Table 3, comprises six core skill sets and personality characteristics. These include individual excellence (IE), organisational skills (OrSs), courage (C), results (Rs), strategic skills (SSs), and operating skills (OpSs). While these categories initially draw from Lominger’s standard set of 44 competencies, they can be extended beyond education to other domains, such as competency training. The competency wheel offers additional resources to identify core competencies critical for an organization’s success [88]. These resources include clear definitions, proficiency levels, sample interview questions, and activities aimed at skill development, all geared toward helping organisations achieve their goals.

The six qualities or success factors can be categorised into two main types: hard and soft skills. Hard skills are teachable abilities or skill sets that lend themselves to quantification. In contrast, soft skills are more subjective and challenging to measure. Among the core skill sets, we can consider individual excellence, courage, results, and strategic skills to be soft skills. These enable effective collaboration, direct communication, goal-oriented action, and pursuit of longer-term objectives. On the other hand, operating skills and organisational skills fall into the hard skills category. They encompass the practical skills for daily task management, relationship building, and effective communication across diverse organizational contexts. With this restricted and adequate number of core competencies, the mapping aligns with the technical and personal competence needs based on the expected results.

Comprising 37 competencies referred to as success factors, this set of categories, while aligning with the Lominger matrix, is not limited to formal education. Its versatility extends to areas like skills training and shares striking similarities with other performance standards, such as the Baldrige Education Criteria for Performance Excellence, defined by the International Society for Technology in Education and the National Standards for Educational Technology [89]. Another important aspect is that this competency standard is recognised by UNESCO, whose general competencies are ICT-related competencies [84,90] and application competencies [88].

To conclude this section about the competency standards, it is essential to differentiate between certification and qualification as they carry distinct meanings. According to the European Qualifications Framework (EQFs), qualification represents the formal outcome of an assessment. In contrast, certification involves a validation process conducted by a competent body to determine whether an individual has achieved specific learning outcomes according to a predefined standard.

Numerous international and national standards govern professional certification. Notably, the ISO/IEC 17024 standard, developed by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC), specifies requirements for certification bodies [91]. These standards apply independently of any specific area of expertise. The European Community has also adopted ISO/IEC 17024. In the United States, the National Organization for Competency Assurance (NOCA) has established standards and an accreditation process for certification programmes since the late 1970s.

A professional certification effort, a journey of empowerment, involves three relatively independent dimensions. Firstly, the professional role characterisation includes defining the specific professional role to be certified. Secondly, the list of required abilities and skills to identify the abilities and skills necessary for professionals in that role. Finally, the description of the certification process outlines the certification process and its organizational aspects, all designed to empower professionals in their respective roles.

ISO/IEC 17024 is a crucial standard for individual certification. It is a benchmark for recognising certification bodies and their national and international certification schemes. This standard plays a pivotal role in defining the certification process, encompassing all activities through which a certification body establishes that a person fulfils specified competence requirements.

The competence certification system, a beacon of recognition, is a powerful tool that enables professionals working in the labour market to gain recognition based on their qualifications. By achieving specific parameters, professionals demonstrate their competency, empowering them to showcase their skills and knowledge. For instance, the EQFs assesses whether an individual has acquired learning outcomes aligned with relevant standards. The validation process involves four phases: identification, where dialogue is used to identify an individual’s experiences; visibility, which consists of making these experiences visible through documentation; formal assessment, which evaluates the experiences formally; and recognition, which leads to certification for partial or complete qualifications. This process focuses on assessing the skills and knowledge demonstrated by learners in specific tasks, ensuring their competency applies to real-world scenarios.

4. Triadic Certification Method

Creating virtual environments conducive to learning through games represents a significant milestone. These environments guarantee success in acquiring knowledge and experience for players and students, driven by motivational and engaging elements. Previous literature reviews have highlighted the characteristics and strategies contributing to successful game learning.

While individuals can be trained to achieve expected results in various situations and contexts, assessing skills remains challenging. Specifically, further progress is required to certify the knowledge and skills acquired during the learning processes conducted through SGs.

This research addresses the development of new methods to maximise the benefits of successful learning through games. To achieve convergence, we integrate certification of competency training. Unlike assessing learning, this approach validates the knowledge and skills acquired for professional functions or activities.

The Triadic Certification Method (TCM) aims to incorporate certification into SG development (from conception to design and implementation). This method involves four steps that influence game design, ensuring elements necessary for certification success. Communication between key stakeholders—the trainer/instructor and the designer—guarantees fundamental decisions regarding SG functionality.

The method, a testament to its versatility, is not restricted to game taxonomy and applies universally regardless of game type; it applies to any training context and adapts to diverse training scenarios. Skills acquired during training levels align with proficiency levels, reflecting the learning state. Certification occurs only when all defined competencies are successfully trained, reassuring professionals of its comprehensive applicability.

To address the research question about integrating certification into game development, we must embed the context of training and competency certification within the game design process. This inclusion necessitates rethinking the entire SG development chain. Additionally, we introduce a new team member—the instructor/coach—who defines skills and competencies. The instructor actively contributes to specifying contextual elements, such as characteristics, missions, specific objectives, and expected learning outcomes.

4.1. Relationship between Game Taxonomy and Competencies Development Survey

This research significantly defines the correlation matrix between game taxonomy and competencies. The effectiveness of training-based games, especially Serious Games, hinges on their ability to provide challenges that facilitate the acquisition of knowledge, experience, and professional aptitudes. However, there is no ideal design methodology to support this process.

A critical piece of information for game designers aiming to adapt mechanics for practical certification through SGs is the game genre. To address this, this study analyses standard options used in challenges based on a set of competencies. The evaluation draws from various game taxonomies, including those proposed by Adams and Dormans [92], Adams and Rollings [26], ESA [93], Bateman and Boon [94], Stahl [95], and Wolf [28]. Since no standard or universally accepted taxonomy exists, the researchers define their taxonomy as consisting of 8 categories subdivided into 22 subcategories.

Table 4 provides a quantitative overview of the analysed games, categorised by genre and subgenre. The genre with the highest number of Serious Games analysed is simulation, followed by puzzle and adventure games.

The chosen competency model, between the previous reference structures, is educational competencies, developed by Microsoft. This set of competencies aligns with current references and is considered essential for future success in performing various functions.

The research analyses 116 Serious Games from different sites and open repositories available in [96]. For each game, they collect information such as description, classification, domain areas, game genre, topics, audience, and type of realism. By analysing available data and, when possible, playing the games, the researchers identify the specific competencies involved.

The study’s results provided a cross-reference of the genre categories with a set of competencies in a matrix, allowing us to identify some areas with significant intersections to achieve learning outcomes [97,98,99].

Many genres’ potential to support aptitude learning is significant. It contributes in the same way to developing various game design strategies. The contribution of this study results in the mixture/combination of genres or the reinforcement of challenges to reach skills such as Decision Quality and Problem-Solving (SS3) and Technical Learning (SS8); Organisation (OpS5) and Timely Decision-Making (OpS9); and both results category competencies, Action-Oriented (R1) and Move for Results (R2), which can be synchronised in different strategies to achieve better student performance.

In summary, this research bridges the gap between game design, competencies, and certification, emphasising the importance of aligning game mechanics with desired learning outcomes. Developers can create more effective Serious Games for training and certification by understanding the interplay between game genres and competencies.

4.2. Design of Triadic Certification Method

The Triadic Certification Method (TCM) represents an important advancement in SGs, particularly concerning competency and skills certification. The TCM enables performance measurement during training missions by directly integrating training guidelines into SG design.

The TCM architecture comprises four steps, each contributing to evaluating skills acquisition and certification. Unlike traditional post-game questionnaires [99], the TCM assesses player performance within the game itself. This approach provides clear guidelines to the development team, especially designers, ensuring competencies are seamlessly woven into the game construction alongside an evaluation map.

The following figure, Figure 1, shows the method design:

In the first step, the analysis/diagnosis of the training context involves collaboration between the trainer/instructor and the development team. The training needs and competencies required for specific learning objectives are defined. Identifying situations, scenarios, and learning outcomes allows for detailed training planning, focusing on knowledge, skills, and aptitudes.

The TCM also leverages two methodologies: The Mission Essential Task List (METL) [100,101], which hierarchically lists essential tasks and activities, and the CRAWL-WALK-RUN Approach [102,103,104], which defines task sequences to promote progressive learning. Constructing a reference table for training scenarios associates training stages with lists of essential tasks, ensuring successful training by achieving expected performance levels. To visualise the scenarios, a concept similar to the use case diagrams (through the Unified Modelling Language—UML [105]) can be used to reference training scenario actors.

The second step of the method involves mapping educational competencies to align with the list of tasks and activities determined in the preceding stage, with the trainer or instructor remaining the central figure. During this phase, the focus shifts to defining the fundamental skills required by each target group profile, which will be developed through training. This definition is derived from aligning training competencies with educational competencies [87]. This mapping exercise contextualises the specific training environment with a standardised reference point, ensuring that skills acquisition can be appropriately assessed across different scenarios.

Initiating this stage of the method involves utilising a chosen reference matrix as a competency model to identify the essential competencies necessary for performing or training in a specific role across various contexts such as employment, occupation, organisation, or industry. The aim is to construct a behavioural description representative of the function to be performed based on the definition of competencies associated with each occupational role, as suggested by Fogg [106].

Once the competencies for a specific task or position have been identified through mapping from the reference matrix, the next step involves determining the most suitable actions within the game to achieve the learning objectives, known as game mechanics. Upon completing this step and defining the basic skills profile, attention shifts to the subsequent step: selecting the genre of the SG based on the correlation between game mechanics and basic skills.

The third step entails choosing the SG genre that best aligns with the previously defined basic skills profile. To accomplish this, extensive research on various Serious Games is conducted to comprehend the contributions of different gaming genres towards competency acquisition. While identifying the optimal game mechanics for skill acquisition can be challenging, analysing game genres aims to uncover patterns of competencies that specific game genres effectively encompass.

By conducting a high-level analysis of the mechanics in various Serious Games, designers can determine the most suitable game mechanics for acquiring specific skills. This methodology aspect falls under the designer’s responsibility, providing a guiding framework for game design while allowing room for creativity.

In certain instances, the choice of genre may not be straightforward but rather a combination of genres, where insights from the correlation matrix combine mechanics and challenges from various genres to train the desired skills effectively. A key conclusion drawn from this correlation matrix is the importance of leveraging past successful experiences with Serious Games to inform future development, thereby facilitating correct and efficient implementation.

The fourth and final step of the method involves integrating previous design contributions into the new game. This module aims to adapt the serious game design for skills training while maintaining autonomy in operation and configuration, contingent upon receiving values/elements related to player performance within established mechanics and challenges. Additionally, this step finalises the development of the Triadic Certification Method (TCMd).

The TCM serves as a communication tool among stakeholders, aiming to standardise game design for competency acquisition by balancing three components: identified skills and competencies (basic skills), mechanics and challenges based on game type (mechanics), and training levels (reality). Through the “in-game” certification method facilitated by the TCM, design contributions are defined, certified, and validated, ensuring that games effectively foster learning. Figure 2 is the Certification Triadic Model for training local tour guides, which comprises three reference axes: vertical, horizontal, and oblique. Each axis assumes a specific function to achieve defined competencies [107].

Starting with the vertical axis, it encompasses competencies aligned with various proficiency levels (basic, intermediate, advanced, expert), distributed across a mechanics framework. The progression of skills throughout the training sessions is marked by assigning the achieved proficiency level.

The horizontal axis represents the mechanics of skill acquisition, applied transversally across competencies. Learning progresses linearly through the accumulation of successful tasks, with higher competency levels indicating previous success.

The oblique axis refers to sessions or training levels, illustrating the progression of competencies from basic to expert levels. Each subsequent level builds upon the results of the previous level, fostering a cumulative advancement in proficiency profiles.

While the model emphasises progressive learning, it acknowledges that certain competencies may depend on a single mechanic or that a single training level may incorporate multiple mechanics. It also recognises the possibility of continuity or discontinuity in learning objectives between different training levels, with some outcomes remaining consistent across multiple proficiency profiles.

After presenting the model’s guidelines, we propose demonstrating its application as a customisable process since the starting point is always the context in which the skills are trained (reality). The Triadic Certification Method shown in Figure 2 refers to the training of local tour guides. Starting from this specific reality, it is a priority to understand the elements of certification for this activity, such as which tourist region and tourist resources contribute to a variety of tourist experiences: cultural, gastronomic, and traditional points of interest, whether it is one location or a group of locations. The certification of competencies for this professional is based on a successful profile that requires mastery of various areas of knowledge and skills, such as the geography of tourism, history, and cultural and architectural heritage, as well as culture and traditions of the regions, various types of communication (oral, written and active listening), group facilitator, mastery of several languages, and planning and organising tourist routes and circuits [99].

Following the TCM, various steps were defined (training scenarios, identifying the educational competencies to support certification, and finally, identifying the correlation of the expected competencies with the game genre and its most appropriate mechanics). Bearing in mind that scenarios for exploring the tourist region have been defined, the following competencies are defined:

-: Planning (OpS6), Organisation (OpS5), and Time Management (OpS8).
-: Written Communications (OrS4) and Presentation Skills (OrS3).
-: Action Orientated (R1).
-: Decision Quality and Problem Solving (SS3), Technical Learning (SS8), and Strategic Agility and Innovation Management (SS7).

These competencies correspond to various tasks related to the acquisition and demonstration of knowledge in the following aspects:

-: Monuments;
-: Cultural and architectural heritage (centuries-old and specific stories and traditions);
-: Understanding the people’s traditions as a key to organising and planning different thematic and tourist itineraries for different target audiences.

Considering the third step, the most suitable game genre combines both adventure genres: graphic and puzzle.

The game aims to use an avatar to explore an interactive story, which together takes on mental challenges in mini-games (puzzles and challenges) about tourist resources. Three training levels were defined to demonstrate the Triadic Certification Method’s construction: navigation, knowledge, and recommendation about a tourist region. With more detail on each training level, on the navigation level, the player will use a map of the tourist region, monitored by a GPS device whose route taken between two points of interest (POIs) will help validate route choices, time, and distance travelled. This level also ensures the physical recognition of routes and their POIs. As for the second level, knowledge acquisition occurs by identifying the POI and answering questionnaires and other challenges in various on-site situations. The questions will be trivia about random locations, destinations, or other more specific contexts relating to current or past events. The third and final level, the recommendation, has a double meaning: the extra motivation to share opinions and ratings on the spot of the various resources encountered and the collection of other helpful information from other participants that will be fundamental to the planning component of the thematic and other more specific itineraries that will have to be trained. The final classification of the route planning depends on the knowledge already acquired of the POIs included in the route, as well as the recommendations made to them.

However, we must bear in mind that the levels are cumulative, and it is necessary to collect several navigation routes between crucial tourist spots and the respective POIs found to ensure that all the other knowledge acquisition actions happen.

In conclusion, the Triadic Model offers a comprehensive and customisable framework for understanding and implementing skills training through Serious Games in different areas. It emphasises the integration of competencies, mechanics, and training levels to facilitate effective learning outcomes. In summary, the TCM bridges game design, competencies, and certification, emphasising the importance of aligning game mechanics with desired learning outcomes. By involving players in the design process, the TCM supports effective Serious Games for training and certification purposes.

5. Case Study of Driver’s Licences—Comprehensive Training Method for Light Vehicle Driving

To test our research hypothesis, we develop gaming applications, specifically prototypes, to support the case study of competence training for obtaining a driver’s licence. This case study focuses on road safety, which remains an enduring priority. Utilising games as a learning tool offers scalability of results, cost reduction, and solid consolidation of learning.

Acquiring driving skills is considered complex and dynamic because it involves various psychological processes on the driver’s part. This complexity can be broken down into three acquisition stages: information gathering, information processing, and action. New Serious Game-based learning tools can enhance the quality of driving skills acquisition and training to promote safer and more responsible drivers.

5.1. Case Study Context and Implementation in DRIS

The motivations for studying the acquisition of driver’s licences are twofold. Firstly, analysis of OECD (Young Drivers—The Road to Safety: Conference of Ministers of Transport (ECMT): OCDE 2006; https://www.oecd-ilibrary.org/transport/young-drivers_9789282113356-en (accessed on 7 July 2024)) data revealed that road accidents were a major cause of death among individuals aged 15 to 24. Deaths and serious injuries resulting from road accidents pose a significant public health issue, with young drivers being major contributors. Seeking validation solutions for driving tasks could gradually reduce this ongoing catastrophe.

Secondly, we tested this case study in the context of automotive driving learning using the virtual automobile simulator (DriS) in the Traffic Analysis Laboratory of the Civil Engineering Department at the Faculty of Engineering of the University of Porto [108,109]. The virtual simulator was configured with a training environment to monitor students’ real-time performance evolution and corresponding validation according to predefined plans/assignments and learning objectives. The simulation room has an image projection system with a projector and a screen. The driving position consists of a real vehicle (customised Volvo 440 turbo). Figure 3 shows images of the simulation room and its driving position. The vehicle’s integrated instrumentation includes sensors for actuating the pedals (clutch, brake, and accelerator). The car also has instrumentation for reading the gear engaged, direction indicators, the position of the ignition key, and all the light controls.

Utilising DriS, our objective was to test the application of the concept of validating vehicle control and mastery skills (operational) and adaptation to constant changes in the road environment (tactical) based on the learning support matrix of Driving: GDE—Goals for Driver Education [110,111,112]. This matrix hierarchically defines the driving task, emphasising individual driver characteristics impacting driving, including experience, attitudes, skills, motivations, decisions, and behaviours. This matrix allows for defining educational objectives and performance indicators in driver training as a tool for defining the skills necessary to become a safe driver. Understanding learning guidelines is vital, as they indicate that some areas must be learned before others may progress and that the development of different components has varying timings.

Feedback is another crucial aspect of learning to drive, informing driving practises at higher- and lower-order factors. In the first case, feedback acts as a regulator and behavioural motivator, while in the second case, it actively engages the trainee throughout the driving task, connecting to necessary automatisms and procedures. The amount of feedback perceived in the driving task at this lower level is greater, suggesting that low-level skills are learned faster than high-level skills. A prototype was implemented for the training certification system module to validate the acquisition of a driving licence. Developed for the Windows Operating System, the module aims to integrate with any serious game, providing a set of metrics representing the necessary mechanics for real-time training. The module’s coupling is intended to be generic and established through communication via “sockets” in TCP/IP networks. The programming module is designed to be compatible with other application cases through the definition of distinct projects associated with a separate database file. This option ensures independence and portability for each application case.

The tool allows each participant to independently interact with three distinct work areas in the training process: trainer, who defines training competencies; game designer, who defines game elements and mechanics associated with training; and certification, as the training outcome component, which analyses student performance data and provides feedback on the training plan. The feedback system is a critical component in this implementation. It enables monitoring of tasks performed by students during implementation through visual and auditory means.

5.2. TCM Design

The implementation of the TCM commences with the analysis of training scenarios. To conduct this competence diagnosis, it was imperative to consult the legal code of the road and driving instructional manuals, which organise various theoretical themes of road safety, traffic rules, and traffic signals. Additionally, technical files developed by the IMT (Portuguese public entity regulating mobility and transport) and testimonies from professional instructors were utilised to understand the procedures for initiating the practise of driving a Category B motor vehicle.

Driving instruction in Portugal follows a matrix structure allowing for the organization of various levels of learning for drivers. The legislative and regulatory responsibility for driving education lies with the Portuguese State through the Institute of Mobility and Transport (IMT, IP). Based on the gathered information about the practical learning of driving tasks, frameworks of learning phases were developed, aligning with the Portuguese driving education system.

In the initial step of this Triadic Method, we conduct a meticulous analysis of the tasks and competencies essential for driving light vehicles. Our objective is to design distinct training levels that ensure thorough mastery of these competencies, ultimately preparing drivers for real-world scenarios.

We have identified three core competencies crucial for effective driver training: speed adaptation and vehicle control, complete vehicle mastery, and traffic situation resolution. Each competency is broken down into specific learning topics, which are then linked to targeted tasks, ensuring a comprehensive learning experience.

Our training strategy is founded on the proven “Crawl-Walk-Run” methodology. In the “Crawl” phase, trainees begin with fundamental tasks, building a solid foundation of basic skills. In the “Walk” phase, the difficulty and realism of tasks gradually increase, promoting steady progress. Finally, in the “Run” phase, trainees achieve high-level performance through advanced practise, simulating real-world driving conditions.

Using the Mission Essential Task List (METL) methodology, we create a structured hierarchy of tasks for each training mission. Training scenarios are designed to increase in complexity progressively, allowing learners to build on their knowledge and skills systematically. For this case study, we focus on three key competencies: speed control, navigating crossings and intersections, and manoeuvring through wide curves.

To instil automatic responses in trainees, we include specific tasks such as safe vehicle startup and stopping, speed changes emphasising the coordination of gearbox and pedals, light driving on straight and curved tracks, and defensive driving decision-making based on road signs. A variety of routes are created to incorporate different driving situations, connecting various activities through specific route signalling, ensuring comprehensive learning.

The routes (T1 to T5) include right and left curves with appropriate signalling. We simulate real traffic conditions by placing other vehicles in the trainee’s path and at intersections, enhancing the realism of the training. Driver performance is assessed based on compliance with signage and successful task execution along the routes, covering all competencies cumulatively.

The interconnected routes allow for continuous competency training. During evaluation, these connections ensure that the training paths align seamlessly with the assessment scenarios, promoting a cohesive learning experience. The sequence of training paths, detailed in Figure 4, showcases the interconnection of routes and competencies:

This comprehensive training framework ensures drivers progressively acquire essential driving skills, leading to superior performance in real-world situations. Our meticulous approach guarantees that every driver is thoroughly prepared, confident, and safe on the road. Embrace this training method to master the art of driving light vehicles and transform your driving experience today.

The second step in our Triadic Method involves competency mapping, focusing on four specific competencies: speed limit control, in-road vehicle control, and approach to crossings and junctions, distinguished by STOP signalling and signalling with and without right of way. By aligning these competencies with our educational competency matrix, we identified that they align best with strategic skills (SSs) and operational skills (OpSs).

From our reference matrix (Figure 5), the following competencies emerged as critical:

Strategic skills (SSs): Decision Quality and Problem Solving (SS3) and Functional/Technical Skills (SS4)
Operational skills (OpSs): Planning (OpS6), Priority Setting (OpS7), and Timely decision-making (OpS9)
Results (Rs): Action Oriented (R1)

In the third step, we selected the game genre based on a correlation matrix between game genres and competence benchmarks. This step ensures the skills identified are seamlessly integrated into the game design, facilitating effective training. Our analysis revealed that action, strategy, and simulation genres had the highest success rates. Thus, our case study involves a hybrid of simulation games with vehicles (such as rally or heavy-vehicle driving) combined with action and strategy elements. This blend allows us to leverage individual mechanics effectively, with lower levels engaging in action mechanics and higher levels incorporating strategic elements.

In the fourth step, we implemented the game using this combination of mechanics, aligned with the chosen game taxonomy. This approach enabled us to pinpoint specific challenges that correlate with the game’s objectives. For each selected genre, we identified mechanics and actions crucial for training the anticipated competencies:

Spatial perception: enhancing the ability to navigate through the game environment to develop a spatial relationship essential for reaching destinations.
Points: providing feedback on progress within the scenario, enhancing visualisation and goal tracking.
Levels: introducing new sets of challenges in different scenarios to demonstrate progression.
Detailed simulation actions: including acceleration, deceleration with pedals and gearbox, braking, coordinating the vehicle within the lane, and stopping the car.

Figure 6 illustrates the mapping of competencies within the training plan, where each horizontal axis corresponds to a different path (e.g., Path 1, Path 2). The alignment between skills and paths is established through the mechanics or challenges implemented in the game.

Each defined competency has a scalable profile (1—basic, 2—intermediate, 3—advanced, 4—expert) associated with it, classifying the use of mechanics in the task. This profile classification serves a dual purpose: categorising all mechanics and competency alignments with a learning level and contextualising the student during training tasks to achieve learning outcomes.

Each training path incorporates one or more skills that can be learned or trained sequentially. This mapping allows us to track the evolution of competency along the scale, contingent on the successful performance of the mechanics. This structure frames the learning of competencies within each path, enabling students to accumulate evidence of their learning by successfully performing tasks through mechanics. Post-training, the learning outcome is determined by combining successful evidence from various mechanics, creating a profile, and then assigning the achieved degree of competence.

By adopting this method, we ensure a structured, progressive, and practical approach to mastering light vehicle driving. This method prepares drivers comprehensively and instils confidence and ensures safety on the road, ultimately transforming the driving experience.

Mapping of competencies along the competency training plan allows for alignment between skills and paths through mechanics or challenges implemented in the game. Each training path incorporates one or more skills that can be learned or trained sequentially.

If the mechanics’ performance is successful, the evolution of competency along the scale is verified. This structure enables framing the learning of competencies within each path, accumulating evidence of learning through successful task performance. After training, the learning result is achieved by combining successful evidence from various mechanics, creating a profile, and assigning the achieved degree of competence.

This comprehensive methodology ensures a systematic approach to competency training in driving education, facilitating effective learning outcomes for drivers.

5.3. Analysis of Results

Following the completion of the tests, 50 volunteers participated, comprising 38 men and 12 women aged 18 to 65. Unfortunately, four participants withdrew prematurely due to simulation-induced nausea. The remaining 46 participants were categorised into two groups based on whether they held a driver’s licence: 31 participants held licences, while 15 did not, as delineated in Table 5.

All participants underwent a comprehensive familiarisation process with the simulator throughout the experiment. This involved understanding acceleration, deceleration, steering wheel control, and traffic signs. Even those without a driver’s licence were fully briefed on traffic signs and their meanings. The data collected from the experiment provided a meticulous analysis of each competency, showcasing the individual learning progression under the well-structured training plan.

Although the study presented in this document has a more methodological focus, the instrument validation must be addressed. To analyse and process the data collected with the user tests, we considered various statistical instruments before and after the descriptive analysis of the results, which we do not present here. Firstly, we check the internal consistency of the data using Cronbach’s alpha test, with two separate analyses: one for performance by competence and the other for overall performance. Secondly, the sample distribution was validated using the frequency and standard distribution curve, and lastly, the correlation between the variables through Pearson’s coefficient (r).

We considered qualitative variables such as gender, driving ability, and age to assess participant performance during training (paths 1, 2, 3, and 5) and evaluation (paths 7 and 8). Performance was analysed based on both components’ success in training and evaluation and the transition from training failure to evaluation success. Graphical data analysis allowed us to evaluate overall performance differences and individual progress.

The analysis of speed competence revealed high failure rates globally and in specific segments. Only 18 out of 46 participants succeeded, possibly due to the training design and the imposed 40 km/h speed limit. The repetitive nature of the training routes likely caused disinterest and monotony, contributing to the low success rate. In contrast, the control of the car within the lane showed a positive learning evolution, with a 10% improvement despite a reduced growth margin. This positive trend was further reinforced by the fact that most participants (37 out of 41) had completed training and evaluation.

There was an improvement in the competence of approaching crossings and junctions with signalling. The success rates increased from 9 participants in training to 23 in evaluation, marking a remarkable 255% improvement. However, the slight difference between training and both moments (7 participants) indicated that many struggled with mandatory STOP signalling.

The right-of-way competence also saw positive results, with 31 out of 46 participants (67%) achieving success. The desired behaviour improved significantly, with 17 participants showing positive evolution between training and evaluation. Globally, the performance evaluation indicated that 86% of successful participants were qualified to drive, validating our training plan. Only seven participants (six males and one female, with four under 23) successfully acquired all four competencies. The low overall success rate could be attributed to the repetitive nature of the training routes and the 40 km/h speed limit, which caused frustration and demotivation.

The CRAWL-WALK-RUN method, which involves sequential and repetitive task performance, yielded limited success in skills acquisition. Despite this, the high success rate among participants with driver’s licences (86%) confirms the validity of our training method for designing and validating competencies.

In conclusion, despite some challenges, our comprehensive training method effectively prepares participants for real-world driving scenarios. It ensures a structured, progressive, and practical approach to mastering light vehicle driving. This method instils confidence, ensures safety, and transforms the driving experience.

6. Conclusions

In recent years, Serious Games have emerged as a compelling alternative for acquiring, training, and certifying skills because they provide a more engaging and meaningful learning experience. By incorporating rules, behavioural simulations, and feedback mechanisms, Serious Games create an environment where learners can make mistakes without real-life consequences and receive instant feedback. However, Serious Games must be designed using appropriate training validation and certification methodologies. The gameplay element is crucial for progression and successful learning outcomes.

This research was conducted with a rigorous and innovative approach to answer whether using player performance and interaction with the game to certify acquired competencies is possible. A comprehensive literature review confirmed the relevance of this question, leading to a focused exploration of integrating evaluation processes and certification within game-based learning, following specific norms for measuring student performance.

The conclusions of this research underscore the unique features of the Triadic Certification Method, distinguishing it from the existing literature and providing it with strong academic value. The integration of the correlation matrix between competencies and game taxonomy is a pioneering contribution, enabling game designers to craft Serious Games that both entertain and foster effective and specific learning of competencies. This breakthrough sets a benchmark that can be applied to diverse training contexts, potentially replicating best practises in developing new Serious Games.

In addition, the study comprehensively identifies the aspects of game design that favour competency-based assessment and training, bringing them in line with the demands of the 21st century. Structuring the four-step Triadic Certification Method provides a systematic approach to measuring and certifying player performance, using learning metrics that go beyond the traditional ones and incorporate elements such as agility, attention, cooperation, and precision.

Integrating competence matrices into serious game design offers practical and context-specific training scenarios. This aspect empowers designers to tailor game mechanics for skill acquisition, enhancing Serious Games’ effectiveness. The introduction of the Triadic Certification Method, which balances basic skills, game mechanics, and reality, stands out for its ability to keep those involved in the training and certification process independent while promoting effective communication between them. This visual model enables situational awareness of the progression of skills acquisition over the various training levels, ensuring a robust and visually comprehensible methodology.

Another important result was the integration of learning metrics into in-game evaluation systems. Implementing performance validation based on scores derived from successful activities within a training plan could assess the effectiveness of knowledge and skill acquisition and offer a viable alternative to traditional validation methods such as surveys and observations. The ability to quantify training success based on in-game results reinforces the effectiveness of the Triadic Certification Method, opening up new possibilities for assessing and certifying competencies in innovative learning environments.

Overall, this research validates using Serious Games in skills certification and introduces pioneering tools and methodologies. These innovations have the potential to transform educational and training practises in various areas, offering new and effective ways to assess and certify competencies.

Author Contributions

Conceptualization, R.B.; methodology, R.B., A.C. and C.V.d.C.; software, R.B.; validation, R.B., A.C. and C.V.d.C.; investigation, R.B.; resources, R.B.; data curation, R.B.; writing—original draft preparation, R.B.; writing—review and editing, A.C. and C.V.d.C.; supervision, A.C. and C.V.d.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

The data presented in this study are openly available in Ricardo Baptista Ph.D. Thesis (https://hdl.handle.net/10216/110820 (accessed on 7 July 2024)).

Conflicts of Interest

The authors declare no conflicts of interest.

References

Uskov, A.; Sekar, B. Serious games, gamification and game engines to support framework activities in engineering: Case studies, analysis, classifications and outcomes. In Proceedings of the IEEE International Conference on Electro/Information Technology, Milwaukee, WI, USA, 5–7 June 2014; pp. 618–623. [Google Scholar]
Boller, S.; Kapp, K. Play to Learn; ATD Press: Alexandria, VA, USA, 2017. [Google Scholar]
Sousa, L.; Figueiredo, M.; Monteiro, J.; José, B.; Rodrigues, J.; Cardoso, P. Developments of Serious Games in Education. In Handbook of Research on Human-Computer Interfaces, Developments, and Applications; IGI Global: Hershey, PA, USA, 2016; pp. 392–419. [Google Scholar]
Mayer, R.E. Computer Games for Learning: An Evidence-Based Approach; MIT Press: Cambridge, MA, USA, 2014. [Google Scholar]
de Freitas, S.; Jarvis, S. A framework for developing serious games to meet learner needs. In Proceedings of the Interservice/Industry Training, Simulation and Education Conference, Orlando, FL, USA, 1 December 2006. [Google Scholar]
Luo, L.; Yin, H.; Cai, W.; Lees, M.; Zhou, S. Interactive scenario generation for mission-based virtual training: Interactive scenario generation. Comput. Animat. Virtual Worlds 2013, 24, 345–354. [Google Scholar] [CrossRef]
Huang, W.-H.D.; Tettegah, S.Y. Cognitive Load and Empathy in Serious Games: A Conceptual Framework. In Gamification for Human Factors Integration: Social, Education, and Psychological Issues; Jonathan, B., Ed.; IGI Global: Hershey, PA, USA, 2014; pp. 17–30. [Google Scholar]
Kiili, K.; Lainema, T.; de Freitas, S.; Arnab, S. Flow framework for analysing the quality of educational games. Entertain. Comput. 2014, 5, 367–377. [Google Scholar] [CrossRef]
Emmerich, K.; Bockholt, M. Serious Games Evaluation: Processes, Models, and Concepts. In Entertainment Computing and Serious Games: International GI-Dagstuhl Seminar 15283; Dörner, R., Göbel, S., Kickmeier-Rust, M., Masuch, M., Zweig, K., Eds.; Springer International Publishing: Dagstuhl, Germany, 2015; pp. 265–283. [Google Scholar]
Andreoli, R.; Corolla, A.; Faggiano, A.; Malandrino, D.; Pirozzi, D.; Ranaldi, M.; Santangelo, G.; Scarano, V. A Framework to Design, Develop, and Evaluate Immersive and Collaborative Serious Games in Cultural Heritage. J. Comput. Cult. Herit. 2017, 11, 1–22. [Google Scholar] [CrossRef]
Coleman, T.E.; Money, A.G. Student-centered digital game-based learning: A conceptual framework and survey of state of the art. High Educ. 2020, 79, 415–457. [Google Scholar] [CrossRef]
Mitgutsch, K.; Alvarado, N. Purposeful by design? A Serious Game design assessment framework. In Proceedings of the FDG ’12: International Conference on the Foundations of Digital Games, Raleigh, NC, USA, 29 May 2012; pp. 121–128. [Google Scholar]
Wilson, D.W.; Jenkins, J.; Twyman, N.; Jensen, M.; Valacich, J.; Dunbar, N.; Wilson, S.; Miller, C.; Adame, B.; Lee, Y.-H.; et al. Serious games: An evaluation framework and case study. In Proceedings of the 2016 49th Hawaii International Conference on System Sciences (HICSS), Washington, DC, USA, 5–8 January 2016; Sprague, R.H., Bui, T.X., Eds.; IEEE Computer Society: Washington, DC, USA, 2016. [Google Scholar]
Ulrich, F.; Helms, N.H. Creating Evaluation Profiles for Games Designed to Be Fun: An Interpretive Framework for Serious Game Mechanics. Simul. Gaming 2017, 48, 695–714. [Google Scholar] [CrossRef]
Tahir, R.; Wang, A.I. Codifying game-based learning: The league framework for evaluation. In Proceedings of the 12th European Conference on Game Based Learning—ECGBL 2018, Sophia Antipolis, France, 4–5 October 2018; pp. 4–5. [Google Scholar]
Huizinga, J.; Hull, R.C. Homo Ludens. A study of the Play-Element in Culture, 1st ed.; Beacon Press Boston: Boston, MA, USA, 2014. [Google Scholar]
Caillois, R.; Halperin, E.P. The structure and classification of games. Diogenes 1955, 3, 62–75. [Google Scholar] [CrossRef]
Juul, J. The repeatedly lost art of studying games. Game Stud. 2001, 1. Available online: https://www.gamestudies.org/0101/juul-review/?ref=driverlayer.com (accessed on 30 May 2024).
Salen, K.; Zimmerman, E. Rules of Play: Game Design Fundamentals; The MIT Press: Cambridge, MA, USA, 2004. [Google Scholar]
de Carvalho, C.V.; Batista, R. Work in progress-learning through role-play games. In Proceedings of the 38th Annual Frontiers in Education Conference, Saratoga Springs, NY, USA, 22 October 2008; p. T3C-7. [Google Scholar]
Bergson, B. Developing Serious Games; Game Development Series, 1st ed.; Charles River Media, Inc.: Needham, MA, USA, 2006. [Google Scholar]
Apperley, T.H. Genre and game studies: Toward a critical approach to video game genres. Simul. Gaming 2006, 37, 6–23. [Google Scholar] [CrossRef]
Breuer, J.S.; Bente, G. Why so serious? On the relation of serious games and learning. Eludamos J. Comput. Game Cult. 2010, 4, 7–24. [Google Scholar] [CrossRef] [PubMed]
Adams, E. Fundamentals of Game Design; Pearson Education: London, UK, 2014. [Google Scholar]
Pinelle, D.; Wong, N.; Stach, T. Using genres to customize usability evaluations of video games. In Proceedings of the 2008 Conference on Future Play Research, Play, Share-Future Play ’08, New York, NY, USA, 3 November 2008; pp. 129–136. [Google Scholar]
Adams, E.; Dormans, J. Game Mechanics: Advanced Game Design, 1st ed.; New Riders: San Francisco, IN, USA, 2012. [Google Scholar]
Grace, L. Game Type and Game Genre. Available online: http://aii.lgracegames.com/documents/Game_types_and_genres.pdf (accessed on 30 May 2024).
Wolf, M.J. The Medium of the Video Game, 1st ed.; University of Texas Press: Austin, TX, USA, 2002. [Google Scholar]
Michael, D.R.; Chen, S. Serious Games: Games That Educate, Train, and Inform; Muska Lipman/Premier: Cincinnati, OH, USA, 2005. [Google Scholar]
Tadayon, R.; Amresh, A.; Burleson, W. Socially relevant simulation games: A design study. In Proceedings of the 19th ACM International Conference on Multimedia, Scottsdale, AZ, USA, 28 November 2011; pp. 941–944. [Google Scholar]
Abt, C.C. Serious Games; University Press of America: Lanham, MA, USA, 1987. [Google Scholar]
Marlow, C.M. Games Learning in Landscape Architecture. In Proceedings of the 10th International Conference on Information Technologies in Landscape Architecture-Digital Landscape Architecture, Valletta, Malta, 5–7 June 2024; pp. 129–136. [Google Scholar]
Becker, K. How are games educational? Learning theories embodied in games. In Proceedings of the DiGRA 2005 Conference: Changing Views: Worlds in Play, Vancouver, BC, Canada, 1 January 2005. [Google Scholar]
Zyda, M. From visual simulation to virtual reality to games. Computer 2005, 38, 25–32. [Google Scholar] [CrossRef]
Wiberg, C.; Jegers, K. Satisfaction and learnability in edutainment: A usability study of the knowledge game ‘Laser Challenge’ at the Nobel e-museum. In Proceedings of the HCI International—10th International Conference on Human Computer Interaction, Crete, Greece, 22 June 2003. [Google Scholar]
Wigfield, A.; Guthrie, J.T.; Tonks, S.; Perencevich, K.C. Children’s motivation for reading: Domain specificity and instructional influences. J. Educ. Res. 2004, 97, 299–310. [Google Scholar] [CrossRef]
Deci, E.L.; Koestner, R.; Ryan, R.M. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychol. Bull. 1999, 125, 627. [Google Scholar] [CrossRef]
Lepper, M.R.; Greene, D.; Nisbett, R.E. Undermining children’s intrinsic interest with extrinsic reward: A test of the “over justification” hypothesis. Psychol. Bull. 1973, 28, 129. [Google Scholar] [CrossRef]
Susi, T.; Johannesson, M.; Backlund, P. Serious Games: An Overview. 2007. Available online: https://web.archive.org/web/20240327104441/https://www.diva-portal.org/smash/get/diva2:2416/fulltext01.pdf (accessed on 30 May 2024).
Stratton, G.; Ridgers, N. Energy expenditure in adolescents playing new generation computer games. Br. J. Sport Med. 2008, 42, 592–594. [Google Scholar]
Lieberman, D.A. Interactive video games for health promotion: Effects on knowledge, self-efficacy, social support, and health. In Health Promotion and Interactive Technology: Theoretical Applications and Future Directions; Street, R.L., Gold, W.R., Manning, T.R., Eds.; Routledge: New York, NY, USA, 2013; pp. 103–120. [Google Scholar]
Forstner, B.; Szegletes, L.; Angeli, R.; Fekete, A. A general framework for innovative mobile biofeedback based educational games. In Proceedings of the 2013 IEEE 4th International Conference on Cognitive Infocommunications (CogInfoCom), Budapest, Hungary, 2 December 2013; pp. 775–778. [Google Scholar]
Griffiths, M. The therapeutic use of videogames in childhood and adolescence. Clin. Child Psychol. Psychiatry 2003, 8, 547–554. [Google Scholar] [CrossRef]
McCallum, S.; Boletsis, C. A taxonomy of serious games for dementia. In Games for Health: Proceedings of the 3rd European Conference on Gaming and Playful Interaction in Health Care; Springer Science & Business Media: Berlin, Germany, 2013; pp. 219–232. [Google Scholar]
Ratan, R.A.; Ritterfeld, U. Classifying serious games. In Serious Games, 1st ed.; Routledge: London, UK, 2009; pp. 32–46. [Google Scholar]
Mohamed, H.; Jaafar, A. Development and potential analysis of heuristic evaluation for educational computer game (PHEG). In Proceedings of the 5th International Conference on Computer Sciences and Convergence Information Technology, Seoul, Republic of Korea, 30 November 2010; pp. 222–227. [Google Scholar]
Boyle, L.; Hancock, F.; Seeney, M.; Allen, L. The Implementation of Team Based Assessment In Serious Games. In Proceedings of the 2009 Conference in Games and Virtual Worlds for Serious Applications, Marrakesh, Morocco, 23 March 2009; pp. 28–35. [Google Scholar]
Gunter, G.A.; Kenny, R.F.; Vick, E.H. Taking educational games seriously: Using the RETAIN model to design endogenous fantasy into standalone educational games. Educ. Technol. Res. Dev. 2008, 56, 511–537. [Google Scholar] [CrossRef]
Kirkpatrick, D. Great ideas revisited. Train. Dev. 1996, 50, 54–59. [Google Scholar]
Baker, E.L.; Mayer, R.E. Computer-based assessment of problem solving. Comput. Hum. Behav. 1999, 15, 269–282. [Google Scholar] [CrossRef]
Wilcox, D. Design and implementation of educational games: Theoretical and practical perspectives. In Design and Implementation of Educational Games: Theoretical and Practical Perspectives; Zemliansky, P., Wilcox, D., Eds.; IGI Global: Hershey, PA, USA, 2007. [Google Scholar]
Roodt, S.; Joubert, P. Evaluating Serious Games in Higher Education: A Theory-based Evaluation of IBMs Innov8. In Proceedings of the 3rd European Conference on Games Based Learning, Graz, Austria, 12–13 October 2009; pp. 332–339. [Google Scholar]
Mislevy, R.J.; Steinberg, L.S.; Almond, R.G. Focus article: On the structure of educational assessments. Meas. Interdiscip. Res. Perspect. 2003, 1, 3–62. [Google Scholar] [CrossRef]
Becker, K.; Parker, J. Serious Instructional Design: ID for digital simulations and games. In Proceedings of the Society for Information Technology Teacher Education International Conference, Austin, TX, USA, 25 March 2012; pp. 2480–2485. [Google Scholar]
Arieli-Attali, M.; Ward, S.; Thomas, J.; Deonovic, B.; Von Davier, A.A. The Expanded Evidence-Centered Design (e-ECD) for Learning and Assessment Systems: A Framework for Incorporating Learning Goals and Processes Within Assessment Design. Front. Psychol. 2019, 10, 853. [Google Scholar] [CrossRef] [PubMed]
Muftuoglu, C.T.; Sahin, M. Framework of Assessment Design Based on Evidence-Centered Design for Assessment Analytics. In Assessment Analytics in Education: Designs, Methods and Solutions; Springer International Publishing: Cham, Switzerland, 2024; pp. 157–172. [Google Scholar]
De Freitas, S.; Liarokapis, F. Serious games: A new paradigm for education? In Serious Games and Edutainment Applications; Ma, M., Oikonomou, A., Jain, L., Eds.; Springer: London, UK, 2011; pp. 9–23. Available online: https://link.springer.com/chapter/10.1007/978-1-4471-2161-9_2#citeas (accessed on 7 July 2024).
Van Staalduinen, J.P.; De Freitas, S. A game-based learning framework: Linking game design and learning outcomes. In Learning to Play: Exploring the Future of Education with Video Games; International Academic Publishers: Hershey, PA, USA, 2011; Volume 53, pp. 29–45. [Google Scholar]
Harteveld, C. Triadic Game Design: Balancing Reality, Meaning and Play; Springer Science Business Media: Berlin, Germany, 2011. [Google Scholar]
Kellerhals, U.; Burgess, N.; Wetzel, R. Let it Bee: A Case Study of Applying Triadic Game Design for Designing Virtual Reality Training Games for Beekeepers. In Proceedings of the 16th International Conference on the Foundations of Digital Games, Montreal, QC, Canada, 3 July 2021; pp. 1–12. [Google Scholar]
Troiano, G.M.; Schouten, D.; Cassidy, M.; Tucker-Raymond, E.; Puttick, G.; Harteveld, C. All good things come in threes: Assessing student-designed games via triadic game design. In Proceedings of the 16th International Conference on the Foundations of Digital Games, Bugibba, Malta, 15 September 2020; pp. 1–4. [Google Scholar]
Winn, B.M. The Design, Play, and Experience Framework. In Handbook of Research on Effective Electronic Gaming in Education; Ferdig, R.E., Ed.; IGI Global: Hershey, PA, USA, 2009; pp. 1010–1024. [Google Scholar]
Mellecker, R.; Lyons, E.J.; Baranowski, T. Disentangling fun and enjoyment in exergames using an expanded design, play, experience framework: A narrative review. Games Health Res. Dev. Clin. Appl. 2013, 2, 142–149. [Google Scholar] [CrossRef] [PubMed]
Romero, M.; Ouellet, H.; Sawchuk, K. Expanding the game design play and experience framework for game-based lifelong learning (GD-LLL-PE). In Game-Based Learning across the Lifespan: Cross-Generational and Age-Oriented Topics; Springer: Berlin/Heidelberg, Germany, 2017; pp. 1–11. [Google Scholar]
Kiili, K. Content creation challenges and flow experience in educational games: The IT-Emperor case. Internet High. Educ. 2005, 8, 183–198. [Google Scholar] [CrossRef]
Zapušek, M.; Rugelj, J. Learning programming with serious games. EAI Endorsed Trans. Game-Based Learn. 2013, 13. Available online: https://eudl.eu/doi/10.4108/trans.gbl.01-06.2013.e6 (accessed on 7 July 2024). [CrossRef]
Cetto, A.; Netter, M.; Pernul, G.; Richthammer, C.; Riesner, M.; Roth, C.; Sänger, J. Friend inspector: A serious game to enhance privacy awareness in social networks. arXiv 2014, arXiv:1402.5878. [Google Scholar]
Song, M.; Zhang, S. EFM: A model for educational game design. In Proceedings of the Technologies for E-Learning and Digital Entertainment: Third International Conference, Edutainment 2008, Nanjing, China, 25 June 2008; Springer: Berlin/Heidelberg, Germany, 2008; pp. 509–517. [Google Scholar]
Abd Elsattar, H.K.H. Designing for game-based learning model: The effective integration of flow experience and game elements to support learning. In Proceedings of the 14th International Conference on Computer Graphics, Imaging and Visualization, Marrakesh, Morocco, 22 May 2017; pp. 34–43. [Google Scholar]
Conole, G. Assessment as a Catalyst for Innovation. Available online: https://eprints.soton.ac.uk/9735/ (accessed on 7 July 2024).
Conole, G.; Warburton, B. A review of computer-assisted assessment. ALT-J 2005, 13, 17–31. [Google Scholar] [CrossRef]
Wills, G.B.; Bailey, C.P.; Davis, H.C.; Gilbert, L.; Howard, Y.; Jeyes, S.; Millard, D.E.; Price, J.; Sclater, N.; Sherratt, R. An e-learning framework for assessment (FREMA). Assess. Eval. High. Educ. 2009, 34, 273–292. [Google Scholar] [CrossRef]
Armstrong, M. A Handbook of Human Resource Management Practice, 10th ed.; Kogan Page: London, UK, 2006. [Google Scholar]
Flippo, E.B. Personal Management; Tata McGraw Hill: New York, NY, USA, 1992. [Google Scholar]
Aswathappa, K. Human Resource and Personnel Management; Tata McGraw-Hill Education: New York, NY, USA, 2005. [Google Scholar]
Flarey, D.L. Is Certification the Current Gold Standard? JONA’S Healthc. Law Ethics Regul. 2000, 2, 43. [Google Scholar]
Byrne, M.; Valentine, W.; Carter, S. The Value of Certification—A Research Journey. AORN J. 2004, 79, 825–835. [Google Scholar] [CrossRef]
Briggs, L.A.; Brown, H.; Kesten, K.; Heath, J. Certification a benchmark for critical care nursing excellence. Crit. Care Nurse 2006, 26, 47–53. [Google Scholar] [CrossRef] [PubMed]
Kaplow, R. The value of certification. AACN Adv. Crit. Care 2011, 22, 25–32. [Google Scholar] [CrossRef]
Aswathappa, K.; Atif, A.; Richards, D.; Bilgin, A.; Marrone, M.; Nyberg, A. The value of certification. In Proceedings of the 6th International Conference of Education, Research and Innovation, Seville, Spain, 18 November 2013; p. 34. [Google Scholar]
Koeppen, K.; Hartig, J.; Klieme, E.; Leutner, D. Current issues in competence modeling and assessment. Z. Psycholo-Gie/J. Psychol. 2009, 216, 61–73. [Google Scholar] [CrossRef]
Coles, M. Qualifications Frameworks in Europe: Platforms for Qualifications, Integration and Reform; EU, Education and Culture DG: Brussels, Belgium, 2007. [Google Scholar]
Méhaut, P.; Winch, C. The European Qualification Framework: Skills, Competences or Knowledge? Eur. Educ. Res. J. 2012, 11, 369–381. [Google Scholar] [CrossRef]
Ferrari, A. Digital Competence in Practice: An Analysis of Frameworks; Publications Office of the European Union: Luxembourg, 2012; Volume 10, p. 82116. [Google Scholar]
Lombardo, M.M.; Eichinger, R.W. The Leadership Machine; Lominger Limited: Minneapolis, MN, USA, 2006. [Google Scholar]
Garman, A.N.; Johnson, M.P. Leadership competencies: An introduction. J. Healthc. Manag. 2006, 51, 13–17. [Google Scholar] [CrossRef]
Education Competency Wheel. Available online: https://download.microsoft.com/download/3/4/7/3477e49d-315d-4ee7-a8ca-ff653a4455d6/Competency_Wheel.pdf (accessed on 30 May 2024).
Murthy, S. Academagogical Framework for Effective University Education-Promoting Millennial Centric Learning in Global Knowledge Society. In Proceedings of the 2011 IEEE International Conference on Technology for Education, Washington, DC, USA, 14 July 2011; pp. 289–290. [Google Scholar]
Fletcher, G.H. Building a Better CTO: CoSN Creates a New Framework of Skills That Draws a Greater Range of Responsibility for the 21st Century Technology Leader. The Free Library. 1 January 2010. Available online: https://thejournal.com/articles/2010/01/08/building-a-better-cto.aspx (accessed on 14 May 2024).
Hine, P. UNESCO ICT Competency Framework for Teachers; United Nations Educational: Paris, France, 2011. [Google Scholar]
Hubregtse, M.; Moerbeek, S.; Veldkamp, B.P.; Eggen, T. Testing competences worldwide in large numbers: Complying with ISO/IEC 17024: 2012 standard for competence testing. In Proceedings of the Computer Assisted Assessment. Research into E-Assessment: 18th International Conference, Zeist, The Netherlands, 22–23 June 2015; pp. 27–39. [Google Scholar]
Adams, E.; Dormans, J. Fundamentals of Game Design, 1st ed.; Game Design and Development Series; Prentice-Hall, Inc.: Hoboken, NJ, USA, 2006. [Google Scholar]
2015 Essential Facts about the Computer Video Game Industry. Available online: https://www.theesa.com/wp-content/uploads/2024/02/2015-EF-FINAL.pdf (accessed on 30 May 2024).
Bateman, C.; Boon, R. 21st Century Game Design; Game Development Series; Charles River Media, Inc.: Massachusetts, MA, USA, 2005. [Google Scholar]
Stahl, T. Video Game Genres. The History of Computing Project 2005. Available online: www.thocp.net/software/games/reference/genres.htm (accessed on 2 June 2014).
Baptista, R. Jogos Sérios Para Treino E Certificação de Competências. Ph.D. Thesis, University of Porto, Porto, Portugal, 21 December 2017. [Google Scholar]
Baptista, R.; Coelho, A.; de Carvalho, C.V. Relationship Between Game Categories and Skills Development: Contributions for Serious Game Design. In Proceedings of the ECGBL2015-9th European Conference on Games Based Learning, Steinkjer, Norway, 8 October 2015; p. 34. [Google Scholar]
Baptista, R.; Coelho, A.; de Carvalho, C.V. Relation Between Game Genres and Competences for In-Game Certification. In Proceedings of the Serious Games, Interaction, and Simulation: 5th International Conference, SGAMES 2015, Novedrate, Italy, 16–18 September 2015; Springer International Publishing: New York, NY, USA, 2015; pp. 28–35. [Google Scholar]
Baptista, R.; Coelho, A.; de Carvalho, C. Methodology for In-Game Certification in Serious Games. In Proceedings of the EDULEARN13, Barcelona, Spain, 1 July 2013; pp. 3143–3152. [Google Scholar]
Tritten, J.J. Joint mission-essential tasks, Joint Vision 2010, core competencies, and global engagement: Short versus long view. Air Space Power J. 2004, 11, 32. [Google Scholar]
Grimaila, M.R.; Mills, R.F.; Haas, M.W.; Kelly, D.J. Mission Assurance: Issues and Challenges. In Proceedings of the 2010 International Conference on Security & Management, Las Vegas, NV, USA, 12–15 July 2010; pp. 651–657. [Google Scholar]
Filipović, J. Update of the crawl, walk, run methodology framework. Facta Univ. Ser. Econ. Organ. 2019, 16, 229–238. [Google Scholar] [CrossRef]
Myers, D. Computer games genres. Play. Cult. 1990, 3, 286–301. [Google Scholar]
Munro, I.; Mavin, T.J. Crawl-walk-run. In Proceedings of the 10th International Symposium of the Australian Aviation Psychology Association, Sydney, Australia, 10 November 2012. [Google Scholar]
Jacobson, I.; Booch, G.; Rumbaugh, J. The Unified Software Development Process; Addison-Wesley Educationa: Boston, MA, USA, 2011. [Google Scholar]
Fogg, C.D. Implementing Your Strategic Plan: How to Turn”Intent” into Effective Action for Sustainable Change; AMACOM/American Management Association: New York, NY, USA, 1999. [Google Scholar]
Baptista, R.; Nóbrega, R.; Coelho, A.; de Carvalho, C.V. Location-based tourism in-game certification. In Proceedings of the INTED2015, Madrid, Spain, 2 March 2015; pp. 3602–3610. [Google Scholar]
Campos, C.; Leitão, J.M.; Coelho, A.F. Building Virtual Roads from Computer made Projects. In Proceedings of the HCI International 2015-Posters’ Extended Abstracts: International Conference, HCI International 2015, Los Angeles, LA, USA, 2 July 2015; pp. 163–169. [Google Scholar]
Campos, C.J.; Pinto, H.F.; Leitão, J.M.; Pereira, J.P.; Coelho, A.F.; Rodrigues, C.M. Building Virtual Driving Environments From Computer-Made Projects. In Interface Support for Creativity, Productivity, and Expression in Computer Graphics; IGI Global: Hershey, PA, USA, 2019; pp. 306–320. [Google Scholar]
Engström, I.; Gregersen, N.P.; Hernetkoski, K.; Keskinen, E.; Nyberg, A. Young Novice Drivers, Driver Education and Training: Literature Review. Available online: https://www.diva-portal.org/smash/get/diva2:675234/FULLTEXT01.pdf (accessed on 30 May 2024).
Keskinen, E. What is GDE all about and what it is not. In Proceedings of the GDE-Model as a Guide in Driver Training and Testing, Umeå, Sweden, 7–8 May 2007; pp. 3–13. [Google Scholar]
Peräaho, M.; Keskinen, E.; Hatakka, M. Driver competence in a hierarchical perspective: Implications for driver education. Rep. Swed. Road Adm. 2003, 1–52. Available online: https://media.freeola.com/other/23050/howtoteachthegde-matrix_1.pdf (accessed on 30 May 2024).

Figure 1. Workflow of Triadic Certification with used tools and defined goals.

Figure 2. Certification Triadic Model for training local tour guides.

Figure 3. Picture of the simulation room and the driving position.

Figure 4. Table of interconnection between routes and competences, where each competence has a colour and corresponding training and assessment context.

Figure 5. Summary grid highlighting the competencies identified for the training scenario.

Figure 6. Mapping of the training (paths) with the level of competence acquired through the mechanics identified by the game taxonomy.

Table 1. Taxonomy of game genres.

Game Genre	Goals	Sub-Genre
Action	To overcome mental or physical challenges against one or more opponents by engaging in a series of actions (timing—reaction speed, in which accuracy may be emphasised). Realism is not relevant.	Beat-’em-ups Beat ’em ups, Shooter games (1st and 3rd person), Platform games
Strategy	Deploy tactics/strategies to overcome complex challenges against one or more opponents by planning a superior series of actions (physical challenges are not emphasised).	4X (eXplore, eXpand, eXploit, eXtermine), Real-time strategy games, Real-time tactics, Turn-based strategy, War games
Role-Playing	Victory is achieved through superior planning or out-thinking the opponents (physical challenges and chance take a more minor role). Distinct from action games, RPGs seldom test a player’s physical skill (combat is more tactical than physical) and involve other non-action gameplay (resource management).	RPGs, MMORPGs
Sports	Similar to action games, except for the realism of movements and techniques, which are very important.	Exergames, Sports/management games
Management Simulation	To overcome economic challenges, a series of actions must be planned. Direct action upon an opponent is not emphasised. They are typically designed to be never-ending (no-win scenario). Goal example: to build a collection of objects.	Racing games/Vehicle, Virtual worlds/Pets, Life simulation/social games, Business
Adventure	To use an avatar for the exploration of an interactive story and to overcome challenges in isolation (puzzle adventure) by planning a superior series of actions (physical challenges are not emphasised).	Graphics Adventure, Puzzle adventure
Puzzle	To overcome mental challenges in isolation (not around a conflict with another opponent) by planning a superior series of actions. Games usually involve shapes, colours, or symbols that the player must directly or indirectly manipulate into a specific pattern.	Action/Arcade puzzle (timed), Reveal the picture game, Physics game.
Quiz	Gamepad controlled, mouse keyboard, Wii balance board.

Table 2. Categories for classifying serious games.

Label/Tag Category	Exemplary Labels
1. Platform	Personal Computer, Sony Play Station 3, Nintendo Wii, Mobile Phone
2. Subject Matter	World War II, sustainable development, physics, Shakespeare’s works
3. Learning goals	Language skills, historical facts, environmental awareness
4. Learning principles	Rote memorisation, exploration, observational learning, trial and error, conditioning
5. Target audience	High school children, nurses, law students, the general public, preschoolers, military recruits
6. Interaction mode(s)	Multiplayer, co-tutoring, single-player, massively multiplayer, tutoring agents
7. Application area	Academic education, private use, professional training
8. Controls/interfaces	Gamepad-controlled mouse keyboard, Wii balance board.
9. Common gaming labels	Puzzle, action, role-play, simulation, card game, quiz

Table 3. Educational Competency Wheel [87].

Educational Success Factors
Individual Excellence (IE)	Organisational Skills (OrSs)	Courage (C)	Results (Rs)	Strategic Skills (SSs)	Operating Skills (OpSs)
Building Effective Teams (IE1)	Comfort Around Authority (OrS1)	Assessing Talent (C1)	Action Oriented (R1)	Creativity (SS1)	Developing Others (OpS1)
Compassion (IE2) Organisational)	Organisational Agility (OrS2)	Conflict Management (C2)	Drive For Results (R2)	Dealing with Ambiguity (SS2))	Directing Others (OpS2)
Customer Focus (IE3)	Presentation Skills (OrS3)	Managerial Courage (C3)		Decision Quality and Problem Solving (SS3)	Managing and Measuring Work (OpS3)
Humour (IE4)	Written Communications (OrS4)			Functional / Technical Skills (SS4)	Managing Through Processes Systems (OpS4)
Integrity and Trust (IE5)				Intellectual Acumen (SS5)	Organising (OpS5)
Interpersonal Skills (IE6)				Learning on the Fly (SS6)	Planning (OpS6)
Listening (IE7)				Strategic Agility and Innovation Management (SS7)	Priority Setting (OpS7)
Managing Relationships (IE8)				Technical Learning (SS8)	Time Management (OpS8)
Managing Vision and Purpose (IE9)					Timely Decision-Making (OpS9)
Managing Vision & Purpose (IE9)
Motivating Others (IE10)
Negotiating (IE11)
Personal Learning and Development (IE12)
Valuing Diversity (IE13)

Table 4. Quantitative summary by genre and subgenre.

Genre	Subgenre	Subtotal	Total
Action	“Beat-’em-ups” “Beat ’em ups”	2	8
	1st/ 3rd person game	1
	Platform games	5
Strategy	4X (eXplore, eXpand, eXamine, eXtermine)	1	8
	Real-time strategy	1
	Real-time tactics	1
	Turn-based strategy	4
	War games	1
Role-Playing	Action RPGs	2	4
Role-Playing	MMORPGs	2	4
Sport	Exergames	2	3
Sport	Sports/management games	1	3
Management simulation	Racing/vehicle games	6	41
	Virtual worlds/fantasy/pets	24
	Business	8
	Social games and life simulations	3
Adventure	Graphics adventures	17	19
Adventure	Puzzle adventures	2	19
Puzzle	Arcade/Action puzzle (timed)	14	22
	Physics games	4
	Hidden images games	2
	Traditional games	2
Quiz		11	11

Table 5. Description of participants.

Category	Groups	Frequency	Percentage
Age	18–23	26	56.5%
	24–30	12	26.2%
	31–40	4	8.7%
	41–50	3	6.5%
	52–65	1	2.2%
Gender	Female	11	23.9%
Gender	Male	35	76.1%
Driver’s Licence	Yes	31	67.4%
Driver’s Licence	No	15	32.6%

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Baptista, R.; Coelho, A.; Vaz de Carvalho, C. Training and Certification of Competences through Serious Games. Computers 2024, 13, 201. https://doi.org/10.3390/computers13080201

AMA Style

Baptista R, Coelho A, Vaz de Carvalho C. Training and Certification of Competences through Serious Games. Computers. 2024; 13(8):201. https://doi.org/10.3390/computers13080201

Chicago/Turabian Style

Baptista, Ricardo, António Coelho, and Carlos Vaz de Carvalho. 2024. "Training and Certification of Competences through Serious Games" Computers 13, no. 8: 201. https://doi.org/10.3390/computers13080201

APA Style

Baptista, R., Coelho, A., & Vaz de Carvalho, C. (2024). Training and Certification of Competences through Serious Games. Computers, 13(8), 201. https://doi.org/10.3390/computers13080201

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu