“AR The Gods of Olympus”: Design and Pilot Evaluation of an Augmented Reality Educational Game for Greek Mythology
1
M.Sc. in Applied Informatics, Department of Applied Informatics, School of Information Sciences, University of Macedonia, GR-54636 Thessaloniki, Greece
2
Department of Applied Informatics, School of Information Sciences, University of Macedonia, GR-54636 Thessaloniki, Greece
*
Author to whom correspondence should be addressed.
Multimodal Technol. Interact. 2023, 7(1), 2; https://doi.org/10.3390/mti7010002
Submission received: 29 November 2022
/
Revised: 14 December 2022
/
Accepted: 18 December 2022
/
Published: 22 December 2022
(This article belongs to the Special Issue Virtual Reality and Augmented Reality)
Download keyboard_arrow_down
Figure 1
<p>The start screen of the game.</p> "> Figure 2
<p>The target image of Poseidon.</p> "> Figure 3
<p>Scanning the target image.</p> "> Figure 4
<p>AR preview of the goddess Artemis.</p> "> Figure 5
<p>Testing of the app from a student in the class.</p> "> Figure 6
<p>Correct matching of two cards.</p> "> Figure 7
<p>The icons disappear after the correct matching.</p> "> Figure 8
<p>Correct answer in the “Quiz game”.</p> "> Figure 9
<p>Wrong answer in the “Quiz game”.</p> "> Figure 10
<p>Third grade students’ performance (correct answers) in the pre-test.</p> "> Figure 11
<p>Third grade students’ performance (correct answers) in the post-test.</p> "> Figure 12
<p>Third grade students’ mean grade in the pre-/post-test.</p> "> Figure 13
<p>Fourth grade students’ performance (correct answers) in the pre-test.</p> "> Figure 14
<p>Fourth grade students’ performance (correct answers) in the post-test.</p> "> Figure 15
<p>Fourth grade students’ mean grade in the pre-/post-test.</p> ">
Versions Notes
<p>The start screen of the game.</p> "> Figure 2
<p>The target image of Poseidon.</p> "> Figure 3
<p>Scanning the target image.</p> "> Figure 4
<p>AR preview of the goddess Artemis.</p> "> Figure 5
<p>Testing of the app from a student in the class.</p> "> Figure 6
<p>Correct matching of two cards.</p> "> Figure 7
<p>The icons disappear after the correct matching.</p> "> Figure 8
<p>Correct answer in the “Quiz game”.</p> "> Figure 9
<p>Wrong answer in the “Quiz game”.</p> "> Figure 10
<p>Third grade students’ performance (correct answers) in the pre-test.</p> "> Figure 11
<p>Third grade students’ performance (correct answers) in the post-test.</p> "> Figure 12
<p>Third grade students’ mean grade in the pre-/post-test.</p> "> Figure 13
<p>Fourth grade students’ performance (correct answers) in the pre-test.</p> "> Figure 14
<p>Fourth grade students’ performance (correct answers) in the post-test.</p> "> Figure 15
<p>Fourth grade students’ mean grade in the pre-/post-test.</p> ">
Abstract
:Teaching and learning theoretical subjects, such as History, although important, is considered by many students to be non-appealing. Alternative teaching approaches include the use of educational games and augmented reality (AR) applications, or more recently, AR educational games. Such games are considered to increase students’ interest for the subject and lead to better learning outcomes. However, studies about the use of AR educational games in the classroom are sparse and further research is necessary. In this article, we present an AR-enhanced educational game for teaching History (Greek Mythology) to 3rd grade Primary school students in Greece. The game, called “AR The Gods of Olympus” consists of three mini games: an AR game with the gods/goddesses of Olympus using narration; a memory game with cards depicting the gods and their symbols; and a quiz game. In order to study the effectiveness of the game and students’ experience and perceptions on it, a study was carried out with primary school students that used the game in classroom. The study utilized a pre-/post-test design, a brief questionnaire based on the MEEGA+ model for evaluating educational games, and observation of students during game playing. Students’ performance was improved after playing the game but the difference was not statistically significant, while the game was positively perceived by students. Especially the AR mini game raised students’ interest and as the students themselves stated helped them “learn while playing”.
1. Introduction
Finding alternative ways for teaching and learning various subjects in schools, as well as motivating students’ interest, has been actively researched for several years. Several subjects are considered to be non-appealing to students, including History. Some widely known problems in the teaching and learning of History are summarized in the work by Koutromanos et al. [1] and include: the utilization of a teacher-centered teaching approach, an emphasis on memorizing historical events without trying to interpret them, and ignoring the differences in the thinking mechanisms of children and adults.
The use of educational games during the learning process gives students the possibility to better assimilate the content of a lesson, as well as to achieve a higher grade compared to students who are taught with the classical teaching method [2]. In addition, the time required to teach the lesson using an educational game is significantly reduced compared to the classical method [2]. By using various kinds of educational games and incorporating technologies such as augmented reality (AR), students’ interest increases, while at the same time, teaching the lesson becomes more fun [3]. A contemporary systematic literature review concluded that using AR and gamification in education results in students’ “positive behavioral, attitudinal, and psychological changes and increased engagement, motivation, active participation, knowledge acquisition, focus, curiosity, interest, enjoyment, academic performance, and learning outcomes” [4] (p. 1).
AR has been used for over 25 years in education with a positive impact [5]. It must be stressed, however, that AR applications should not focus solely on the characteristics of the underlying technology but should also adopt appropriate pedagogical approaches in order to be successful [5]. In a current review of the literature on the use of AR in education, it is highlighted that AR applications are effective in all levels of education [6]. However, at early-level education, the studies focus mainly on the application of AR for alphabet learning and science topics, while in secondary and higher education, the studies focus mainly on the application of AR in STEM (Science, Technology, Engineering, and Mathematics) [6]. When it comes to environmental and history learning, AR is usually utilized in location-based applications, including games [6].
In a contemporary review on AR applications for History and Heritage education, which aimed at investigating whether AR can be used for teaching complex historical events, such as the Holocaust, the following conclusions were drawn [7]: AR correlates to a higher understanding of historical topics in cases that specific cognitive and emotional requirements are met; active learning is promoted, giving students the chance to learn in their own pace; AR applications must be carefully designed in order not to cause cognitive overloading either due to the complexity of the system or the amount of information presented.
Our own search for AR-enhanced educational games for History resulted in twelve games and/or playful activities. The review of existing games and the corresponding studies showed that AR educational games can provide valuable tools for enhancing the teaching and learning of History. However, the research in the field so far has focused mainly on usability issues and far less on their effectiveness as learning tools in formal educational settings.
This article presents a mobile AR-enhanced educational game for teaching Greek Mythology to primary school students. The game was utilized for teaching the unit of the History course on the gods/goddesses of Olympus (Greek Mythology) in a Greek school. A pre-/post-test design was utilized for investigating the impact of the game in students’ performance, while data regarding students’ experience and perceptions on using the game in the classroom were collected through observation during game playing and an anonymous questionnaire.
The rest of the article is organized as follows: In Section 2, the related work is presented, including the main features of AR educational games on History and the main results of the research carried out so far. In Section 3, the game “AR The Gods of Olympus” is presented, while information about the methodology of the study carried out is summarized in Section 4. The results of the study are presented in Section 5, while some limitations are presented in Section 6. Finally, the results are discussed in Section 7.
2. Related Work
In this section, a brief review of AR educational games on History are presented. Google Scholar was used and the following search query was applied: (augmented reality OR AR) AND (educational OR serious OR learning) AND (game) AND (Mythology OR History).
Our initial search focused on AR educational games for teaching and learning History in primary school students. However, we could not find many games covering our criteria and consequently we expanded the search and included games targeted to older students, as well as AR playful activities and learning environments. Twelve AR games or learning environments for History were selected after screening the title and the abstract of the articles returned by Google Scholar in terms of their relevance to the aims of our study. We must note that we did not aim at a systematic literature review, meaning that some relevant studies might have been missed. In Table 1, we summarize the main features and research results for the selected studies/games.
Based on the review of existing games, it turned out that there is only one game, or to be more specific, one AR playful activity, targeted towards the teaching and learning of Greek Mythology that utilizes AR for an enhanced students’ experience. Specifically, Rammos and Bratitsis [3]—in their game about the 12 gods of Olympus, half-gods and mythical monsters—use AR technology in order for students to interact with a character (god, half-god, mythical monster) and create a narrative video with all the knowledge they have about it. The purpose of the game is to assess students’ acquired knowledge. When using the game, the student uses two applications, one for viewing the character and one for recording the screen of the device and his/her narration. Twenty-four 4th grade primary school students participated in the evaluation of the game (target group) and a comparison was made of their performance during the oral examination of the course in relation to their performance in the narrative video they created. The results showed that an increase in performance was recorded for 15 out of the 24 students, 8 remained stagnant and 1 student had a worse performance. In addition, the students commented positively on the use of AR, as it inspired them to create the narratives [3].
The main conclusions drawn from the review of the AR games are summarized in Table 1 in terms of their overall design and the aims are the following:
- Most of the games aim for a guided tour of archaeological sites, such as [10,11,13], or places that have a historical, architectural and/or cultural value [8]. Such games aim to guide students through their tour in the corresponding archaeological site or place of local history, raising their curiosity and interest for exploring the place of interest, providing them with the right content at the right time, and finally providing an enhanced user experience. Common challenges faced in this type of games include [1]: internet connectivity; overcrowded archaeological sites; coordination of student groups under the aforementioned circumstances.
- In some cases, an AR application might not have been designed as a game, but it can be used in a playful manner. A typical example is KnossosAR [10], a mobile application designed to support a guided tour of the archaeological site of Knossos in Crete, Greece. KnossosAR was utilized in the context of a ‘treasure hunting’-like game, where groups of students had to locate six points of interest (POIs) in 30 min. Another example is the AR playful activity “3D Heroes Introduce Themselves” [3].
The main conclusions drawn in terms of the research carried out in the field so far can be summarized as follows:
- Areas of research. Most of the studies investigated the usability and ease of use of AR games [1,10,13,14,15,16]. The impact of the game on students’ performance was investigated in just three studies [2,3,12]. Finally, in one game, its pedagogical design and educational content [1] was evaluated, and in another game, its scenario [8].
- Methods/Instruments. In most of the studies, a questionnaire was utilized [1,2,10,13,14,15,16] for evaluating mainly the usability of the game, and in some cases, the participants’ enjoyment, satisfaction and feelings during game playing. In three studies, a widely accepted questionnaire was utilized, namely: the Technology Acceptance Model [16]; the USE questionnaire [14]; the SUS scale [15]. Another method utilized in several studies was the interview or discussion with the participants [1,3,10,12,13,15] and just one study reported utilizing observation for investigating the factors affecting the use of the game under real life conditions, as well as students’ feelings [1]. In four cases, the authors reported carrying out a field study [1,10,12,13] in the archaeological or culture heritage site of the game. Finally, in just three cases where the game was directly connected to a school module on History, students’ performance was investigated with a pre-/post-test [12], a questionnaire on students’ acquired knowledge [2] or a comparison of students’ performance in an oral examination and the narrative video they created in the game [3].
- Participants. All the studies that carried out an evaluation on either the usability and ease of use of AR games or their effectiveness as learning tools, reported a relatively small number of students [1,2,3,8,10,12,13,14,15]. Specifically, the number of students that participated ranged from 13 [13] to 53 [12], while the average number of participants was 25. In two of the reviewed studies, the game was also evaluated by 5 teachers in terms of its scenario [8] and 28 experts in ICT education [1] in terms of its usability, pedagogical design and educational content.
- Conclusions. Positive results were recorded for all the AR games and applications that were evaluated. The main conclusions drawn are summarized as follows: the use of the game resulted in an increased curiosity and interest for investigating the archeological site of the game [10,13]; increased satisfaction [14]; enjoyment [10]; enhanced conceptual understanding and historical empathy [12]; better comprehension of the course [2]; less time for the course [2]; and an improved performance [2,3,12].
Based on the review of related work, we can conclude that AR educational games can provide valuable tools for enhancing the teaching and learning of History. However, the research in the field so far has focused mainly on usability issues and far less on their effectiveness as learning tools in formal educational settings. The game and the study presented in this article aim to contribute to the research in the field by investigating the impact of an AR-enhanced educational game for teaching the subject of History to primary school students.
3. The Educational Game “AR The Gods of Olympus”
The educational game “AR The Gods of Olympus” is targeted at children eight years old and above, namely 3rd grade primary school students based on the Greek educational system. The game aims at supporting students in studying History (Greek Mythology) and specifically in learning the characteristics and the symbols of the gods and goddesses of Olympus.
The educational game includes three mini games (Figure 1), while the graphical user interface and the educational content are currently in Greek. The first mini game is based on narration and AR, the second is a memory game with cards and the third one is a quiz game. Narration has not been widely used in the games for History reviewed, but it was considered ideal for young students. In the educational game MYTH TROUBLES, narration and quizzes are used for supporting students in studying Greek Mythology [17]. Teachers that evaluated the game believe that narration provides incentive for students to study History. Moreover, in the AR learning activity 3D Heroes Introduce Themselves, students were inspired by narration and improved their performance [3]. Using AR for depicting and interacting with the gods/goddesses and simultaneously listening to the narrations was expected to immerse students. On the other hand, quiz games are typical for assessing students’ knowledge in History games [1,13,14,15]. However, since quiz games might be considered by students as another form of a typical test, the decision to include a memory card game as well was taken in order to provide another means of self-assessment that might be more appealing to students.
3.1. Mini Game “AR The Gods of Olympus”
The 1st mini game, called “AR The Gods of Olympus”, utilizes AR and storytelling. Specifically, the student scans, with his/her mobile device, the target image (QR code) (Figure 2) in order to see the corresponding god/goddess using AR (Figure 3, Figure 4 and Figure 5) and to start listening to the narration. The students can listen to the narration as many times as they want, as long as they scan the corresponding target image.
Relevant audio themes are used in the background in order to keep students focused while listening to the narration. For example, for the god Hephaestus, the student hears sounds from a smelter, while for the goddess Artemis, the student hears sounds from the forest. Students can rotate their mobile device in order to see the god/goddess and his/her symbol from every perspective they want.
3.2. Mini Game “Memory Game”
The 2nd mini game falls into the category of memory games. The student has to select the cards to see their hidden content and match the gods/goddesses with the symbols that characterize them. If the matching is correct (Figure 6) then the cards disappear (Figure 7), otherwise the cards return to their initial state.
In this mini game, in addition to the knowledge of the symbols of the gods/goddesses, students also practice their memory, as they have to recall on which card they encountered a god/goddess or symbol. When all the cards have been matched, the number of attempts made is shown along with a motivational message. The motivational messages are different according to the number of attempts made by the student and are presented in Table 2.
3.3. Mini Game “Quiz Game”
The 3rd mini game belongs to the category of quiz games. The student is posed with questions concerning the characteristics of the gods/goddesses. For each correct answer, the student earns one point, while there is no negative score for wrong answers.
The game contains 14 questions, as many as the gods/goddesses presented in the History book. The student has three answers available and they must choose the correct one. If the selection is correct, then the choice is made green and the student wins one point (Figure 8), otherwise, the choice is made red (Figure 9). As long as the student answered all the questions, the final score is presented along with a motivational message. The messages presented based on the final score are presented in Table 3.
3.4. Instructions
In the “Instructions” section, students are provided with brief information about the way of playing each one of the three mini games. Students can play the mini games in any order they want and can repeat them as many times as they want in order to achieve a better score as highlighted in the motivational games presented to them. In the “Memory Game” and “Quiz Game”, the cards and questions are displayed randomly each time the game is played.
3.5. Implementation of the Game “AR The Gods of Olympus”
The game was implemented using the Unity game engine, while Vuforia was used for creating the AR scenes. Gimp was used for creating the icons used in the game, while Audacity was used for creating and editing the narrations as well as the sound themes used in the “AR The Gods of Olympus” mini game. Finally, the graphics which are used in the game are the “LOWPOLY—Olympian Gods” by the Codewart Game Assets company from Unity Asset Store and from the web site Sketchfab.com. The educational game is available for Android mobile devices from the following URL (accessed on 29 November 2022): https://sites.google.com/a/uom.edu.gr/stelios-xinogalos/serious-games/primary-school#h.4iv8lifkihxz.
4. Materials and Methods
4.1. Research Questions
The aim of the study presented was two-fold: investigate whether students improve their knowledge on the gods and goddesses of Olympus through playing the game; evaluate students’ experience with the game. Consequently, the following research questions (RQs) were investigated:
RQ1: Do students improve their knowledge on the gods and goddesses of Olympus through playing an AR-enhanced educational game?
RQ2: How do students evaluate their user experience in the AR-enhanced educational game?
4.2. Participants
Thirty-one students from a primary school in Greece participated in the study. The school was a typical school in a small town in Greece. Thirteen of them were 3rd grade and eighteen of them were 4th grade students. The 3rd grade students comprised of 7 boys and 6 girls, while the 4th grade students comprised of 6 boys and 12 girls.
The gods and goddesses of Olympus are taught in the context of the History section in the 3rd grade. So, students from the 3rd and 4th grade were purposefully selected to participate in the study: 3rd grade students had no prior knowledge on the gods and goddesses of Olympus; 4th grade students had been taught the specific History section during the previous school year. This would help us draw some preliminary conclusions on whether the game should better be used for teaching the underlying educational material in the first place and/or strengthening students’ knowledge on the subject after having completed teaching the module.
4.3. Research Methodology
The study was based on a pre-/post-test design for investigating RQ1. Moreover, a brief questionnaire and observation were utilized for investigating RQ2. The research methodology was highly dictated by the context of the study—teaching History in a typical primary school classroom where the teacher interacts with the students, and the researcher steps aside and discreetly observes. In such classes, teachers often utilize worksheets and/or tests for assessing students’ knowledge, while school textbooks often include (or the teachers themselves provide) brief questionnaires at the end of each unit where students reflect on their perceptions about the specific unit and self-assess their achievements. The data collection instruments selected would provide us the chance to collect both objective quantitative and subjective but still less rigid qualitative data “that make a perfect match”, as Abusabha and Woelfel note [18]. Both the game and the data collection instruments (pre-/post-test, questionnaire) were evaluated and approved by the teachers involved in terms of their relevance to the course and its intended learning outcomes.
Firstly, a pre-test (in paper) was given to students when the instructor had finished teaching the module on the gods and goddesses of Olympus. The test included 28 multiple-choice questions on the attributes of the gods/goddesses and their symbols and was used to assess students’ level of knowledge. Sample questions are presented in Appendix A. The test was taken in class and was anonymous. After one week, students were asked to play the educational game individually. The game was played in class for about 20 min under the supervision of their instructor and the researcher (first author). Qualitative data were collected by the researcher that was present in the classroom by keeping notes for students’ interactions with the game; students’ thinking aloud comments; conversations between the students and their teacher. After finishing playing the game, a post-test was used. The pre- and post-tests were the same and were filled in anonymously. The pre-test was not returned to students prior to finishing the whole pre-/post-test process.
Finally, students were asked to anonymously fill in a questionnaire regarding their player experience with the game. The questionnaire consisted of nine questions adopted from the MEEGA+ evaluation model [19], which is an established model for the evaluation of player experience and short-term learning in educational/serious games. The questions were appropriately adapted so that they could be easily understood by 3rd and 4th grade students.
After collecting the aforementioned data, the mean value, frequency, and percentages were calculated.
5. Results
In this section, the results from the pre-/post-test, the questionnaire and observation are presented; both tests and the questionnaire were anonymously filled in.
5.1. Student Performance
The results regarding the performance for the 3rd grade students in the pre- and post-test are presented in Figure 10 and Figure 11, respectively. After the intervention, the mean grade of students increased by 1.93 (approximately 7%), as shown in Figure 12. Since we had a small sample size (n = 13), determining the distribution of students’ performance in the pre-/post-test was important for choosing an appropriate statistical method for checking whether the difference in students’ performance was statistically significant. So, a Shapiro–Wilk test was performed and did not show evidence of non-normality (pre-test: W = 0.943, p-value > 0.05; post-test: W = 0.879, p-value > 0.05). Following, a two-sample T-Test Equal Variance Analysis showed that there was not a statistically significant difference in students’ performance between the pre- and post-test (p = 0.398 > 0.05).
The results in the pre-test for the 4th grade students are presented in Figure 13, while the results for the post-test are presented in Figure 14. After the intervention, we observe an increase in the students’ mean grade by 2.56 (approximately 9%), as shown in Figure 15. In order to determine whether this difference in students’ performance is statistically significant, the Shapiro–Wilk test of normality was conducted to determine whether pre-test and post-test values were normally distributed. The results indicate that the pre-test fails to reject the null hypothesis (p = 0.457) and conclude that the data were normally distributed. On the contrary, results on the post-test data indicate a rejection of the null hypothesis (p = 0.008) and conclude that the data were not normally distributed. Consequently, a non-parametric test was required to statistically test the scores in the pre- and post-test. A Mann–Whitney U test revealed that there was no significant statistical difference in scores between the pre-test (Md = 16.50) and post-test (Md = 20.50) values, U = 196.50, n1 = n2 = 18, p = 0.279 > 0.05.
5.2. Student Game Experience
All the students who participated in the research evaluated the educational game positively, as they answered that they liked both the game and the graphics used in it. A total of 94% of the participants answered that they would like to play the game again and 97% said that they would like to use similar games in other school courses as well. Moreover, the majority of students (77%) stated that the game was easy to use. The results of the evaluation are presented in detail in Table 4.
In addition, the students were asked to evaluate separately each one of the three mini games. The results are presented in Table 5. Ninety-four percent (94%) of the students stated that they extremely liked the mini game “AR The Gods of Olympus” and the rest 6% of them stated that they liked it very much. For the mini game “Memory game”, 74% of the students stated that liked it extremely, 23% very much and 3% moderately. Finally, 71% extremely liked the mini game “Quiz game”, 19% very much, and 10% moderately.
Lastly, students were asked to identify what they liked most about the “AR The Gods of Olympus” mini game. As shown in Table 6, 81% of them stated that they liked both the AR character representation and the narration and 19% liked more the AR character representation.
5.3. Observation during Game Playing
During game playing, the teacher and the first author that were present in the classroom observed students and kept field notes about their reactions. Moreover, they kept notes regarding their spontaneous comments or their responses to specific instructions or questions posed by their teacher.
The most prominent observation refers to the fact that students were impressed with the projection of the characters through the use of AR. A typical reaction was to look from the side of the mobile phone directly at the target image to see if the character they were seeing through the screen was actually there. Another common reaction was to try to touch the character with their hand. In addition, many students were looking to see if there was someone behind them who was projecting the character they saw in front of them. After viewing three or four characters with the use of AR, several students started to move around their mobile phone so that they could see the characters from other views as well. In cases that students did not explore the AR characters from different viewpoints, they were prompted to do so by the teacher or researcher.
Regarding the other two mini games, namely the Memory Game and the Quiz Game, it was observed that students who either made many attempts to complete the Memory Game or did not answer many questions correctly in the Quiz Game, expressed a desire to play the game again in order to improve their score.
Students’ verbal comments confirmed their responses in the questionnaire. Specifically, students stated that they liked the game and would like to play it again. However, the most encouraging comment was that “the game was very nice, because you play and learn”.
A final note was that all the students were familiar with the use of mobile phones and did not have any problem using the game. Although students did not have any prior experience with AR technology, they were easily accustomed to correctly using the target images for viewing the AR characters and listening to the narrations, and generally interacting with AR technology [10]. Moving from one mini game to the other was also easy for students. This might be due to the simple and straightforward user interface of the game that was designed to be easy to use even by students without any experience with mobile phones with little guidance by their teacher. However, it must be noted that nowadays, mobile phones are part of students’ lives from the age of seven [20].
6. Limitations
The study presented aimed at: reviewing the existing literature on studies regarding AR educational games for History, designing an AR educational game for Greek Mythology targeted at primary school students, and evaluating students’ performance and experiences with the game that was utilized in a typical primary school classroom. In this section, some limitations of the study are presented.
Firstly, since we did not aim for a systematic literature review, some relevant studies might have been skipped. However, effort was made in carefully screening the articles returned using Google Scholar, while the references of the selected articles were also checked.
Secondly, the study that was designed for investigating the research questions applied both quantitative and qualitative methods. The game was utilized in two classes of the 3rd and 4th grade of a typical primary school in a small town in Greece, consisting of 13 and 18 students, respectively. The game was utilized in the classroom with the presence of their teacher, as well as the first author that was discreetly observing students. This study provided valuable data for investigating the research questions of the pilot evaluation of the game. However, it is necessary to further investigate the learning outcomes and player experience in AR games for History with more participants from different primary schools and long-term studies. A control and experimental group design could be utilized [12] in order to investigate whether the use of an AR game for History provides a statistically significant difference in students’ performance in comparison with typical lecturing, while some of the students could be interviewed for collecting more qualitative data that will help us to appropriately interpret the results.
7. Conclusions
The research carried out investigates the contribution of an AR-enhanced educational game to the teaching and learning of a theoretical subject that is often characterized by students as “non-appealing” [1] or even “boring” [13], namely History. The game “AR The Gods of Olympus” was designed with the aim of supporting 3rd grade primary school students in studying Greek Mythology. The game was used in the classroom both with 3rd grade students that had been recently taught the specific module, as well as 4th grade students that had been taught the specific module the previous school year.
In order to study the effectiveness of the game as a learning tool, a pre-/post-test design was adopted in our study. Both the 3rd grade and 4th grade students improved their performance after using the game in the classroom by approximately 7% and 9%, respectively (RQ1). The results of our study are in accordance with those of [2,3,12]. However, it is clear that more studies with more participants have to be carried out in the classroom in order to investigate whether the positive results recorded so far are confirmed.
Regarding player experience and also students’ perceptions about the use of an AR-enhanced game for learning History (RQ2), we used a subset of the widely known questionnaire based on the MEEGA+ model [19], as well as field notes during game playing. It is encouraging that all the students stated that they liked both the game and its graphics, which certainly results in higher student satisfaction [14]. The most popular out of the three mini games was the one utilizing AR and narration (100%), followed by the memory game (97%) and lastly the quiz game (90%), which is the most common game genre for History. The vast majority of students (94%) stated that they would like to play the game again and also use similar games in other school courses (97%), while just a few students were not sure about replaying this game or generally games in the context of other school courses (6% and 3%, respectively). Students’ reluctance for using games, either AR-enhanced or not, has been recorded in the literature for other subjects as well, such as programming [21], and the time has come to utilize games more actively in formal education. Finally, both students’ responses in the questionnaire and observation showed that the game is easy to use, thus confirming the results recorded for other AR-enhanced games for History [3,10,13,16]. These results indicate that students are not just reluctant in using games in the classroom but are also ready to use them appropriately.
Author Contributions
Conceptualization, E.V. and S.X.; methodology, E.V. and S.X.; software, E.V.; validation, E.V. and S.X.; investigation, E.V. and S.X.; data curation, E.V. and S.X.; writing—original draft preparation, E.V. and S.X.; writing—review and editing, E.V. and S.X.; visualization, E.V. and S.X.; supervision, S.X. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Institutional Review Board Statement
Ethical review and approval were waived for this study, due to non-person-related data acquisition. Since this study was carried out in the context of a History course as a learning activity guided by the teachers of two classes in a primary school, permission by the teachers and school administration was granted.
Informed Consent Statement
Not applicable.
Data Availability Statement
All of the data is contained within the article.
Conflicts of Interest
The authors declare no conflict of interest.
Appendix A
The pre-/post-test consisted of 28 closed-type multiple-choice questions on the attributes of the gods/goddesses and their symbols. Sample questions are presented in Table A1.
Table A1.
Sample pre-/post-test questions.
| Question | Answer |
|---|---|
| Who is the leader of the gods? |
|
| Who is the goddess of beauty? |
|
| Which is the symbol of the god Hephaestus? |
|
| Which is the symbol of the goddess Athena? |
|
References
- Koutromanos, G.; Pittara, T.; Tripoulas, C. ‘Clavis Aurea’: An Augmented Reality Game for the Teaching of Local History. Eur. J. Eng. Technol. Res. 2020. [Google Scholar] [CrossRef]
- Schiavi, B.; Gechter, F.; Gechter, C.; Rizzo, A. Teach me a story: An augmented reality application for teaching history in middle school. In Proceedings of the 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Tuebingen/Reutlingen, Germany, 18–22 March 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 679–680. [Google Scholar]
- Rammos, D.; Bratitsis, T. Alternative teaching of history subject in primary school: The case of the 3D HIT playful activity. In International Conference on Games and Learning Alliance; Springer: Cham, Switzerland, 2019; pp. 457–467. [Google Scholar]
- Lampropoulos, G.; Keramopoulos, E.; Diamantaras, K.; Evangelidis, G. Augmented Reality and Gamification in Education: A Systematic Literature Review of Research, Applications, and Empirical Studies. Appl. Sci. 2022, 12, 6809. [Google Scholar] [CrossRef]
- Garzón, J. An Overview of Twenty-Five Years of Augmented Reality in Education. Multimodal Technol. Interact. 2021, 5, 37. [Google Scholar] [CrossRef]
- Iqbal, M.Z.; Mangina, E.; Campbell, A.G. Current Challenges and Future Research Directions in Augmented Reality for Education. Multimodal Technol. Interact. 2022, 6, 75. [Google Scholar] [CrossRef]
- Challenor, J.; Ma, M. A Review of Augmented Reality Applications for History Education and Heritage Visualisation. Multimodal Technol. Interact. 2019, 3, 39. [Google Scholar] [CrossRef] [Green Version]
- Koutromanos, G.; Styliaras, G. “The buildings speak about our city”: A location based augmented reality game. In Proceedings of the 2015 6th International Conference on Information, Intelligence, Systems and Applications (IISA), Corfu, Greece, 6–8 July 2015; IEEE: Piscataway, NJ, USA, 2015; pp. 1–6. [Google Scholar]
- Rodrigo, M.M.; Caluya, N.R.; Diy, W.D.; Vidal, E.C.E. Igpaw: Intramuros-design of an augmented reality game for philippine history. In Proceedings of the 23rd International Conference on Computers in Education, Hangzhou, China, 30 November–4 December 2015. [Google Scholar]
- Galatis, P.; Gavalas, D.; Kasapakis, V.; Pantziou, G.E.; Zaroliagis, C.D. Mobile Augmented Reality Guides in Cultural Heritage. In Proceedings of the 8th EAI International Conference on Mobile Computing, Applications and Services, MobiCASE, Cambridge, UK, 30 November–1 December 2016; pp. 11–19. [Google Scholar]
- Sintoris, C.; Yiannoutsou, N.; Avouris, N. The fortress of Monemvasia as playground for a location based game. Proc. Play. Hist. 2016, 13. [Google Scholar]
- Efstathiou, I.; Kyza, E.A.; Georgiou, Y. An inquiry-based augmented reality mobile learning approach to fostering primary school students’ historical reasoning in non-formal settings. Interact. Learn. Environ. 2017, 26, 22–41. [Google Scholar] [CrossRef]
- Ekonomou, T.; Vosinakis, S. Mobile augmented reality games as an engaging tool for cultural heritage dissemination: A case study. Sci. Cult 2018, 4, 97–107. [Google Scholar]
- Azhar, N.H.M.; Diah, N.M.; Ahmad, S.; Ismail, M. Development of augmented reality to learn history. Bull. Electr. Eng. Inform. 2019, 8, 1425–1432. [Google Scholar]
- Low, J.Z.; Kamaruddin, A.; Hamzah, M.D.; Hamzah, M.D. Mobile augmented reality application for high school history subject (SejarAR). In Proceedings of the Conference on Human Factors in Computing Systems, Glasgow, UK, 4–9 May 2019; pp. 1–7. [Google Scholar]
- Trista, S.; Rusli, A. HistoriAR: Experience Indonesian history through interactive game and augmented reality. Bull. Electr. Eng. Inform. 2020, 9, 1518–1524. [Google Scholar] [CrossRef]
- Evangelopoulou, O.; Xinogalos, S. MYTH TROUBLES: An Open-Source Educational Game in Scratch for Greek Mythology. Simul. Gaming 2017, 49, 71–91. [Google Scholar] [CrossRef]
- Abusabha, R.; Woelfel, M.L. Qualitative vs quantitative methods: Two opposites that make a perfect match. J. Am. Diet. Assoc. 2003, 103, 566–569. [Google Scholar] [CrossRef] [PubMed]
- Petri, G.; von Wangenheim, C.G.; Borgatto, A.F. MEEGA+: An evolution of a model for the evaluation of educational games. INCoD/GQS 2016, 3, 1–40. [Google Scholar]
- Kopecký, K.; Fernández-Martín, F.-D.; Szotkowski, R.; Gómez-García, G.; Mikulcová, K. Behaviour of Children and Adolescents and the Use of Mobile Phones in Primary Schools in the Czech Republic. Int. J. Environ. Res. Public Health 2021, 18, 8352. [Google Scholar] [CrossRef] [PubMed]
- Giannakoulas, A.; Xinogalos, S. A pilot study on the effectiveness and acceptance of an educational game for teaching programming concepts to primary school students. Educ. Inf. Technol. 2018, 23, 2029–2052. [Google Scholar] [CrossRef]
Figure 1.
The start screen of the game.
Figure 2.
The target image of Poseidon.
Figure 3.
Scanning the target image.
Figure 4.
AR preview of the goddess Artemis.
Figure 5.
Testing of the app from a student in the class.
Figure 6.
Correct matching of two cards.
Figure 7.
The icons disappear after the correct matching.
Figure 8.
Correct answer in the “Quiz game”.
Figure 9.
Wrong answer in the “Quiz game”.
Figure 10.
Third grade students’ performance (correct answers) in the pre-test.
Figure 11.
Third grade students’ performance (correct answers) in the post-test.
Figure 12.
Third grade students’ mean grade in the pre-/post-test.
Figure 13.
Fourth grade students’ performance (correct answers) in the pre-test.
Figure 14.
Fourth grade students’ performance (correct answers) in the post-test.
Figure 15.
Fourth grade students’ mean grade in the pre-/post-test.
Table 1.
Review of studies on AR applications for History.
| Study | Type of AR Application | Name | Theme | Participants | Methodology | Conclusions |
|---|---|---|---|---|---|---|
| Koutromanos and Styliaras (2015) [8] | Location-based AR mobile game | The Building Speak About Our City | The tobacco warehouses buildings in Agrinio, Greece |
|
|
|
| Rodrigo et al., (2015) [9] | Location-based AR mobile game | Igpaw: Intramuros | The history of the Intramuros area (city of Manila) in the Philippines | N/A | N/A | N/A |
| Galatis et al., (2016) [10] | Mobile AR guide used as a treasure-hunting game | KnossosAR | The archaeological site of Knossos in Crete, Greece | 16 students 17–19 years old |
|
|
| Sintoris et al., (2016) [11] | Location-based AR mobile game | If … in Monemvasia | The historical fortress town Monemvasia in Peloponnese, Greece | N/A | N/A | N/A |
| Efstathiou et al., (2018) [12] | AR mobile learning environment | Young Archaeologists | A field trip to the Neolithic settlement of Choirokoitia in Cyprus (UNESCO World Cultural Heritage site) | 53 3rd grade students |
|
|
| Ekonomou and Vosinakis (2018) [13] | Location-aware AR mobile game | Oracle of Delphi app | Guided tour of the archaeological site of Delphi | 13 High school students |
|
|
| Schiavi et al., (2018) [2] | AR game | Teach me a story | The history of the city of Ur in Mesopotamia | 26 6th grade students |
|
|
| Azhar (2019) [14] | Mobile AR book | N/A | The fall of Melaka Empire history (Malaysia) | 20 participants aged 13–25 |
|
|
| Low et al., (2019) [15] | AR game | SejarAR | Malaysian history | 25 High school students |
|
|
| Rammos and Bratitsis (2019) [3] | AR playful activity | 3D Heroes Introduce Themselves | The 12 gods of Olympus, half-gods and mythical monsters | 24 4th grade students |
|
|
| Trista and Rusli (2020) [16] | Interactive AR Game | HistoryAR | Indonesian history | N/A |
|
|
| Koutromanos et al., (2020) [1] | AR game | Clavis Aurea | The local history of the archaeological site of the Castle in Naxos |
|
|
|
Table 2.
Motivational messages in the “Memory game”.
| Number of Attempts | Motivational Message |
|---|---|
| 7–13 | Excellent!!! You have a very good memory!!! |
| 14–20 | Nice try! If you want, you can play the “AR The Gods of Olympus” mini game again. |
| More than 21 | Keep trying! If you want, you can play the “AR The Gods of Olympus” mini game again. |
Table 3.
Motivational messages in the “Quiz game”.
| Final Score | Motivational Message |
|---|---|
| 14 | Excellent!!! You answered all the questions correctly!!! |
| 13–11 | Very good!!!! If you want, you can play the “AR The Gods of Olympus” mini game again. |
| 10–7 | Nice try!!!! If you want, you can play the “AR The Gods of Olympus” mini game again. |
| Less than 6 | Keep trying! Play the “AR The Gods of Olympus” mini game again. |
Table 4.
Evaluation of the educational game.
| Question | Yes | No | I Do Not Know |
|---|---|---|---|
| Did you like the game? | 100% | 0% | 0% |
| Did you like the graphics of the game? | 100% | 0% | 0% |
| Would you like to play the game again? | 94% | 0% | 6% |
| Is the game easy to use? | 77% | 13% | 10% |
| Would you like to use similar games in other school courses? | 97% | 0% | 3% |
Table 5.
Evaluation of the mini games.
| Question | Not at All | Slightly | Moderately | Very much | Extremely |
|---|---|---|---|---|---|
| Did you like the mini game “AR The Gods of Olympus”? | 0% | 0% | 0% | 6% | 94% |
| Did you like the mini game “Memory Game”? | 0% | 0% | 3% | 23% | 74% |
| Did you like the mini game “Quiz Game”? | 0% | 0% | 10% | 19% | 71% |
Table 6.
Evaluation of the content of the “AR The Gods of Olympus” mini game.
| Question | The Representation of the Gods (I Moved the Mobile Phone to See the Character from Other Points as Well) | The Narration and the Background Sounds | Both (the Representation and the Narration) |
|---|---|---|---|
| When you played the “AR The Gods of Olympus” what did you like the most? | 19% | 0% | 81% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Ventoulis, E.; Xinogalos, S. “AR The Gods of Olympus”: Design and Pilot Evaluation of an Augmented Reality Educational Game for Greek Mythology. Multimodal Technol. Interact. 2023, 7, 2. https://doi.org/10.3390/mti7010002
AMA Style
Ventoulis E, Xinogalos S. “AR The Gods of Olympus”: Design and Pilot Evaluation of an Augmented Reality Educational Game for Greek Mythology. Multimodal Technologies and Interaction. 2023; 7(1):2. https://doi.org/10.3390/mti7010002
Chicago/Turabian StyleVentoulis, Evangelos, and Stelios Xinogalos. 2023. "“AR The Gods of Olympus”: Design and Pilot Evaluation of an Augmented Reality Educational Game for Greek Mythology" Multimodal Technologies and Interaction 7, no. 1: 2. https://doi.org/10.3390/mti7010002
APA StyleVentoulis, E., & Xinogalos, S. (2023). “AR The Gods of Olympus”: Design and Pilot Evaluation of an Augmented Reality Educational Game for Greek Mythology. Multimodal Technologies and Interaction, 7(1), 2. https://doi.org/10.3390/mti7010002