Keywords

1 Introduction

During the process of educational games design, besides the main challenge of balancing fun and learning [1, 2] and proposing high quality content in one significant approach [3], there is as well a challenge to adapt story, environment, mechanics (interactions) and learning content to the largest range of users as possible part of the targeted audience, embracing variety of users preferences [4, 5]. Once established the main goals of the game according to the audience; the game gender definition, the story choice, scenarios and mechanics should be based on real user information. The methodology of Personas [6–8] constructed from potential user’s interviews is a possibility to guide the development. The comparison between Play-personas originated from users evaluation of final product with Personas previously created allows verifying the robustness of those Personas along with the game’s degree of adaptation to the variety of user-players. Since players not only follow rules, but constantly push against them, testing and demanding more from the system and that there is never just one game (as players experience it differently [9]); play-Personas is one important source for game design improvement and adaptation. Considering the design of educational digital games to learn languages, the 3D environment is one lever to promote the environmental – interactive storytelling [10, 11] and embodied learning [12], favoring learning in context. The player has the possibility of attaching words and expressions to actions, goals and dialogues [1], to see the consequences of his actions and to relate them to environment [13, 14]. In addition, player has the possibility to have the help of spatial memory related to the story when remembering certain content. Context in communication in real life can be defined as what is contained in this scenario: people, dialogues, gestures and the implicit cultural knowledge shared by agents [1]. Making a parallel with a game, it is possible to understand that each scenario of a game: part of the story in one specific environment with its mechanics provided by the system, with characters actions and dialogs, managed by the rules of the game, is a different context. In a foreign language learning game, this context should be as close as possible to the real use of the language to be learned [15], while understanding real not as necessarily a realistic game, but as a source of information able to be used in the real life. To evaluate the adequacy of educational content and missions, potential users’ game evaluations are one important source of information [16] as well as the definition of patterns and identification of players’ behaviors and preferences.

1.1 Play-Personas

Play-Personas are the models of players created from the evaluation of real users playing experiences. For their construction, players are observed and patterns are identified in terms of players’ navigation and their interaction with game mechanics. Temporal and spatial criteria are some of the metrics used [4]. In the case of educational games, the interaction related to learning content of each different context presented is also analyzed.

2 Methodology

According to principles of Research Design in Education [17] and User Centered Design [18, 19], during all game design process of the 3D Adventure Game “Back to the knowledge the foreign student and the book of language and culture”, university students contributed voluntarily to the project, totalizing 106 university students from 27 nationalities who were living in Brazil, Portugal and Mexico. After defining Personas to guide the design process [20], narratives creation, establishment of pre requirements [21], game design, programming and usability and user’s experience evaluations; real users playing performances/experiences were analyzed. Ten (10) players, students of Universidad Autónoma de Mexico – UNAM were chosen from the targeted audience (university students Spanish speakers) and then observed. The invitation was made at Language Center of UNAM and at CCADET of UNAM. They were invited to play the game for an average of one and a half (1.5) h in one monitored room. Before the evaluation, they had to complete a consent and a user profile form. Results were analyzed quantitatively over 111 questions of the 10 game missions with metrics set as either right/wrong/wrong answers followed by second try/absence of answers. Scenarios were compiled based on screens’ video recording, aiming to verify navigation behaviors. The qualitatively study was based on after game-playing interviews in order to identify preferences in terms of game interactions-mechanics proposed. Questions were open: “How was the play experience?”, “What are the positive aspects of the game?”, “What are the negative aspects of the game?”, “Do you have any suggestion?”. Interviews were transcribed and content analyzed in order to identify most mentioned items.

2.1 Participants

Students were on average 27,9 years old, five (5) male, five (5) female. Five of them had studied Portuguese as a Foreign Language during one semester at the Language Center from UNAM and self-evaluated themselves as “basic level knowledge of the language” while the remaining five (5) affirmed had never studied Portuguese in a formal way before. Out of the last five (5), four (4) of them self-evaluated themselves as without any knowledge of the language and one (1) as “basic level of the language” (learned Portuguese in an informal way with online apps). In regards to their playing digital games habit, one (1) was not used to play digital games, one (1) played occasionally and the others were considered familiar to digital games. Classification was as follows: almost never/2 or 3 times per month/once a week/2 or 3 times a week/daily.

In resume, students profile were: Student 1 (S1)- 37 (Age), female, bachelor in literature, studied Portuguese, not used to play digital games; S2- 24, female, bachelor in biology, studied Portuguese, played digital games 2 or 3 times per month; S3- 24, male, bachelor in mechanical engineering, didn’t studied Portuguese, played digital games once a week; S4- 36, male, bachelor in psychology, studied Portuguese, played digital games daily; S5- 23, male, bachelor in biology, studied Portuguese, played digital games 2 or 3 times per week; S6- 25, female, bachelor in computer science, didn’t studied Portuguese, played digital games once a week; S7- 29, male, bachelor in computer science, didn’t studied Portuguese, played digital games once a week; S8- 25, female, bachelor in mechanical engineering, didn’t studied Portuguese, played digital games 2 or 3 times per week; S9- 22, male, bachelor in mechanical engineering, didn’t studied Portuguese, played digital games daily; S10- 34, female, master in dance, studied Portuguese, played digital games once a week.

3 Results and Discussion

3.1 The Game Design

In brief the game design (which will be detailed in another article currently in progress), allows the understanding of the evaluations, where 111 questions with multiple choice (between two and three options) and word/sentences typing answers were part of 10 story main missions-scenarios. According to the persona chosen, the game was built to international students Spanish speakers without previous knowledge of Portuguese or basic level of the language who aimed to learn Portuguese in autonomy (self-directed learning). The 3D environment is the city of Sao Paulo, Brazil including a university campus. The main missions were: 1- Rent a room, Interact with characters; 2- Discover objects in the house, find objects with audio, respective written name and cultural information by clicking on them; 3- Arrive at university’s library, interact with characters, ask for directions, check the map, check text with direction instructions; 4- Discover following missions by texts interpretation, read texts at library and answer questions about their meaning; 5- Buy food and drink at the Coffee Shop, interact with characters; 6- Find street signs, walk around to find street signs and interpret them; 7- Find advertisements, walk around to find advertisements, texts interpretation; 8- Go back home, find the way back to home; 9- Find out the meaning of the message left by flatmates, words used to order food and drinks and interpretation of informal expressions; 10- Arrive at the bar, interact with characters, ask for directions, informal expressions interpretation.

3.2 Patterns Definitions

Patterns to guide both quantitative and qualitative analysis and to build Play-Personas were chosen: habits of playing digital games and existence or not of previous knowledge of Portuguese. This choice aimed to verify the adaptation of each game mission to students from those groups given that one big challenge in the design process of educational games to learn languages in autonomy is to balance content to students without previous knowledge of the language and, in the present research, as well without making it less challenging to students with basic level of the language.

3.3 Quantitative Analysis

To identify the navigation experience of each student, the percentage of questions’ completion of each mission was analyzed. Results are in Table 1.

Table 1. Navigation – percentage of completion/student per mission
Full size table

According to the habits of playing digital games, students with highest percentage of game completion played digital games either daily or 2/3 times per week. Student 1 (S1) with lowest level of total game completion (didn’t do 3 of the missions proposed) didn’t have the habit of playing digital games. S4 with a daily habit of playing digital games and S7 with habit of playing once per week discovered in the final mission unexpected ways to complete the mission, without doing all activities required (pushing against the system [9]). Students 3, 6 and 10 didn’t do mission 3 because they chose a second way to solve this mission, already expected in the game design (not asking characters but following text instructions or checking the map). The design of mission 1 also wasn’t linear, giving liberty to players, which explain the variety of completion percentages. S3 didn’t see the Coffee shop or wasn’t interested in doing this activity. As the game has predominantly a linear story, 100 % of the students did mission 9. Also since this mission (9) had the highest level of difficulty, some students tried to guess the answer and therefore it took longer than expected. One (1) student without previous knowledge of Portuguese asked to the evaluation monitor to pass to the following mission. To evaluate the level of difficulty of each mission and its relation with previous knowledge of Portuguese, Figs. 1, 2, 3, 4, 5, 6, 7, 8, 9 and 10 were created with the quantity of right/wrong/wrong answers followed by second try/absence of answers for each students over the 10 missions.

Fig. 1.
figure 1

Game metrics per student mission 1. (Color figure online)

Full size image
Fig. 2.
figure 2

Game metrics per student mission 2. (Color figure online)

Full size image
Fig. 3.
figure 3

Game metrics per student mission 3. (Color figure online)

Full size image
Fig. 4.
figure 4

Game metrics per student mission 4. (Color figure online)

Full size image
Fig. 5.
figure 5

Game metrics per student mission 5. (Color figure online)

Full size image
Fig. 6.
figure 6

Game metrics per student mission 6. (Color figure online)

Full size image
Fig. 7.
figure 7

Game metrics per student mission 7. (Color figure online)

Full size image
Fig. 8.
figure 8

Game metrics per student mission 8. (Color figure online)

Full size image
Fig. 9.
figure 9

Game metrics per student mission 9. (Color figure online)

Full size image
Fig. 10.
figure 10

Game metrics per student mission 10. (Color figure online)

Full size image

S6 and S7, without previous knowledge of Portuguese made more errors than other students, as it is also shown in Table 2, with the comparison of percentage of the players’ right answers to the average percentage of right answers per missions.

Table 2. Percentage of right answers per mission
Full size table

Analyzing results from the graphs and Table 2, it is possible to observe that the percentage of correct answers for S4, with previous knowledge of Portuguese, was always equal or above the average. When comparing students with previous knowledge of Portuguese S1 (1), S2 (4), S4 (0), S5 (3), S10 (2), and students without previous knowledge S3 (3), S6 (6), S7 (6), S9 (3), it is possible to see (highlighted in the table in grey) results where the percentage is lower than the average from students of the second group. S8, who learned Portuguese in a non-formal way, had less answers below the average (2) than students of the second group. With exception of S7 in mission 3, percentages of correct answers were not lower than 50 %, indicating that students from both groups were able to do activities and missions proposed, leading to a consequently initial adaptation of the game design to both groups.

3.4 Qualitatively Analysis

The content analysis of interviews showed cultural information presented in the game as the most positive aspect mentioned (mentioned by 6 students), followed by presence of informal expressions (by 5 students), street signs and advertisements interpretation (by 5 students), text interpretation of library books (by 4 students), dialogs with characters (by 3 students), audio presence in the second scenario-mission (by 3 students) and the purchase of typical food and drink at the Coffee (by 2 students). Negative aspects and suggestions about missions and interactions were (A) the absence of possibility in other scenarios to interact to objects, hear audio and see written form of word as in scenario 2 (mentioned by 3 students - 2 with and 1 without previous knowledge of Portuguese), (B) the difficulty in mission-scenario 9 (by 2 students - without previous knowledge of Portuguese), (C) absence of audio in all of dialogs (by 2 students), (D) not many activities to write/type the answer (by 1 student - without previous knowledge of Portuguese), (E) many texts in the library (by 1 student - without previous knowledge of Portuguese) and (F) not much action in the second scenario (by 1 student - without previous knowledge of Portuguese, played digital games 3 times per week). Students who suggested the interaction of scenario 2 in other scenarios as well (find the object, hear its pronunciation and see the correspondent word) reinforce the premises of environmental storytelling and embodied learning in terms of memory relating to environment and story. From the group of students who played the game, it was possible to identify five play-personas. Play-Persona 1: not used to play digital games, with previous knowledge of Portuguese (formal learning); Play-Persona 2 and 3: used to play digital games 3 times per month, with/without previous knowledge of the language. Play-Persona 4 and 5: used to play digital games 2 or 3 times per week/daily, without previous knowledge of Portuguese. A Play-Persona not used to play digital games and without previous knowledge of Portuguese wasn’t in the sample. Characteristics of Play-Personas identified were the appreciation of cultural information, informal expressions, original texts interpretation (street signs, advertisements and library books), dialogs with characters, audio and typical food vocabulary. When comparing those aspects to the pre-requirements of the game design (discovered during the construction of Personas at the beginning of its research [20]), it was possible to find the same items (items in bold in the below text): For a Spanish speaker university student persona, the most important Portuguese learning content are: ask for directions, ask questions, buy, food vocabulary, expressing plans, house vocabulary, informal vocabulary and typical informal expressions, vocabulary to rent a room, phonetics. The learning strategy for this persona was to learn with authentic audio and original texts. The negative aspects and suggestions identified will help the improvement of game design such as changing the design of the Coffee Shop to make it more attractive/visible, moving from a linear to a nonlinear story in mission 4 and 9, (allowing students with no previous knowledge of Portuguese to choose the missions’ order or to avoid them), adding more audio and more exercises of writing/typing.

4 Conclusion

The persona created at the beginning of this research, from qualitative content analysis of 30 interviews with potential users (international students who were living in Brazil and Portugal interested in learning Portuguese as a Foreign language) had elements repeated as most mentioned items in the content analysis by the group of students (Spanish speakers interviews living in Mexico) who evaluated the final game version. This evidence indicates the robustness of the Persona that guided the game design. From quantitatively and qualitatively game-playing evaluations, it was possible, among others, to observe lower total missions completion percentage by Student 1 (S1) who hadn’t the habit of playing digital games, decided to reduce his game experience by not doing the middle missions. S1 indicated as well, during the interview, the belief that the game could help with learning Portuguese in parallel of classes. The other students indicated they could learn with the game in autonomy if repeating it and continuing the levels, without the necessity of classes. This difference in opinions is one interesting association between digital games playing habits and the belief about teaching/learning capacity of digital educational games. The evaluations showed that missions were adapted to both students’ groups with and without previous knowledge of Portuguese independent of their digital game playing habits and indicated suggestions of improvements in the game design.