Keywords

1 Introduction

Computer programming course are being included as part of the curriculum at many universities in Japan. In a conventional class, students have to practice to describe program codes as well as study theoretical programming knowledge [1]. However, there is sometimes not enough time for them to do these activities. One of the effective methods for cultivating computer programming skills is a collaborative learning method in a voluntary community. However, it is difficult to improve student’s skill and motivation in such a voluntary community, causing individual differences from person to person in their understanding and attitude. There are many e-learning systems such as MOOCs [2]. There has also been research done on communication assistance systems such as Wiki, chatting, or SNS [3, 4]. But there is very little research that discusses their learning attitudes and motivations.

2 Research Backgrounds

2.1 Activities in Voluntary Community

In our past research [5], it became clear that the following three activities are important to improve user’s motivation in a voluntary community (Fig. 1).

Fig. 1.
figure 1

Activities in voluntary community

Full size image

  • Communication activity: Share member’s status, condition, problem and goal

  • Collaboration activity: Work on common subjects to maintain their motivation

  • Competition activity: Compete with other members to improve their motivation

Moreover, it is important to connect these activities seamlessly to improve student’s motivation.

2.2 Gamification

Gamification has attracted much attention recently as a method to help users maintain and improve their motivation [6, 7]. Gamification is defined as “to use gaming elements, such as concept, design, and mechanics of a game, for social activities or services other than the game itself [8]”. We think that Gamification is effective to maintain and improve member’s motivation in voluntary communities.

3 Research Purpose

In this study, we targeted the voluntary community for programming learning. We propose a support system to emphasize collaborative learning in such a voluntary community by using Gamification.

4 Proposal of Collaborative Learning Support System for Programming Education

We propose a support system that has the following functions by applying the concept of Gamification.

  • Peer Review Function to support Collaboration activity

  • Competition Function (called Programming Challenge) to support Competition activity

Moreover, our system has the platform of the Communication Function such as “Bulletin Board” and “Message function”. Communication Function supports activity in the voluntary community as well as the platforms of Peer Review Function and Competition (Programming Challenge) Function. These functions are connected seamlessly.

4.1 Peer Review Function

Our system consists of 15 exercises, each constituting a different level. Each exercise has 5 questions. Users are designated to have obtained level clearance when users have cleared these questions. The user who has finished solving the questions submits the outcome on the contribution page on our system. The submitted outcome is reviewed by the other users chosen by the peer review function (Fig. 2). But, reviewers are chosen from the users who have level clearance for that exercise. The contents of the review consisted of two fields, the judgment of the result of the exercise and comments on the outcome. Also the reviewer name is not specified. Peer Review can reinforce the learning by reviewing the outcome created by the other users. We think that it is important for users to help and teach each other when learning programming skills.

Fig. 2.
figure 2

Image of peer review function

Full size image

4.2 Programming Challenge Function

We prepared the overall questions for the Programming Challenge (Fig. 3). The questions have two characteristics such as “There can be multiple descriptions” and “The difference is easily noticeable between the users”. There is a submission time limit for an outcome. The administrators evaluate the outcome based on 3 items, earliness, correctness and easiness of reading. The competition function ranks the outcome based on these evaluations. Only the top 3 users are commended in the system after the competition. The program codes of those top 3 users are exhibited in the system, and everyone can read. The title is added to the top 3 users in the site as well as the commendation. Programming Challenge measured the check points of learning achievements. Moreover, this function was made for the purpose of improvement of the desire to learn by competing with the other users for the value of the outcome. Also we put the programming challenge into effect twice under the experiment.

Fig. 3.
figure 3

Image of programming challenge function

Full size image

4.3 Prototype System

To test the feasibility of our proposal, we made a prototype system. Screen hierarchy of our prototype system is shown Fig. 4.

Fig. 4.
figure 4

Screen hierarchy of our prototype system

Full size image

5 Evaluation

To evaluate our system, we conducted the experiment targeted at the voluntary community aimed at programming learning for about 3 months. We set the subjects who belong to a science faculty department of the same university (between 21 to 22 years old, 5 males and 2 females). First we explained beforehand about how to use the proposed system. The account ID was given to each subject at the same time. The programming language learned during this experiment was Python. Furthermore, the authors made the teaching materials used for learning.

In order to verify the effectiveness of our system, we conducted interview survey about the whole system and each function, on 4th, 8th and 12th weekend. Moreover, we put the programming challenge in the 5th and 11th week. We handed out a questionnaire asking about learning situation, whole system and each function after the programming challenge.

5.1 Evaluation of Peer Review Function

We show a questionnaire about the validity of Peer Review function. We established two evaluation items for Peer Review, which were: “Q1: The reviews from other users to user’s outcome” and “Q2: The reviews for the other user’s outcome”. The questionnaire asked the subjects to answer based on 5 stages of Likert scale (5: Useful, 1: Not useful). We show the average value of answers in Table 1.

Table 1. Evaluation of Peer Review function (average value)
Full size table

From the results of Table 1, it became clear that the review activities are effective in learning for about 90% of the subjects in the experiment. Next, based on the result of the interview survey, we classified the effective reviews for motivation improvement after the experiment. The characteristics of an effective review were: “The contents of the submitted review are written clearly”, “Improvement methods for the program are indicated clearly” and “There are descriptions about assumed errors”. On the other hands, the characteristics of an ineffective review were: “Only a judgment result” and “There are no directions in detail on how to think”. From the results of questionnaire and interview survey, the validity of the Peer Review system was confirmed. Moreover, the features of effective reviews became clear.

5.2 Evaluation of Programming Challenge Function

We compared the outcomes during the experiment. Specifically, outcome just after the course starting (1st week), outcome of first programming challenge (5th week) and outcome of second programming challenge (11th week) were used. Three teachers who participated in information education at the university evaluated these outcomes on a scale of 5. Furthermore, alphabets (A~H) are the user number. We considered 2 metrics for evaluation, correctness and easiness of reading. We show the result of evaluation of each item to Figs. 5 and 6.

Fig. 5.
figure 5

Change in the evaluation value about correctness (average value)

Full size image
Fig. 6.
figure 6

Change in the evaluation value about easiness of reading (average value)

Full size image

Figures 5 and 6 show that each evaluation metric is increasing from starting to challenge 1 or 2. Focusing on the correctness metric, about half of the users improved the evaluation value from challenge 1 to challenge 2. Big improvements were seen in the users whose starting evaluation values were low in particular. Focusing on the easiness of reading metric, most users improved the evaluation value from the challenge 1 to challenge 2. We think the improvement of subjective programming skill was confirmed from the above mentioned result.

6 Conclusion and Future Studies

In this paper, we proposed a support system to motivate students’ individual learning in a voluntary community. We proposed three functions to support the three types (Communication, Collaboration and Competition) of activities based on the analysis of their current conditions. To examine feasibility of our proposal, we implemented a prototype system and conducted an evaluation. Through the practical use of our prototype system, the validities of our proposed functions were confirmed.

In our future studies, we revise the prototype system and evaluate it in actual uses.