Keywords

1 Introduction

The use of technological resources in the teaching and learning process has grown in recent years. One of these resources is learning objects, which can be defined in general terms as a way to organize and structure digital educational materials. According to Wiley [18], learning objects include any digital resource that can be reused to support teaching and learning. Thus, learning objects can be characterized as different types of media, such as images, videos, graphics, and sounds.

The use of learning objects to promote sharing content has been discussed in the context of e-learning [17], which can be defined as a way to deliver information and instruction to someone through the Internet. The possibility of delivering and sharing content across multiple locations is appealing. However, it requires the adoption of standards in order to be successful. The Sharable Content Object Reference Model (SCORM) was proposed to handle this challenge. According to Bailey [3], SCORM aims to incorporate various specifications of teaching objects, thus enabling compatibility with various types of learning management software (LMS). In other words, the SCORM defines the interface between web-based content and learning technology using existing specifications. The advanced distributed learning (ADL) model developed the SCORM in agreement with the US Department of Defense in order to formulate a strategy for education and training.

Several types of media can be used as learning objects. However, new needs arise regularly as new media possibilities, such as using virtual reality (VR) and augmented reality (AR), are developed. AR is defined as a variation of VR [4], but while VR completely immerses the user in a computer-generated environment, AR adds virtual artifacts to real viewing, giving the impression that these artifacts are part of the physical environment.

The early development of AR and VR can be traced back to Ivan Sutherland who created the Ultimate Display in 1965 where matter could be controlled by a computer [14]. Three years earlier, Sutherland constructed the first VR system, complete with a head-mounted display and seeing throw optics. This research was conducted during the 1970s and 1980s by members of the United States Air Force Armstrong Laboratory, NASA Ames Research Laboratory, Massachusetts Institute of Technology and North Carolina University at Chapel Hill [15].

This paper presents a new tool called Web Augmented Reality-based Authoring (WebAR), which can be used with Experience API (xAPI). This tool allows teaches to create educational applications of AR that can benefit the learning process. For example, the applications can assist students with difficulties related to spatial skills. This paper’s major contributions are:

  • A discussion about AR applications as valuable didactic material;

  • A tool that can generate AR applications easily;

  • A tool that packages AR applications compatible with xAPI;

  • A discussion about the sharing of AR applications through repositories.

This study is structured as follows: Sect. 2 discusses previous work related to this research; Sect. 3 describes AR and provides details about the tracking system used in our solutions; Sect. 4 discusses e-learning systems and the xAPI; and Sect. 5 presents the WebAR tool, as well as conclusions and suggestions for future work.

2 Related Works

Jee, Lim, Youn, and Lee [11] have created an authoring tool called An Immersive Authoring Tool for Augmented Reality-based E-learning Applications. In their paper, they show that online teaching has evolved from models without interaction to those that use a list of activities and questions and answers as a form of interaction and capturing the attention of students. However, they find these activities to be dull and unintuitive, and believe that AR can be more efficient in these respects. The paper also identifies the existence of libraries that allow applications with AR authorship to be constructed, such as ARToolKit, which uses computational vision resources to identify markers for 3D model overlays. The efficiency of this system was evaluated by a comparative experiment with primary school students in English and science classes, in which half of them took a class with traditional audiovisual material and the other half were exposed to the immersive e-learning system. Teachers and students reported that they saw more benefits from the immersive system because the students concentrated and participated more actively in class.

Farias, Dantas, and Burlamaqui [9] published a study presenting the creation of the Educ-AR Application Programming Interface based on ARToolKit. The main contribution pointed out by the authors is that it does not require the instructor to have programming knowledge, AR applications, or 3D modeling. Instead, they created a web tool where educators could register themselves, create classes, and submit markers and 3D objects. AR artifacts are generated within a class by associating 3D objects with markers. The application also allows for the management and creation of custom bookmarks. Through experiments, the researchers concluded that the use of AR technology could not be considered an efficient solution by itself.

This paper by Farias, Dantas, and Burlamaqui [12] presented AR as an assistance technology for students with special needs, particularly with communication difficulties. The current paper identifies the possibility of using a hybrid method, that is, not only recognition made through computer vision, but also through the use of a global positioning system (GPS) to add the virtual objects to the display of the real content.

Cubillo et al. [5] mentioned that new technologies such as Web 2.0, mobile devices, virtual environments, and AR environments have the potential to transmit information and acquire knowledge. According to the authors, the AR environment proves to be timely for teaching subjects that involve practical training and outside the classroom, where students need interaction with the real world. However, many educational applications of RA are subject-specific, fixed content that was made available by developers.

This analysis of the literature shows that AR has been the object of study of some researchers who aim to ally it with the learning process through electronic tools, such as E-learning systems. The objectives of the papers analyzed here were to create tools that would make the process of authoring AR comfortable for anyone, without the need for the involvement of AR specialists.

An important point raised in our paper is the flexibility of the tools. Some applications aim to use AR in education. However, these programs are restricted to a discipline and, with the flexibility of the tools, it becomes possible to create content that is applicable to several areas of education and, especially, disciplines that require interaction with the real-world medium. It is also noted that the use of AR can help people with special needs learning, thus leading to greater social inclusion.

3 Augmented Reality

AR is most commonly presented to the user through a video display in real time, although the system needs to track the environment to decide how to display the virtual objects. Figure 1 depicts a sequence of tasks performed when an AR scenario is assembled on a handheld device, such as a smartphone [16].

Fig. 1.
figure 1

Sequence of events to assemble an AR scenario on a handheld device.

Full size image

Each phase can be described as follow:

  • Physical environment: the real environment is seen by the user without any interference of the computer system;

  • Capture: the video camera of the device captures the physical environment scene;

  • Tracking: the system tracks the markers;

  • Application: the system interprets the markers to decide how and what virtual objects will be added to the scene;

  • Render: the system draws the virtual objects; and

  • Display: the system combines the real image from the environment and the virtual objects to display to the user or users.

The tracking technique employed in our solution is based on fiducial markers, which are based on and performed by the JavaScript (JS) library JS-Aruco, which is a JS port of ArUco, a library based on OpenCV for AR [2]. JS-Aruco is capable of detecting markers with a 7 × 7 pattern in the external border, as depicted in Fig. 2(a). Figure 2(b) provides an example of a complete marker.

Fig. 2.
figure 2

(a) A pattern of a JS-Aruco expected marker; (b) a complete marker, using the inside patterns for rows (1, 2, 3, 4, 1).

Full size image

A combinatorial analysis shows the number of possible markers that can be distinctly used by JS-Aruco:

  • Possible rows (P): 4;

  • Number of rows at one Marker (N): 5; and

  • Distinct possibilities: Ar(P, N) → PN = 45 → 1024.

This combination shows that JS-Aruco can detect up to 1024 different markers. Three.js is the library responsible for rendering the chosen 3D model. It is an open-source API/Library JS library that allows for 3D programming for web browsers and it is capable of working on several rendering engines (WebGL, Canvas, and SVG) [8]. WebAR also supports JS library posit that estimates the position of 3D objects in the rotation and position axes, x, y, and z [1].

4 E-learning Systems and Experience API

The first generation of e-learning systems released in the early 1990s was similar to a black box. These were proprietary systems with no user tracking [6] that were commonly designed for a specific course. On the other hand, e-learning standards were coming on the scene, such as Dublin Core, IMS Learning Resource Metadata, and IEEE Learning Object Metadata. In 1997, two important learning management systems (LMSs) had come to light—Web Course Tools (WebCT) and Blackboard—and the following year saw the birth of the Modular Object-Oriented Dynamic Learning Environment (Moodle).

In 2000, ADL released the first version of a reference model called SCORM. It was designed as a set of unified specifications for content packaging and description that also determined how the content would communicate with an LMS. SCORM was designed to support interoperability and portability for web and has been widely adopted. However, the appearance of new platforms and the requirements for learning records with analytic purposes have contributed to the development of proprietary or middleware technologies, indicating that SCORM has achieved its limits [13].

With the intention to standardize new elements in the e-learning platforms, especially learning records, ADL issued a call to the community that led to the creation of Xapi, which is focused on defining an interoperable data model for storing data about students and sharing this data among systems [7].

xAPI is focused on tracking learning experiences (LE), which can reside in any activity executed in the learning environment. To achieve a functional way store and share LE, xAPI specified a data model that cognitively called statements in the following format: <actor> <verb> <object>, with <result>, in <context> [7]. JavaScript Object Notation (JSON) is the chosen format to present this data. Figure 3 depicts a JSON xAPI statement.

Fig. 3.
figure 3

Adapted example of a JSON xAPI statement

Full size image

This statement is compounded by the properties exposed in Table 1.

Table 1. Properties of an xAPI statement.
Full size table

A generic architecture for xAPI may consist of the following three components [10]:

  • Learning record provider (LRP), previously called the activity provider (AP);

  • Learning record store (LRS); and

  • Learning record consumer (LRC).

The LRS is a server that receives, stores, and gives access to the records generated during an LE. The experiences are tracked by an LRP, the entity in charge of sending the records to the LRS. An LRC is a client to the LRS that can access the records after authentication. Figure 4 overviews this architecture, showing that an LRS can have several clients in an 1..n relationship, as the LRP and LRS have 1..1 relationships.

Fig. 4.
figure 4

xAPI Architecture overview.

Full size image

5 WebAR: Web-Based Tool for AR with Support for xAPI

Figure 5 presents an overview of the WebAR architecture, according to the teacher’s view. The teacher uses this architecture to create the learning objects based on AR and stores them in a learning object repository. To develop the application, it is fundamental to consider the content of a given class and the teaching strategy used.

Fig. 5.
figure 5

Teacher’s view of the WebAR.

Full size image

Figure 6 presents an overview of the WebAR architecture from the student´s point of view. Students have two options for using objects: (1) via an LMS, such as Moodle, teachers can import objects into the virtual learning environment; or (2) via a mobile viewer running on Android or iOS. In this case, the student needs to install the application on their smartphone in order to be able to import the previously available objects into a repository.

Fig. 6.
figure 6

Student’s view of the WebAR.

Full size image

WebAR uses navigator.getUserMedia API to acquire access to the device camera where the user is authoring the AR scene, at which point the video is called back to an HTML5 <video> as a stream. This stream also feeds the object AR.Detector through its method detect(), helped by a HTML5 <canvas> where the ImageData object is captured. Figure 7 contains the HTML code (a) and the JavaScript part of the implementation described above (b).

Fig. 7.
figure 7

(a) The HTML5 elements for video handling; (b) JavaScript implementation for the Marker detection.

Full size image

The Three.js library handles the 3D model. First, the user must inform the appropriate URL of the model on the format Three.js JSON, using the field labeled “Model.” The model is then loaded by the object, THREE.ObjectLoader(). Figure 8 shows the loading function used to load the model.

Fig. 8.
figure 8

Screenshot of WebAR with a model of a teapot on a detected ID-233 marker.

Full size image

The function implemented on WebAR also captures the three axes (x, y, and z) of the model to provide its manipulation. Figure 6 shows the implementation of this function. Figure 9 also shows the controllers that allow the user to adjust the model on the referenced marker, including the rotation, distance marker, and the size of the model.

Fig. 9.
figure 9

Function CreateModel3d (url) responsible for inputting the 3DModel in the AR scene.

Full size image

As soon as the user detects the desired marker, he can push it and work freely to load and manipulate the model. This feature facilitates the authoring process. The Gerar pacote button packages the AR application into an xAPI-compliant package. When the user presses this button, WebAR collects the relevant information about the scene, including the marker ID, the model position, rotation and size, and copies the model files to a temporary directory. Next, the tool uses a template to create an HTML file that implements the scene adjusted by the user.

The last part of this task consists of creating a zip file containing the HTML file, model files, and the manifest file in the XML format used by LRS.

6 Conclusions

The use of AR as a learning tool can support educators in several disciplines. In order to do this, educators must have authoring tools that are capable of facilitating the entire development process of AR applications and simplifying everything from creating content (3D objects, images, and sounds) to using them in educational institutions. Another important issue addressed in this paper is that while these tools are capable of generating educational AR applications for any context, different teachers do not reuse them. None of these tools deals with, for example, packaging applications in the format of learning objects. Therefore, an opportunity and challenge exists to create authoring tools that are easy to use and are reusable by educators. In line with the related literature, this research shows that there is a need to create user-friendly authoring tools, but that there are still challenges that need to be overcome. Moreover, none of these articles addressed the reuse of such applications by other teachers.

Thus, our paper present an authoring tool that is capable of generating easy-to-use AR content that can also be reused. The main idea is to package these applications in the format of learning objects in order to satisfy the reuse of the xAPI standard. In the future, we intend to improve the WebAR interface, and we also aim to create AR applications that can assist students with specific learning disorders.