ARena: Improving the Construction Process of Line-Follower Robot Arenas Through Projection Mapping

The RoboCupJunior Rescue Robot League [20] is a competition whose primary purpose is to create robots able to cross a space that mimics inhospitable landscapes and, in the end, save balls that mimic human beings. At these competitions, the field is simulated as an arena with a white background and a black line representing the track to be followed to the area where the victims of the disaster area. Although in real life, saving people is the most import part of a rescue, at the competition, the track is as important as the rescue itself.

But, for the event organizers, building the arenas is one of the most time-consuming infrastructure parts of the event, given the number of arenas to be produced and the process involved, which is manually performed by the arena judges. Currently, the organization of the Brazilian Robotics Olympiad (OBR) [15] uses software such as the Tournamenter [2] to compute competition scores, but they do not collaborate in the process of producing the tracks themselves. From the concept of Projection Mapping, it was possible to build ARena, a web-based solution that allows projecting the track over a surface, facilitating and making the production of the tracks faster, ending the first, and most difficult and time-consuming part of the building process. Since the tool was made available on the Internet and runs inside the browser, it can be used by anyone who needs to construct tracks for line-follower robots competitions. We have validated the proposed tool with different users, of different levels of experience, and it proved to be useful, according to their feedback and time to accomplish the tasks given during tests.

The remainder of this paper is structured as follows. Section 2 lists some works related to the process of constructing line-follower tracks. Section 3 presents other software used by OBR. Section 4 explains the importance of line-follower robot competitions in the world and specifically in the Brazilian context. Section 5 details Projection Mapping, which is the main technique used by the proposed tool. Section 6 describes the ARena tool, how it was implemented and its most important functionalities. Section 7 explains how the tool was validated according to the user tests performed, while Sect. 8 provides an analysis of the data obtained. At last, Sect. 9 provides final remarks about the work and gives future directions related so that the work can be improved.

Projection Mapping has been used mainly for artistic, but also for educational, health and advertisement purposes. Some examples of these cases are listed as follows.

In [11], the authors propose the use of Spatial Augmented Reality to enhance physical artistic creation. The application works by projecting an image in order to aid users to create 3D drawings without technical knowledge. The tests showed adults drawing uninterrupted for 35 to 45 min and they reported that it was the most sketching experience they have since kindergarten.

Applications of Projection Mapping have also become frequent in cultural exhibitions and advertising campaigns. In 2017, Magic Kingdom Park used Projection Mapping, a technology being employed since 2010, at one of its major attractions named “Star Wars: A Galactic Spectacular”. Other examples of Projection Mappings within Disney park attractions include “Happily ever after” show, where guests become characters in the show. Other famous examples of Projection Mapping being used in popular culture are the “Fête des lumières”, a yearly religious festival taking place in the French city of Lyon every December, and the cinematic movie named Oblivion (2013).

A Projection Mapping technique using a Kinect-Projector system focused on irregular surfaces was published in [13]. One of the first uses of Projection Mapping [22] developed an immerse virtual environment for genomics studies. Other known uses include medical sciences [14] and even face projection [23].

There are several tools for general use Projection Mapping. Lightform is a commercial tool offering a graphics interface to simplify Projection Mapping with an RGB camera and a projector. Resolume Arena is a Projection Mapping tool intended for video jockeys. Lumo Play is another commercial Projection Mapping tool which has been adapted for indoor games. A simple projector-camera calibration system has been written in the C++ language [12], which is also available for use. 7th sense, Ventuz, and Coolux are all examples of commercial tools intended for both indoor and outdoor Projection Mapping applications, especially focused on architectural uses such as building fronts, museums, opera houses and stages. A Projection Mapping tool, DynaMapper, has also been developed specifically for mobile devices. A thorough listing of several tools is available online at [10].

3.1 Olimpo System

The Olimpo System [8] is the site of registration of the competing teams by the responsible teachers and of the management of the teams and events by the coordination of the Olympiad.

It is also used by other robotics competitions and scientific events such as National Robotics Show (MNR) and Latin American Robotics Competition (LARC), as seen in Fig. 1.

With its database, the Olimpo System provides information for other software, the Tournament, explained below.

Olimpo System webpage.

Full size image

3.2 Tournamenter

The other software used in the Brazilian edition of the event is Tournamenter [2]. It is an open source software whose function is to manage time, scores and classification of the teams. since it is a general purpose software designed to help various championships, it has a specific extension for the OBR and RoboCupJunior Rescue League.

The OBR extension communicates directly with the competition information server, downloading specific competition information, such as previously allocated teams for that event. It is also possible to allocate teams at competitions tables, generate schedules, count points of each participating team and send them to the competition information server (Fig. 2).

Tournamenter app setup page.

Full size image

Besides Olimpo System and Tournamenter, there is no official tool adopted by the Brazilian national committee of the RoboCup Junior Rescue League organization responsible for OBR nor by the International Committee for the RoboCup Junior Rescue League that is capable of supporting the construction process of line-follower arenas.

4.1 Importance at the Brazilian Context

In other countries, it is possible to observe a strong creative technological culture since childhood. In these countries, many products that have a strong market potential are patented. In Brazil, this process is still at its initial level, not having a culture that stimulates a better utilization of robotics technology in technological areas or even inside homes [18].

From the point of view of education, robotics has become an obligatory element in modern schools due to its possibility of actuation in many knowledge areas such as chemistry, mathematics, physics, history, geography, even language.

This gives rise to teaching methodologies such as the STEM, which aims to link science, technology, engineering, and mathematics in order to practice what has been taught and prepare students for life and the job market by solving real problems, present in the students’ daily life. Second, [19] STEM should contribute on three main fronts:

1. 1.

   Develop an attentive and capable society in Science, Technology, Engineering, and Mathematics.

2. 2.

   Train students and teachers who are capable of developing the skills of the 21st century [7] within an integrated school environment.

3. 3.

   Generate a research and development force in STEM-focused on innovation.

Along with the methodologies, robotic platforms emerge for educational purposes. Some are “closed” platforms, which do not allow the use of non-platform components such as SPRK+ from Spheros [5], Mindstorms EV3 [4] from LEGO Education and VEX IQ, VEX EDR and VEX Pro from Vex Robotics [21], some of them even have their own teaching plans. Other platforms such as Arduino [1], Texas Instruments Robotics System Learning Kit [9], and BBC-Microbit [6] are also tools used in methodologies, but they are “open platforms” and allow non-platform components to be used, expanding the horizons of learning.

As a promoter of robotics and automation, important areas for a country development, the Brazilian Robotics Olympiad plays a major role, since it works directly with children and young people, approximating them to technological development, robotics and automation. This way, they become direct agents of this construction and not just mere users. The OBR enters this context as a problem-providing space, where students can construct their own robotic devices with a real purpose in a space of collaborative competition. In such competitions, there is no direct confront and the search for innovation is a constant, allowing students to work on content that would only be worked on universities such as PID (Proportional–Integral–Derivative) algorithms, Fuzzy Logic, and State Machines.

4.2 The Brazilian Robotics Olympiad

The Brazilian Robotics Olympiad is the national stage in Brazil of the RoboCup Junior Rescue modality, which aims to simulate an environment hostile to humans related to rescuing in a disaster space.

Nowadays, the current challenge imposed on competitors is the construction of a robot and an efficient program capable of:

• Following the line track,

• Being robust to light variations on the lane and

• Identifying and collecting the victims.

In an analogous way, the challenge imposed on competition organizers is to build new arenas in a short time interval. There are approximately 50 regional phases, 27 state phases [16], and one national final, in which an average of 6 different arena models are produced per event using a paper blueprint as a reference with the measurements of each line and the position of each object in the arena, as shown on Fig. 3.

Schematic sample used for arena construction. Blue rectangles represent obstacles, orange rectangle represent a portal and green squares representing turn signals. Source: OBR Database (Color figure online)

Full size image

4.3 Competition Arena Contents

The main component of the scenario is the safe path, determined by a black line on a white wood substrate [17]. This line has small speed reducers up to 1 cm thick over which the robot must pass without leaving the safe path. It also has obstacles with maximum dimensions of 12\(\,\times \,\)10\(\,\times \,\)25 cm (Length\(\,\times \,\)Depth\(\,\times \,\)Height), which must be bypassed by the line-follower robot after an evaluation of which path is free.

The safe path contains gaps, mimicking small holes in the way. Those should be overcome without changing the robot’s direction. Also are present turn signals indicating the safe path to follow, they are marked as a green square with a 25 mm \(\times \) 25 mm area. Later, in the track, there are orange portals that limit the height of the autonomous robots as at the end of the ramp leading to the victims’ area. This, known as Room #3, has no track and only small balls covered with tinfoil or painted black and they are randomly positioned at the area. The final objective is to put the balls in a triangle area marked in one of the room corners. When the last victim is saved the clock stops. Competitors have about 5 min to complete all the round. Different track models can be seen in Fig. 4.

Arena samples used in previous competitions. The left one has an orange portal and it is an easier level one. The right one represents a harder level.

Full size image

Projection Mapping is a well-known technique in Computer Vision and Augmented Reality sciences used to transform surfaces present in the surroundings into display surfaces for video projection. The main idea behind Projection Mapping is that any surface, be it flat or irregular, may be mapped either by an automated or user-guided procedure and used as a display. Since video output is usually designed for flat screens, the transformation from a flat surface into the desired output surface needs to be computed and applied to the image prior to projection.

For specific purposes of this application, the desired projection surface is a flat board of unknown parallelepiped dimensions, implying a projective transformation is generally sufficient for a precise projection. Since the transformation must be computed only once, considering that the camera/projector/surface system will remain static in respect to each other, a semi-automated method was chosen.

The transformation from the screen (camera) coordinates to the surface to be projected can be calculated by multiplying two sub-transforms, one from screen to projector and the other one from a projector to surface. However, since both operations are computed by homographies, the transformation matrix can be compounded into a single operator. Let (X, Y) be the coordinate vectors of the corners of the image desired to be projected. The coordinates are transformed into a homogeneous coordinate system such that \((x,y)\rightarrow (x/w,y/w,w)\). Note that in spite of the added number of dimensions in the vector, the number of degrees of freedom remains unchanged by the transformation. This important property will guarantee that a unique inverse transformation exists.

In this method, an RBG camera is used to capture the projection surface. To find each of these transformations, the user must provide four reference points to be correlated to reference ones. The four selected points within the captured image will possess another set of coordinates \((X',Y')\) (likely distinct from the original set of coordinates). Therefore, a projective transformation exists such that \(Ma=b\), where M is a matrix operator, a is a 2D-homogeneous coordinate in the original image and b is a 2D-homogeneous coordinate in the camera captured image. This matrix operator must satisfy the constraints imposed by all four points indicated by the user. In order to solve for this matrix operator, a homography is computed [24]. As stated, this is equivalent to a projective transformation that maps the coordinates from one system (image) into another; thus projective mapping. A second homography must be computed to transform the points into the projected surface. In this manner, the first transformation maps from the camera to the projector and the second transformation maps from the projector to the surface. Once the mappings have been done, the coordinates are transformed from homogeneous coordinates back into Euclidean space by dividing each element by the last (w).

For instance, Fig. 5 illustrates the pattern that is projected in full screen and later captured by the camera. The user is asked to click at the center of each colored circle, in a specific order. This way, a relationship is made for both coordinate systems. The next four points to be provided by the user should be the corners in which the pattern with the line track should be projected, also clicked in a specific order. These points provide information that enables acquiring the projection to surface transformation. Using the final transformation, composed of the two sub ones, one can warp the original image to the projected real scenario, according to the input given by the user.

The four corners of the pattern projected for screen(camera)/projector calibration. The specific color order is used: red, green, blue and yellow. (Color figure online)

Full size image

6.1 Browser Implementation

The ARena tool runs inside a web browser and was implemented using solely HTML and Javascript. Its usage is straightforward, however, a proper equipment setup is required before using this tool and the image containing the blueprint of the track must be previously produced in a graphics editing software and exported in advance.

The use of HTML + Javascript allows the solution to be portable, with easy visual prototyping (HTML) and of fast behavioral programming (Javascript). Since Javascript is an interpreted programming language, it does not require compiling the code every time a change is made. This allows developers to almost instantaneously visualize changes made directly on the browser.

6.2 Setup and Use

Besides the computer that will run ARena (and the common materials used for the constructing line-follower arenas, such as the whiteboard and the black tape), a specific hardware setup comprising a projector and an RBG camera is required. A tripod is also recommended for positioning the projector at an adequate place. The projection surface must be placed in a position such that the projection area contains the board to be projected. A small extrapolation should be fine. The camera must be also fixed in the environment (it is usually placed on top of the projector), looking at the center of the surface to be projected (shown in Fig. 6). It is also important that the entire projection area is seen by the camera. This enables the user to click on the borders at the final stages of the process.

ARena equipments’ setup. While the computer is omitted from this image, the green line highlights the camera field of view while the red one highlights the projector window. (Color figure online)

Full size image

After the equipment’s setup, the user simply has to open ARena’s website^{Footnote 1} and follow the step-by-step instructions provided on the screen.

In the case of multiple platforms with the same model, if a new whiteboard is placed in exactly the same place the old one was, no recalibration process is necessary. The user only has to load the new blueprint image and it will be projected following the previous calibration performed. In case the user is just replicating the whiteboard using the same reference image, this last step (loading of a new image) is not necessary.

7.1 Test Methodology

Two different blueprints of tracks were used to conduct user tests, as shown in Fig. 7. Despite different, the organizers of the competition reported that both arenas chosen were equivalent in terms of construction difficulty.

A total of ten users were separated into two groups of five users each, defined by the arena type to be constructed using ARena software and a manual scale drawing method. A uniformly random process selected whether the user would first build the path using the ARena software or the manual scale drawing. This way, it was possible to make sure that the timing of the test would not be influenced by the previous knowledge of the arena design at the second attempt.

Different but equivalent arena blueprints used during tests: type 1 (top), type 2 (bottom). (Color figure online)

Full size image

The manual method consisted in drawing on the official whiteboard by mimicking a scaled schematics of the track and then putting adhesive black tape over it or simply by drawing control points and connecting them with black and green tapes. Instructions on how to do the manual method were passed to test users.

Using ARena software, test users were oriented as well as how to configure the software by one of the authors and, then, they were told to start the process. The steps to perform ARena configuration are shown on Figs. 8, 9, 10, 11, 12 and 13.

Step 1 of ARena configuration: Open ARena website and authorize the use of the camera.

Full size image

Step 2 of ARena configuration: Select the image to be projected.

Full size image

The construction time was measured considering both the user’s preparation in the manual scaled drawing method and the ARena configuration in the ARena Software method. The time count was ended after checking that there were no missing parts and the tape was not peeling off the surface.

It is important to consider that some of the tested users had previous experience drawing arenas before. This leads to different test results when compared to other users that had no previous experience constructing line-follower arenas. Moreover, there were different ways to draw the track with the manual method, and each person chose their preferred method. The most popular technique was sketching the whole path before taping it. The second most used approach was marking the extremities of each line, based on the schematic, on the arena and then linking those marks with black tape.

Step 3 of ARena configuration: Press the “P” key to go full screen, then press the “Q” key to capture image from camera and then press “ESC” key until canvas image comes back to its original size.

Full size image

Step 4 of ARena configuration: Click on the center of the colored dots in sequence and then click the corners of the board in the same sequence then press the “R” key to render the projection. (Color figure online)

Full size image

Shortly after the end of each test, pictures of the board were taken so later the user could, also, compare the final result of both methods.

7.2 Hypothesis Testing

Given the conducted user tests, a hypothesis test was made to infer whether the software usage had any actual advantage that could be statistically significant. Given the number of available samples, a two-tailed T-Student statistic test was performed [3]. The status quo was that the usage of the software did not incur at any advantage when compared to the traditional manual construction. This meant that a statistically significant difference between means showed that there was a clear difference between both methods. If the software-assisted meantime was lower than the manual meantime, again in using the software-assisted construction method was implied.

Step 5 of ARena configuration: After the calibration is done, close the dialog window by clicking “Ok” and then press the “S” key on the keyboard to enter fullscreen mode.

Full size image

Step 6 of ARena configuration: The calibration is completed and the image is projected on the board. The setup is ready to be followed by placing the adhesive black tape over the projected track.

Full size image

For arena type 1, the difference between means of both groups was of 1039.40 s (as shown on Table 1). With a 95% confidence interval, this translated into a 419.14 to 1659.66 range. The computed two-tailed P-value was of 0.0048, which meant a statistically significant difference between both groups. Since the software-assisted time was lower than the manual construction, this meant a clear advantage in using the proposed software.

The same hypothesis testing was done for the second track (look Table 2). This time the difference between both groups was lower, 722.20 s. However, with a 95% percent confidence interval the expected range was between 201.17 s and 1243.23 s. This meant the means for the second arena type were also statistically significant with a two-tailed P-value of 0.0127. Since the mean for the software-assisted group was also lower, the tests pointed out a clear advantage in using the software-assisted technique also for the second track. Given the disparities between mean user construction times for both arena types, it is not unreasonable to expect a similar behavior for other track types that follow these same assembly principles.

After the tests, it was possible to confirm that all users had less track production time when using ARena (as shown in Tables 1 and 2). Tests performed in type 1 arena pointed a reduction of time around 65%, while tests performed in type 2 arena pointed a reduction in total time around 55%.

By interviewing users after the tests, they pointed as main challenges of constructing the track without ARena: the correct use of the scaled drawing and keeping it uniform; the necessity of using many tools such as a square ruler, rulers, and compass. Both challenges made the manual approach demand more time in the users’ opinion.

After the tests, all users agreed that the most efficient method was the one using ARena software. It has advantages such as precision on scale and simplicity to tape the most complex shapes. The biggest problems identified were curve-based paths and long diagonals. Test users claimed that without ARena software it was almost impossible to make all those shapes correctly and identical in every single one of the tables. This fact is shown on Figs. 14 and 15. They compare the same track done by a single person with and without the software. As a result of the proposed method, different people were able to stick to a pattern for the tracks better than anyone drawing two of them, as shown in Fig. 15. This pattern is important for the Brazilian Robotics Olympiad so that the championship is kept fair for every participating team and none of them will be harmed due to human error.

Type 1 arena constructed using manual (conventional) method.

Full size image

Type 1 arena constructed using ARena software.

Full size image

Another interesting observation is that users that completed the track using ARena software first always started complaining about the time spent drawing using the conventional method and how hard it was to do it. On the contrary, the ones that did the process backward were relieved to have this tool and impressed by their time.

Some suggestions were pointed by users, such as an upgrade on the ARena’s visual interface and the reduction of the number of commands to set up the software.

Therefore, ARena reached its objective by reducing the production time of line-follower arenas, while maintaining a very similar pattern of the geometric shapes (especially curves) between them.

Users also commented about the facility of making a more precise and faithful path, due to the scale adjustment of the software, independently of the camera position and the table size/position. Some issues about ARena’s included its visual interface, which was not made considering best practices on user interfaces, so it is still pretty rudimentary, leading to a less intuitive program with a series of commands to execute the desired function. Another problem is the external setup, which requires a considerable space and a base support for the projector. Definitely, the most time-consuming procedure of the method is the structure set up. The projector support can be avoided, for instance, if the arena is constructed in a vertical position.

Nonetheless, ARena users only needed to set up the software configuration once, before starting to use the black tape, which only took a few minutes for them to have the full track design projected on the surface. After that, they were free to keep working on the arena and finish it in a short time.

It is also important to mention that some testers did not draw on the whiteboards before tapping them, which resulted in more imprecision, but faster procedures that must be considered at the result analysis. The Brazilian Robotics Olympiad needs that arenas be as similar as possible so that the championship remains fair for all teams.

The proposed tool showed to be effective in reducing the time needed for producing line-follower arenas, according to the tests performed.

Another important gain that comes from ARena’s use was the standardization of the tracks, even when made by different people and on different surfaces, as long as the calibration is not changed. Such quality has a positive impact on competition’s fairness since all participating teams compete on the same track conditions.

However, some improvements are needed as far as the user experience is concerned. From a simple click confirmation when performing the calibration steps to a more friendly interface, such improvements are planned to be performed in the course of this year, according to the feedback received when ARena is used by organizers from all country. ARena’s next version should contain a track editor along with its Projection Mapping functionality. This way it will be self-contained, freeing the user from having to create the track image using a third party graphics editor application.

From the combination of Projection Mapping with line-follower robots, the idea of creating a dynamic arena emerges. Such arena would have a projector underneath it, and tracks could be easily changed, reducing the construction time to near zero. This will be further investigated in a near future.