Keywords

1 Introduction

Enactive systems have been defined as computer systems consisting of human and technological processes dynamically linked, i.e., forming feedback loops using sensors and data analysis, enabling a seamless interaction between human and computer [4, 22]. The presence of new technologies and new forms of interaction (tangible, wearable and natural interfaces), coupled with the ubiquity of computing and social networks, present challenges that require the consideration of new factors (emotional, physical and cultural) in the design of systems we are naming socio-enactive. Such systems represent a complex scenario for which there is still no theoretical and methodological basis and no practical experiences suitable for its design. The ways people will interact in socio-enactive environments is not easy to explain and to predict at design time. Consequently, the essence of how to develop systems of this nature requires the treatment of technological aspects in which the consideration of the social dynamics is enhanced by concepts such as Universal and Participatory Access, Affective Computing, Ubiquitous, Values and Pragmatics.

In this paper, we present preliminary results from Project Socio-enactive systems [6] in which the concept of a socio-enactive system is being constructed. Adopting a socially-aware view [5] for the design of systems, we applied a Semiotic Framework inspired structure and the Principles for Affectibility to analyze artefact solutions proposed for hospitalized children. The results provided better understanding of the context, more comprehensive view of the state of the art, pointing out to gaps that represent new opportunities for research. More than contributing to problem understanding in the context of our study, the paper shows how a Semiotic Framework can be used as a thinking tool for organizing related works and guiding discussions on the subject. This article is organized as follows: Sect. 2 informs the theoretical background; Sect. 3 shows related works on digital solutions for hospitalized children; Sect. 4 analyses the related works based on a Semiotic Framework inspired structure and the Design Principles for Affectibility; Sect. 5 concludes this article.

2 Background

In earlier days, computer use was limited to performing tasks that were well defined and most often spatially confined to individual offices. Current ubiquitous and immersive scenario demands new forms of interaction. More in line with contemporary needs is the concept of “enactive systems”. According to Kaipainen et al. [12], an enactive system does not assume traditional interfaces (i.e., one that uses graphical interface manipulated by traditional input devices such as mouse, keyboard and touchscreen), and do not follow patterns of goal-oriented interaction models and conscious human actions.

Enactive Systems come from the concept of enaction discussed by Varela et al. [22]. The premise is that interactions occur in an “embodied” way – that is, it is guided by the body’s involvement and the human agent’s spatial presence. Moreover, in these interactions, a conscious control of the system is not assumed to exist [12]. In an enactive system, the system can pick up information (e.g., collect user data) during interaction and respond accordingly. The user then reacts to the system’s response, generating new inputs to the system. This cycle goes on in a dynamic and nondeterministic way. Other concepts that characterize the enactive approach are: autonomy (‘autopoiesis’), sense-making, embodiment, and experience [21].

Usually, enactive systems rely on findings from Artificial Intelligence and Affective Computing. In Affective Computing [14], users’ emotional responses are recognized in an automated way by computer systems and they are used to promote interaction improvements. This type of approach, based on objective information, can be considered as an “informational approach”. In contrast to informational approaches, there are the “interactional approaches” [7]. Such approaches seek to preserve the subjective nature of emotions. Boehner et al. [7] argue that affective interactions are dynamic, culturally mediated and socially constructed experiences. That is, affection is a product of society. The values and culture of society are important in building meaning for affective states, affective behaviors, and words that denote such states and behaviors. On the one hand, purely informational approaches lose the richness of meanings and accuracy of actual emotions. On the other hand, purely interactive approaches can produce results that are considered too subjective or too specific for a certain group. We argue in favor of both approaches in enactive systems that are meant to be enacted within social groups. One possible way to get closer to this balance in enactive systems is to consider the Affectibility of the systems, while still counting on advances from Affective Computing and Artificial Intelligence.

The term Affectibility [9] is similar to the terms usability, learnability or playability in the sense that it is concerned with aspects that provide improved affective quality to interaction with computational systems or devices. While the principles for Affectibility might not be enough to guarantee users’ satisfaction, they indicate elements that can be directed when aiming at enhanced quality in ‘interactional’ interfaces. The principles are summarized in Table 1. Examples can be found in [9].

Table 1. Design principles for affectibility.

Sensors and automatic recognition of emotions provide for enactive systems in terms of inputs for a system to be capable of responding to one user. A new generation of enactive systems should go beyond that; and the principles for Affectibility could address this need. This is one of the facets that shall compose the socio-enactive [6] concept. Socio-enactive systems shall not only transcend individual use by accommodating entire communities, but also it shall encompass dynamic, social and cultural aspects of interaction. The principles can be used both to evaluate an existing solution and guide the creation of new ones. In this article the principles are used to analyze and compare solutions from the literature review.

3 Entertaining Hospitalized Children

When exploring the literature we searched for investigations on computational technology for hospitalized children. Only articles that present a specific artefact were included. It was not in our scope: studies or evaluations of existing commercial applications; solutions directed to specific diseases (e.g., children with asthma) or the treatment or monitoring of health; and solutions in which children were not the patients (e.g., technology for children visiting the hospital). We searched for combinations of terms like “children”, “hospital”/”hospitalized” and “interaction”, “design” at the IEEE and ACM digital bases. Citations that appear in the resulting articles were also included.

The simplest solution is the robot CareRabbit [8]. The authors argue it is quick, easy and secure to implement. It is a device in the form of a white rabbit that receives and plays mp3 files. Family and friends can send messages, stories and music to children in hospitals via the CareRabbit. Orb [24] is another object that, like the CareRabbit, allows one-way communication and was designed to have minimal functionality. Orb is a large light globe placed in a classroom in order to represent the child who is away at the hospital. From the hospital and using a laptop a child can control the color displayed by Orb, thereby suggesting the child’s presence in the classroom. The child is free to choose the color, which may represent an affective response. Children at school are free to interpret the meaning of the color displayed. This communication served as a simple phatic connection between the classroom and the child [23]. The authors of Orb further developed the idea and after conducting co-design workshops, they proposed an application for tablets. The Presence App [23, 24] shows colors as Orb does, but now in a two direction communication, as both classmates and the hospitalized child are able to send and receive colors. The colors are shown as “blobs” that move around the app’s interface. The speed in which the “blobs” move indicate the amount of activity going on in the classroom (the higher the sound level in the classroom, the faster the “blobs” move). Moreover, the app allows photo exchange and it displays the schedule of the class.

In the category of tangible interactions, most of the solutions presented a mix of manipulation of an object and the visualization of video that are displayed in separate monitors. Zootopia is a system composed by a board game (map made of cardboard) with action figures and a monitor. The action figures are in the form of animals, and an avatar to represent the child. The child can freely move the avatar around the map and the avatar is embedded with RFID reader. The map has RFID tags and whenever the avatar is placed in one of them, a video starts playing at the monitor. The video is related to the corresponding animal placed in the map. The same authors of Zootopia also proposed Push Planet [2]. The last one keeps the idea of displaying videos of different animals; however, they make use of many props without the need of a cardboard base. These props include a stuffed planet (a soft ball covered with an atlas made of fabric), a wooden airplane to simulate flights around the planet, among other toys. They also rely on the RFID technology for interaction. The fabrics that cover the toys are removable and washable. Billow [16] is an older tangible proposal (from 1997) that is still relevant for today’s settings. The child handles an object in the shape of an egg in order to control the images of clouds that are displayed on a monitor. The egg can command the clouds to play music. The egg has buttons, a microphone and it connects via radio to a local computer, placed under the child’s bed. The egg can also be used to command the establishment of voice connection with a child from another room in the hospital.

For children who are not confined to a room and can walk around hospital’s common areas, Huerga et al. [10] have conducted two design play workshops, one for each game: Doctor Giggles and X-Safari. With Doctor Giggles, the image of an interactive operating room is displayed on a monitor and children can interact with its elements, like X-ray machine, scissors, and syringe. These objects, however, have different uses in the game (e.g., syringe is a magic wand) and the objective is to make the character (a doctor) laugh. The authors asked children to create their own play characters using X-ray sheets. It is not clear how the characters were incorporated into the game. With the second game, X-Safari, children wear a glove device to hold the characters they create – as in Dr. Giggles they create their play characters out of hospital supplies. With the glove device children control a character that is displayed in the monitor. One solution for playful communication system between child and parent is The Huggy Pajama [20]. This system is intended to allow parents to send hugs to their children. At one side, the parent hugs a doll. The doll, equipped with 12 sensors, senses the hug and transmits it to a wearable device. The wearable device consists of a pajama that changes color and that has 12 air actuating devices. The air actuating devices correspond to the 12 sensors in the doll. Air is pumped into the devices in a way that the pressure it exerts simulates the feeling of being hugged. The pajama has a flower pattern made of thermodynamic ink that is activated by conductive yarn. The changes in color indicate the distance the parent is from the child and the warmth contributes to the sensation of being hugged. Another proposal involving the idea of hugging is Huggable [11, 19]. Huggable is a robot developed at the MITFootnote 1. It is a teddy bear that can be operated remotely; it moves its joints (elbow, shoulder, etc.) and nods and tilts its head. Given a specific coordinate, which is transmitted via wireless communication, Huggable can look at specific directions. The joints of Huggable are flexible enough to allow not only its movements by command, but also to allow children to manually move the robot’s arms and legs, as they would in a regular teddy bear. Moreover, Huggable has eyelids that can move and pupils that can vary in size. All these movements together provide Huggable with emotional expression and nonverbal communication skills. Capacitive touch and pressure sensors are spread throughout the body of the robot. The pressure sensors located in its paws are meant to let children express intensity of pain by pressing the paws in a harder or softer way. These sensors located at the bear’s arms can be replaced by other sensors or parts. For example, in order to let children pretend that they are applying an injection to the bear a place for injection can be attached to the arm. Also, like in Push Planet [2] the fur is removable and washable. This was intended as an infection control mechanism at hospitals. Even though Huggable was designed based on medical staffs’ suggestions, it stills needs further testing. Very similar to Huggable, there is Probo [17]. Probo is also intended to be huggable. It looks like an elephant and it can also express emotions via its eyes, which can move and tilt. Probo’s trunk, ears and mouth move to aid to the expression of emotions. The remote communication with family and friends takes place from Probo’s belly, which has a touch screen interface. Probo is expected to make sounds, as a non-existing, affective language.

4 Literature Results Through the Lens of a Semiotic Framework

Previous works have used the structure of the Semiotic Ladder with the purposes of organizing systems requirements [3], and even of showing different focusses of a literature review [14]. Similarly, we organized the discussion on the main issues related to the artefacts proposed for ‘hospitalized children’ in a Semiotic Framework [13, 18] inspired structure, with six layers: physical world, empirics, syntactics, semantics, pragmatics and social world. The physical world includes physical devices – mainly hardware or other objects – that are used to build structures that represent something else (an animal, the sensation of being hugged, presence, etc.). Most of these devices are in the form of sensors that are set inside an object. Be it a toy, board game or wearable device, usually the interaction happens via a tangible object. Also relevant for the physical layer – however not considered in the works retrieved – are power supply issues: How the object of interaction will be powered? Are they adequate to the hospital environment in terms of not interfering on the functioning of other equipment, for instance? The concerns related to the empiric layer can also be related to the object of interaction: how many users can handle the object? Does the internet server or connection support this amount of information? How long does it take since the recognition from a given sensor takes place until a feedback is displayed? The information captured by sensors or other devices must be treated in a defined structure so that the system can use it. This structure is represented in the syntactic layer and it serves as basis for the semantic layer, informing possible ways to support meaning. The sense people make and the effective use of the solution (pragmatic layer) is the base for the social world: the changes in the daily life of stakeholders and the impacts in society. Based on this appropriation of the Semiotic Framework, we organized related works on digital technology for hospitalized children (Table 2).

Table 2. Main characteristics of related work organized in the layers of a semiotic framework structure.

We used the Principles for Affectibility (PAff) as a method for comparing the technological solutions for children at hospitals regarding their way of implicit or explicitly addressing or evoking affective responses. For this analysis we observed whether the solutions complied with each principle or not. A higher amount of principles observed in a solution should be an indicator of good Affectibility. Table 3 shows this analysis. While most of them were preoccupied in adding varied media and modes of interaction, they did not explicitly involve collaborative construction (i.e., a group of children constructing/learning with each other) nor explicitly displayed instances of child’s particular social contexts/values. Enactive systems are based on pure “informational” data. A possible way to mingle “interactional” is by using PAff. Having a system that complies with all principles is a challenging goal and the development of the socio-enactive concept is a possible path towards that goal.

Table 3. Design principles for affectibility in related work.

4.1 Discussion

One aspect that all articles shown in previous sections have in common is their concern about soothing children’s stay at hospitals. The goal is to provide relief, reduce stress and allow communication. However, this concern is rather isolated, aimed at a child, without considering its surroundings. For example, the communication supported by some of the solutions is usually one to one. At most, games are provided for group play. The concept of Socially Aware Design goes beyond considering more than one child. It is about being responsible in a broader sense, considering the society as a whole; worrying about how the technology can affect the community of users.

To focus on the development of the technology alone is a rather common practice, as the result of the analysis using Semiotic Framework indicated (Table 2). Notice that the row for ‘Social World’ has few items. In this row were considered only works that explicitly discussed social implications of the digital artefact. Most often, it is not due to lack of responsibility of developers, who do care about the wellbeing of users, that this happens. Usually, it is due to lack of resources or lack of awareness of the implications, or lack of an adequate design method. While the robots Huggable and Probo seem to represent the state of the art in terms of technology, Presence App [23] presents relevant concerns regarding the community of users and their values. The analysis using the design principles for Affectibility corroborates this result, as PAf 4 (collaborative construction) is missing in all works. While X-Safari and Doctor Giggles allow more than one player, they do not actually promote collaborative construction as children simply happen to be together in the same room. Not only children interacting together at a given moment, but also children who have been at the hospital before could add to a global socio-enactive experience. None of the works presents characteristics that would make them enactive systems. They all act in deterministic ways. Solutions for children interaction in hospital settings could not only benefit from more natural activities from enactive systems, but also they could be improved if designed for social awareness (PAf. 5) and group constructions (PAf. 4). The product of interactions from groups of users, physically present or not, could allow collective creations, which in turn could be used as new inputs for the system. The continuation of this cycle has the potential of promoting more natural, meaningful and engaging interactions. Moreover, it is important to use methods that could elicit other possible problems of the Social World. Solutions based mainly on informational approaches are usually focused on the three base layers of the Semiotic Framework (physical world, empirics, and syntactic). Interactional ones are often more concerned with the top three layers, for which meanings, intentions and social impacts are main issues. Solutions for embodied interactions for children in hospitals can benefit from both the informational and interactional approaches to design for Affectibility. The informational methods can provide for systems that use automatic recognition via sensors and other devices; the interactional approaches can inform the design that is socially aware and aims for collective creations. The development of such artefacts, keeping in mind possible social impacts and principles for affectibility, is our current work.

5 Conclusion

The design of contemporary technology based in sensors and actuators needs a methodological approach that could value the ubiquitous nature of the technology on the one side and the capacity of the artefacts to evoking affective responses in people on the other side. This article presented how the Semiotic Framework and the Principles for Affectibility were used to organize and analyze related works on technology for hospitalized children. Results point to the potential of that theoretical referential in informing about the lacks in literature making clear the challenges towards research in socio-enactive systems [6]. It is important that the design process be based on the awareness of possible social consequences that the technology might bring to the relation of users and their surrounding environment. We argue that the pervasiveness and ubiquity of systems should be further explored in systems towards the development of the concept of ‘socio-enactive’ systems. These systems should contribute to improved wellbeing for the children and communities in hospitalization contexts.