Wicked, Open, Collaborative: Why Research through Design Matters for HCI Research

2.1 HCI Design Research Today

Scientific practice follows diverging paradigms as Kuhn [18] pointed out prominently. Other than in Kuhn’s historic study of the change from the Ptolemaic system to Copernican heliocentrism the diverging paradigms of HCI exist concurrently, contending in the question, what HCI means and how it should work [15].

The first paradigmatic wave of HCI is mainly inspired by industrial engineering and ergonomics and aims to optimize the man-machine fit of interfaces. The interaction between man and machine is thereby understood as close coupling, that should be optimized in order to function well and safe. An instructive example for this kind of HCI is to research and improve the usability of ticket machines or similar, often graphical, user interfaces. Thus, the first wave of HCI is concerned with fixing specific problems that arise in interaction between, e.g. a ticket machine and its user.

This engineering approach is followed by a second wave originating from cognitive science, with a stronger emphasis on theory. It is characterized by a significant extension of the research interest to the question, what is happening in the human mind while interacting with a computer. In this paradigm, HCI is understood as information communication between entities that process this communication. The efficient design of graphical user interfaces and instruments in airplane cockpits is a prime example for this paradigm. Here, the second paradigm of HCI asks the question, how the efficiency of the information processing between user and computer can be improved.

Harrison, Tatar, & Sengers [15] identified a more recent third, phenomenological wave, consisting of approaches ranging from embodied interaction [10] to situated meaning [32] to social change [2]. These exceed the first two paradigms by emphasizing the role of situation and context of use, that might change the requirements of ‘successful’ interaction and the ways of achieving it. The understandings of HCI here is an interplay of people, interface and context. Lucy Suchman’s work at Xerox PARC is an early example: She lead a work group researching how users’ made sense of a copying machine [32]. The interaction with the machine was thereby not conceptualized as (failed) information exchange, but rather as construction and negotiation of meaning. Drawing on this example, the third wave of HCI tends to be more interested in what goes on around systems, rather than what is happening at the interface. Here, the third wave of HCI asks the question, how people appropriate technologies, and how HCI design can support those appropriations within the complexity of man-machine-interaction.

2.2 Wicked Problems in HCI: Why does Research through Design Matter?

In stark contrast to the first and second wave of HCI which tackle complicated issues of interaction, the goal of the third wave of HCI is to acknowledge and grapple the inherent complexity of the interaction between people and man-made-things. This draws on the notion, that HCI artifacts are generative and influence, even change, the circumstances in which they are employed. Paying close attention to the fact that novel HCI services and artifacts substantially alter the lives of people affected, we argue that research within third wave HCI is fundamentally concerned with problems that are messy [28], infinite [30], or even wicked [27]. It is widely acknowledged, that these classes of problems are inherent to every design activity, from urban planning [27] to HCI [33]. These problems are infinite and limitless in regards to technology, context, politics, individual use, belief, and circumstance. Because not all of these facettes can be completely explored, solutions to such wicked problems are situated and bound to the way and to the expertise designers and researchers frame the problem. As such, this kind of problems does not have a stopping rule: solutions to wicked problems are not right or wrong but rather good or bad depending on one’s framing of the initial problem. Every solution is essentially unique, because every design proposal fundamentally changes the context of the problem. And thus, there is no ultimate test of a solution to those problems, because the solution does not exist independently of the problem framing.

By designing one possible solution to a wicked problem, both the problem and the solution are defined. As such, both problem and solution exist dependent of each other, their scope and connection are the “best guess” of the designer. Overall, the designer’s beliefs, values, and decisions that lead to a specific solution and to the rejection of competing solutions frame the problem, the solution and the connections between them as the so called design space. There are rather many feasible paths within the design space that connect a specific framing of the problem with one particular framing of a solution. There are also many paths within the design space that are not feasible – but are necessary preliminary steps for understanding the design space.

This is to say that a solution to a wicked problem contextualises the space in which the problem exists. In addition, the sum of the successful and unsuccessful solutions demarcate the design space. As such, understanding the proposed solution also allows us to understand the context of the problem.

To understand the methodological contribution of Research through Design it is important to acknowledge that this specific solution is the designerly equivalent to theory in scientific practice. “The ultimate particular is a design concept of the same dignity and importance as truth in science” (Nelson & Stolterman [24]), it is a “theory nexus” [9] that manifests not only the apparent solution, but also its aesthetics, the process that led to its inception, and the encompassing design space. As such, the ultimate particular is coagulated knowledge that comprises of a specific way in which a specific problem space and a specific solution space are connected through choices in regards to technology, context, politics, use, belief, and circumstance.

Design and most of HCI are concerned with creating artifacts like devices, interfaces, services, or algorithms as solutions. Making artifacts is generative, and, depending on which particular philosophy of science one follows, it is arguably the opposite of science [26]. How then, can design research contribute to a methodologically generalizable corpus of knowledge? In order to explain RtD, the next two sections will discuss how design research operates and how design research generates theory.

2.3 How Research through Design Operates

How shall researchers engage in designing and developing HCI artifacts, when these very artifacts change the problem they have been developed to resolve? The perhaps most concise definition of Research through Design and how to tackle this conundrum has been proposed by Gaver et al: “we make things and let people try them, not to develop commercial products, but to learn both about people and new possibilities for technologies.” (Gaver http://www.datacatcher.org).

Likewise, an intentionally made intermediary – a research device – has the potential to unfold complex interactions and dependencies and thus can surface issues without the ramifications of shaping a readymade, black-boxed ‘product’ and unleashing it into the wild. Here, a research device is being developed, field tested, and analyzed in order to help the researchers understand how a specific design space unfolds. It is meant as a tool to better understand the interplay of artifacts, practices and design intention before they develop solutions for specific problems. By deliberately utilizing these research devices as intermediaries, the appropriation of these devices can be better analyzed and their ramifications can be better contained. Oftentimes, these research devices are deployed for periods of weeks to months, to let the interplay between people and these devices unfold. These field tests are usually accompanied by probe studies, participatory workshops, ethnography, interviews, and are framed as design inspiration and reflection. These modes of analysis most importantly do not hinge on a specific research question, but are deliberately kept open to understand the complex modes of interaction – paying close attention to the usage of the research devices, their appropriation, intended and unintended effects, unexpected use, and even non-use.

2.4 How Research through Design Generates Theory

Hence, Research through Design is based on two assumptions. First, the designed artifact is a theory nexus, which means, that the design artifact itself embodies theory [11]. It contains all the choices and decisions that led to its implementation. It also indirectly reveals the assumptions of its makers in regard to technology, context, and politics. As such, it also holds the beliefs and values its designers account to. It even contains the failed and not realized attempts competing solutions. But how then, shall the designed artifact speak? The second assumption is, that the designers themselves can reveal these issues by annotating these artifacts. By doing so, they can explain their aesthetic decisions, their beliefs in regards to user involvement, and their individual design process. By doing so, the knowledge about the design space that encompasses the design artifact, becomes accessible for understanding and reflecting. As such, annotating design artifacts may be the design equivalent to theory building.

This notion has been elaborated by Zimmerman [34] and Gaver [11]. Both argue, that the artifact can be annotated, either through four lenses (process, invention, relevance, extensibility) [34] or shared values (creative process, user-centeredness, developing design spaces, exploring material, attention to aesthetics) [7]. Interestingly, STS literature and empirical research follows a similar approach by reconstructing the “scripts” and theories which are embedded in (even mundane) artifacts [1], [17].

We share Gavers [7] hypothesis, that an array of annotated design examples is the core of what Research through Design can add to the HCI discourse: These “annotated portfolios” and annotated artifacts help in reflecting and understanding made things – ultimate particulars – and also the surrounding design space. As such, RtD is “theory building” in a designerly way [8], because it explicates the decisions, beliefs, circumstances the designers encountered when exploring the design space. This resurfacing of embedded theory is illuminating the aspects that surfaced during making, analyzing, describing, and understanding of this particular design space. And these annotations allow researchers and designers to understand the problem and solution space, the path in between, their interplay, and the ramifications for people and society at large.

The various documentations of the Data Catcher [6], [7], [12] are the prime example of how to annotate a specific research device. The Data Catcher is “a location-aware, tangible and embodied mobile device that displays a continuous stream of statements about its location that are drawn from a large number of data sources and which speak to sociopolitical issues.” (Boucher & Gaver [7]). It is an intermediary for exploring the design space of participation in location based big data and emerging sociopolitical issues. Its designers annotated this research device from three perspectives. First in a pictorial [6] to explicate decisions and questions that accompanied the making of the artifact and aesthetic decisions in addition to the prime research interest. The methodological approach and the results from a field trial of 130 of these Data Catchers is presented in Gaver [12], where questions regarding user involvement and user empowerment were brought to the fore. Lastly both the design process as well as the prime research questions that surfaced are presented in Boucher [7].

As such, these annotation of the Data Catcher can explicate issues that emerge from putting a specific design research device into experimental context, and allows researchers to explore design opportunities arising.

We acknowledge, that the theory building presented here is mainly concerned with making and material, user involvement and empowerment, design methods and processes in order to describe the design space. However, we wish to take this multi-faceted approach of annotation as a starting point and will offer two further lenses for preliminary exploration, we believe are oftentimes under-determined in annotating research devices: interdisciplinary research and open research agendas.

The following section presents Loaded Dice, a research device we proposed to explore the design space of smart connected things for the home together with both expert and lay co-designers. First, we will briefly present this research device and the scope of its application.

We will then contextualize RtD within HCI research on the home and will further elaborate on the use of Loaded Dice for interdisciplinary research within an open research agenda. Employing the research device in various scenarios, a variety of issues and opportunities emerged that we believe are a useful annotation of this device for its merits within an open research agenda and interdisciplinary research.

3.1 Introduction: What are Loaded Dice

The tool consists of two Arduino based, 3D-printed cubes. One of the devices is equipped with a different sensor on each of its six sides, while the other is equipped with a different actuator on each side. Whichever side is on top is active and communicates wirelessly over distance with the top side of the other cube. Loaded Dice support creativity, imagination and exploration in design processes of smart and multisensory devices and services, especially in early ideation stages.

Figure 1

The Loaded Dice with Sensor Die (left) and Actuator Die (right).

Loaded Dice (Figure 1) are a co-design tool to explore the design space of smart connected things together with co-designers. Those lay co-designers provide expertise from their everyday experience and thus valuable, qualitative input for the design of innovative smart connected things for those lifeworlds. However, the co-designers have varying levels of expertise in using and understanding technical devices, and the processes of designing them. The Loaded Dice serve the purpose to bridge the gaps between those different levels in co-designers and us as HCI researchers and enable meaningful discussions and ideation on equal terms.

3.2 Related Work: Co-Design Support Tools for Designing IoT for the Home

Bringing co-designers and experts on a more equal footing proves challenging, and there exist ample ways to bridge these gaps in either way. In order to re-annotate our artifact in this paper, we will position it among other co-design tools. Those tools employ one of two strategies to bridge the experience gap. One is reducing the technical and skill expertise required for co-designers. This is often done by employing analog support tools which aim at non-functional abstract ideation, e.g. in form of cards representing the problem and solution spaces, actors, environment and dynamics. The other strategy aims for functionality in ideation, supplanting required expertise for co-designers.

Related work as discussed here, focuses on tools supporting both expert- and lay-designers in the reflective conversation between idea exploration and idea generation for smart connected things within the IoT. It also employs either one of the two aforementioned strategies. Therefore we consider support tools as analog if they employ the strategy of non-functional, abstract ideation and digital if they provide supplanted functionality.

3.3 Design Concept: Playful Exploration

The main aim of our tool is supporting early stages of design processes, specifically ideation and exploration of smart connected products in domestic settings. The embodiment of the “Loaded Dice” is thereby reduced to a fundamental form, allowing the combination of six by six sensor / actor pairs (Figure 2). This serves our premise, that people have to actively experience functions and connections of actuators and sensors in order to gauge their effects and experiences. We deem this crucial to allow the ideation of design solutions that are novel, creative and incorporate the respective expertise of the co-designers. For that premise our support tool allows for spontaneous connection of one sensor and one actuator at a time. We chose sensors and actuators that are commercially off-the-shelf available in order to reside in the current boundaries of the IoT solution space. Thus, the translation and incorporation of prototypes into domestic applications is kept outside of the laboratory.

Figure 2

The Loaded Dice consist of these sensors and actuators.

To engage in spontaneous prototyping, the Loaded Dice had to make it as easy as possible to re-combine sensors and actuators. We did so by breaking down functionality, interactivity and appearance to elementary inputs and outputs that are open to immediate re-configuration [20].

By choosing the form of a cube we aimed to incorporate three benefits of this form into our design principle. First, by having six equal sides, a cube offers enough space for six sensors or actuators in one hull, without lending any special preference to one of them. Second, the cube form is common enough that it allows for intuitive manipulation, handling and reading. In our case this means the fact that the top side is the active one that communicates. This transports the reading and meaning of an actual die to the users. Splitting actuators and sensors in one Loaded Die each allows for quick one-to-one pairing by rotation of the cubes, experiencing sensor-actuator mappings on the fly. The devices themselves do not force a “right” combination of sensors and actuators, e.g. sound input and sound output (Figure 3). The implemented wireless communication between the Loaded Dice also enables spatial configurations involving different rooms and floors. Thus, sensed physical qualities can be transported over long distances and transmuted into different physical qualities. This way, users can use the microphone to transport sound to the actuator cube where it is translated into heat. This allows to experience new multisensory mappings and thereby help envision creative and innovative use scenarios or design fictions.

Figure 3

The Loaded Dice sense heat and transmits it as light.

3.4 Playfulness in Play: Individual Workshops

In order to explore design scenario building in context, we conducted several workshops with different participant groups. From this we want to present two cases as qualitative sample. Workshop #1 was held with visually impaired and blind students aged 13–17 with a total number of 11 participants from 8th and 9th grade [20]. Workshop #2 involved a mixed group of older adults (aged 60–79) and students (aged 20–30) for a total of 10 participants.

The workshops were held with one researcher as facilitator in their role as expert designer and groups from two to four participants. The participants were given the Loaded Dice and asked to explore the devices and discuss their functions and form. Later they were asked to reflect on problems they faced in their everyday life and imagine how sensors and actuators could solve them. If necessary the facilitator would state, that the envisioned solutions could vary both in form and functional details (e.g. type of sound) from the Loaded Dice. The sessions were concluded by an open discussion about sensor and actuator based technology in general and how participants felt about them. Thus, ending a session was an implicit communication process between facilitator and participants.

Workshop #1 was held as a one-day-workshop for two classes were the Loaded Dice would be explored by the students and compared to another ideation tool, littleBits. The classes were divided into mixed groups of two to four students. The ideation was mixed between technical inspired solutions like an orientation-system for blind people using sonic vibrations to more poetic everyday objects like a weather station using bird voices to communicate if it was dry or wet outside.

Workshop #2 consisted of four sessions with differing participants. Scenarios were ideated in relation to everyday lives of the participants. The framing question was, how sensors and actuators could augment such scenarios. Some scenarios from one session were used as basis for sparking the discussion in another session. By experimenting with the Loaded Dice, the ideation explored many different contexts such as augmenting pets, communicating emotions, or technically hacking clothes. By employing the Loaded Dice, effects such as stigmatization could be discussed. For instance, blind students from workshop #1 remarked how they disliked speaking watches and apps as they marked them as blind and thus ‘handicapped’. As such ideating sensor and actuator based solutions which were not stigmatizing the users was prominent. Also, solutions were imagined beyond technical possibility, incorporating a certain poetic component – proofing the research device to be more than a functional demonstrator. This was evident in an ideated communication scenario where plants would communicate their wants to their owners.

This section should have given the reader an impression of our co-design tool Loaded Dice and its fielding in workshops with co-designers. The next section will discuss the meaning of our artifact when viewed through two more abstract lenses: open research agenda and interdisciplinary research.

4.1 Multi-Perspectivity of HCI Work

Combining researchers and perspectives from different research fields is necessary to tackle complex problems and common to HCI work. Third Wave HCI involves especially perspectives on interaction and situation types, derived from micro-sociology, communication studies and anthropology, as well as reflections on policy and design as process, derived from Science and Technology Studies, design research and innovation studies. With that comes a rich set of methods helping to investigate users interacting with digital devices and designers interacting with users’ lifeworlds like ethnography or even outreach work towards vulnerable communities. These methods focus to understand and explain socio-technical phenomena, they are useful to grapple with the complex intertwining of cultural factors, social interaction, technology, collective sense-making, etc. But these methods come with an epistemological impediment for design: They are suited for understanding, not for making own (material) contributions in the first place. Developing technology for the smart home thus involves creative practices like projecting potential use cases, innovating shapes and functions and the engineering work of actually constructing a functioning product.

The design space of a certain wicked problem – like for example initiating social interaction in an intergenerational apartment house – changes depending which perspective and framing of the design problem is applied. We argue to distinct between disciplinary perspectives from sociology, anthropology, psychology etc. as rather reconstructive, while the aim to transform a given status by creating artifacts – typical for engineering, design and HCI – as constructive epistemology. In order to grapple with the complexity of socio-technical arrangements we argue beyond RtD to refine the distinction between reconstructive elements and constructive elements within a research and development process, as we want to show with our example.

4.2 Open Research Agendas

While interdisciplinary research in HCI is nowadays the norm – including artifact based research – methodological models to shape the processes of design and development for this multi-perspectivity remain scarce. We propose open research agendas for a nuanced view from all included parties and as chance to explicitly negotiate the problem and solution space of design proposals within HCI. This goal may sound mundane, but it is not when we have a look at the circumstances under which HCI in academia is organized: The funding of research requires a proper time framing of the undertaking. Additionally, there is a strict temporal limit to the researchers’ and designers’ work due to the end of the funding period or time for completing a qualification. This causes a paradox in planning: Project executives often have to outline a time frame before they actually get involved with the situations of use and lifeworlds of addressed users. Furthermore, the time scale of most project funding does not match the “Eigentime” [14] of involving people and building relationship to their lifeworlds e.g., [19].

A key goal of developing the “Loaded Dice” as research artifacts was, to not shape beforehand, how they will be used. In particular the definition of the problem that should be solved by the assistive interactive system we create, should carefully emerge from negotiations with users. Thus the major methodological challenge was to enable and maintain such an open-ended process, and to empirically ground decisions in data from the addressed users. A common frame to iteratively investigate in lifeworlds is the Grounded Theory Methodology [31]. This methodology proposes a research process that is reflective and explicit on its decisions on the basis of constant change between gathering and analyzing data. Furthermore the slogan “All is Data” [13] underlines, that Grounded Theory is appropriate to artifact centered methods like RtD.