research-article

Open access

Speculative Histories, Just Futures: From Counterfactual Artifacts to Counterfactual Actions

Authors:

Laura E. Forlano,

Megan K. HalpernAuthors Info & Claims

ACM Transactions on Computer-Human Interaction, Volume 30, Issue 2

Article No.: 22, Pages 1 - 37

https://doi.org/10.1145/3577212

Published: 13 April 2023 Publication History

All formats PDF

Abstract

This article engages with history as a speculative space for the purpose of critically engaging with discourses around the politics of technology in HCI. Drawing on approaches within critical design and based on evidence from two different projects, we develop an approach, counterfactual actions, that moves beyond the creation of artifacts and towards more situated, embodied, and performative engagements. In one project, Reimaging Work, we used a participatory game to engage stakeholders from social and economic justice organizations in Chicago. The other project, Future Design Studio, invited audience members at a futurist festival to create artifacts from the future and then invited improvisational actors to build worlds around them. We argue that a focus on counterfactual actions supports a more relational approach to understanding the politics of socio-technical systems and infrastructures, allowing participants to gain a meaningful understanding of the ways in which technology could be designed otherwise in line with ethics, values and social justice concerns.

1 Introduction

This article engages with history as a speculative space for the purpose of understanding human-technology relations in the service of designing different futures informed by alternative politics, ethics, values, and social justice concerns. Specifically, we build on critical, speculative and experimental design approaches in order to move from the creation of things or artifacts and towards participation in action for the purpose of public engagement with discourses around technology. We use evidence from two projects to develop an approach, counterfactual actions, which are situated, embodied, and performative modes of participatory engagement with technological futures.

Our approach has a few key components as follows, which we will elaborate in more detail throughout the article. We use the word actions deliberately to signal various contexts and meanings: (1) collective action and intervention; game play; and, performative, theatrical actions. What follows is a more detailed explanation of this approach. First, counterfactual actions, engage deeply with history yet are not necessarily constrained by historical facts. Second, we emphasized the ways in which counterfactual actions are made-up, fabulated and speculative in nature as thought-experiments, simulations, role-plays and/or gameplays created to engage with ethics, politics and social consequences of socio-technical systems in ways that can be oriented towards engaging with the past or the future. Third, rather than focusing exclusively on making artifacts, things or new technologies, counterfactual actions are ways of exploring new identities, relations and processes within the context of socio-technical systems using participatory design (PD) and community-based futuring methodologies. While the focus on making things in a workshop context may inadvertently follow the logics and linear assumptions of techno-deterministic innovation pathways, the creation of the conditions for actions such as collective action, game play, and performative actions can not be considered as a mode of infrastructuring that relies on the active engagement of participants for the purpose of building our wider capacity to think about the politics of technology as well as our agency to act and intervene in that future. Thus, by definition, our counterfactual actions are participatory. Fourth, since we are interested in the relationship between society and technology as they are enacted together in socio-technical systems our approach emphasizes relationality. Lastly, counterfactual actions engage people somatically, drawing on somaesthetics, first-person research and praxis to emphasize the affective and sensory dimensions of bodily experience. Ultimately, we argue that counterfactual actions offer ways of acting out or bringing new modes of being (along with alternative ethics and politics) into existence, even temporarily—allowing HCI researchers that are interested in participatory and community-based futuring methodologies to tap into deeper forms of latent knowledge than are possible with existing methodologies. While our ultimate goals for this research agenda are ambitious in terms of scaffolding future discussions around the politics of technology and building agency for change in the broader public, what follows are reflections from two relatively short projects that illustrate small steps towards that vision as well as the bringing together of a set of ideas to support research, pedagogy and practice along these lines.

The first project was a workshop created for the Open Society Foundations’ Future of Work Initiative. The workshop brought together labor activists, digital activities, designers, and academics to consider the future of the relationship between technology and labor first by playing a game about the past, and then by creating a prototype for the future. The second project, sparked in part by the first workshop, was part of Arizona State University's annual festival, Emerge: Artists + Scientists Redesign the Future. The Future Design Studio (FDS), a pop-up creative station at the festival, used a performance to engage with ideas submitted by the audience in the form of artifacts from the future and an “abridged user manual”. At the end of the day, a troupe of improvisational performers selected artifacts to explore through long-form improvisation. Both projects aim at provoking greater reflection on the futures of human life within the context of socio-technical systems and infrastructures, seeding advocacy groups as well as broader publics with a greater sense of agency over technology.

This article asks the following questions: How might engaging speculatively with histories and futures build agency among advocacy groups as well as the broader public? How might design facilitate new multi-stakeholder alliances that can engage politically around questions of technology innovation? How might existing design approaches be adopted to move away from the explicit focus on artifacts and things and towards relations, socio-technical systems and infrastructures? And, how might we draw on more experiential forms of participatory engagement in order to build knowledge about transformative social change?

To begin to answer some of these questions, we first discuss how HCI and related fields conceptualize ideas of the future, the future of work, and genres of speculation and fabulation. We then turn our attention toward speculation and fabulation about the past, introducing the ideas of counterfactual thinking and action. We then present two case studies, which together form a patchwork for better understanding how counterfactual thinking can help open spaces for investigating futures. Specifically, drawing on recent discussions in anthropology, patchwork ethnography refers to “ethnographic processes and protocols designed around short-term field visits, using fragmentary yet rigorous data, and other innovations that resist the fixity, holism, and certainty demanded in the publication process” [1]. A patchwork approach both challenges the dominant approach to the creation of scientific knowledge that underpins HCI research with its focus on positivity, linearity and, perhaps even, novelty itself as well as offers new possibilities for the expansion of the field to accommodate more diverse modes of thinking, making and doing. In this way, patchworking allows us to learn from and bring together ideas from multiple disparate projects conducted at different times, in different geographies, by different people and under different conditions with the funding, time and relationships that were possible at the time. These two projects were not part of a single research program or funding stream controlled by a single principal investigator, rather they were opportunities to explore possibilities for future collaboration and relationships. This bringing together of two different projects takes advantage of the realities of conducting research, celebrating what it is possible to do with the opportunities that are available through chance encounters at various career stages. As such, this article allows us to consider our previous projects in a new light given the continued and growing interest in these kinds of approaches. Finally, we draw on these cases to identify five key themes central to the counterfactual approach and we discuss the limitations and implications for future work.

1.1 Techno-visions and the Problem of the Future

HCI, like other computation and design fields, is fundamentally oriented towards “the future”, which too often refers to a universal, modernist drive towards technological progress. These utopian and optimistic promises of a better tomorrow serve to distract our attention from the situated conditions of the present and disengage our relationship to historical contexts. By continually asking only “What's next?” without a strong understanding of the entanglement of technology and humanity in the past and present, HCI and related design fields are destined to continue the sexist, racist, ableist, and colonial research agendas that dominate the field today.

A number of recent books in HCI and related fields such as anthropology, communications, design, and future studies have sought to better understand the nature of futures research and methodologies for the purpose of engaging more critically with the claims around the role of media and technology [2–4]. Such analysis is not new however, the Covid-19 pandemic has certainly prompted renewed interest in narratives around future uncertainty. At the same time as coronavirus variants spread throughout the globe, prominent technology company billionaires have embarked on competitive ego-boosting space flights paired with confident claims about the vast possibilities of new forms of travel as well as new territories for commerce. At the same time, the labor conditions of their employees continue to make headlines for their dehumanization with the slogan “We are not robots!.” In the United States, discussions about the future have combined a sense of optimism paired with the potential of technology and ideological fulfillment according to historical work by Carey, a prominent communication scholar [5]. In particular, he writes that American popular culture has a tendency to “discount the present for the future” and treat the “future as a solvent” in which as we move forward in time, we simultaneously move away from the problems of the past [5]. According to a new book by Powers about the social construction of the trends, futures and forecasting field, futures are a way in which “culture's lifespan is understood and circulated, bought and sold” for the purpose of making “meaning, and money” [6]. Much research about the future is justified by the need to control uncertainty and manage anxiety, change and disruption while, at the same time, creating market opportunities in which the “select few determine the tastes, lifestyles, and ways of being for the rest” [6]. In the field of HCI, scholars have argued for the need to “embrace uncertainty” creatively and generatively as a means of engaging more deeply with the challenges of social, political, economic and environmental change [3, 7]. Similarly, the field of anticipatory governance [8–11] encourages that suggests that we find ways to engage publics around decisions about technologies that do not yet exist.

Based on an analysis of Xerox's Palo Alto Research Center (PARC), Dourish and Bell write that like earlier techno-visions “PARC's techno tales would also become myths: they would create a way to make sense of the future that appeared simultaneously magically but also manageably. That these myths emanated from the center of Silicon Valley gave them a sense of inevitability as well. After all, if smart engineers and computer scientists say this is our future, then surely it will be true. This kind of rhetorical positioning also meant that to be skeptical of such visions was to be seen as against progress, a Luddite or worse” [12]. Based on a historical study of nanotechnology and space travel, McCray explains the ways in which “visioneers”, often privileged white male tech-enthusiasts, combined “views of the future with technical skills, experience, and research, took speculative ideas out of the hands of sci-fi writers and technological forecasters and put them on firmer ground…By inspiring o(or provoking) people, visioneering helps reveal the future as something other than some neural space that people move into without friction. Instead, it is a terrain made rough by politics, ethics, and economics as well as people's hopes and anxieties” [13].

While these perspectives seek to dismantle the naïve techno-determinism of the claims around computing, technology and the future, other recent books offer a set of methodologies that would-be futurists can use in practice [14]. Such approaches include sense making and scanning as well as creating prototypes and scenarios. But, to a large extent these methods continue to promise creative approaches to managing uncertainty while avoiding difficult political choices that they might suggest. From that perspective, in contrast to the sleek allure of techno-visions, Eveleth's book brings to life the ethical tradeoffs embodied in many claims about the future through visual comic strips that narrate these choices through the lives of an intersectional cast of characters including Black, disabled and poor [15].

Another shortcoming of much futures research is that it is typically set in the context of linear time modeled on Western, European understandings of temporality with a discrete past, present and future. Yet, futures researcher acknowledge that in many non-Western contexts, time is understood to be circular with simultaneous co-existence of past, present and future [14]. For example, the “Futures Cone”, has been critiqued for flattening the diversity of human experience and by reifying the “a singular point of ‘the present’ and implicitly embedding notions of linear progress” [16]. Mazé critiques the false linearity and typically apolitical stance that is common in the futures field, writing that: “Although the future is not empty, it is open,” [17]. While this assumption is often challenged in science fiction, with the possibility of time travel or alternate laws of physics, in our work we did not explicitly challenge notions of linear time in part due to the difficulty of designing a game that used alternative notions of time.

Technovisions (such as the imagined future of leisure) as well as more dystopian cautionary tales (such as “the robots will take our jobs”) continue to shape narratives around automation and the “future of work” in ways that make it difficult to understand the social implications and/or potential of existing technological systems. They serve to distract and confuse the public as well as advocacy organizations, often over-hyping the impact of technology while failing to acknowledge the actual harms. Both these overly utopian as well as the more dire dystopian claims are techno-deterministic in that they attribute a great deal of agency to the technologies themselves as drivers of change. In contrast our work seeks to destabilize such narratives by bolstering the role of human agency in shaping technological outcomes.

1.2 Labor History, Worker Advocacy, and the Future of Work

The field of HCI is engaged in a number of ways in the role of technology in the “future of work” including developing new platforms, applications and tools as well as supporting labor organizing, focusing on worker-centered design and working towards social justice more broadly [18, 19]. In particular, recent scholarship has focused on digital labor, crowd work such as ride-sharing, invisible labor, ghost work and emotional labor [20–23]. With respect to the gig economy, Dubal describes the ways in which work conditions that had long been abolished have returned in the form of “digital piecework” by Silicon Valley technology companies [24].

While HCI researchers are quite comfortable with developing platforms, applications and tools to support specific groups of uses, recent research has illustrated that individual technology interventions are unlikely to result in broad change without similar changes to regulatory, social and economic systems. This realization parallels developments in social science fields such as anthropology, science and technology studies and political economy, which takes broader socio-technical systems and infrastructures as their object of study. Specifically, Ekbia and Nardi have argued that a political economy approach is useful in HCI for understanding the relationship between computing and social inequality [25–27]. They argue that a political economy approach, while similar to other critical theories such as feminism, focuses on the central concern on the ways in which technologies are deeply tied to our socioeconomic systems. Specifically, a political economy approach enables HCI to engage historically, contextually, and politically. Similarly, in PD, the focus of many projects is the “infrastructuring” of relations within social systems in order to build publics around important issues and matters of concern [28, 29].

Irani and Silberman's Turkopticon project created a platform where Amazon Turk workers could exchange information about the conditions of their work [30, 31]. They argue that without more systematic, structural changes that improve conditions for workers, specific design interventions such as new technologies, applications, and platforms are unlikely to yield substantial improvements. Researchers in HCI have also studied the ways in which labor organizers use data tools, arguing local, contextual knowledge that is vital for organizing can get lost with the use of networked tools as well as how to create data-driven worker advocacy [32–34]. Lastly, HCI scholars have been advocating for a worker-centered approach includes a focus on accountability, representation, and responsibility of researchers that are interested in working on labor around issues such as: (1) mitigation of harms on workers; (2) design for “good” jobs; (3) supporting existing coalitions; (4) advocating for funding; and, (5) informing public policy [19].

2 Speculation and Fiction, Fabulation and Fabrication

In order to engage with the problem of the future outlined above as well as with the ways in which technologies might be reimagined from the perspective of worker advocacy groups and the general public, we turned to critical design including speculative design, design fiction and experiential futures as well as specific examples of prototyping the past and creating counterfactual artifacts along with fabulation and science fiction. These design practices and examples are drawn from a wide-range of disciplines including HCI, science and technology studies (STS) and the humanities.

2.1 Connecting Interdisciplinary Approaches to Critical and Generative Research

The role of research through design (RtD), practice-based research and design as knowledge-building, which dates back to the mid-1980s, signals a shift toward arguments that frame design practice as a way of thinking and enquiry rather than merely as a set of methodologies and skills for solving problems as is common in HCI, interaction design and human-centered design [35–39]. Specifically, human-centered design focuses primarily on identifying human needs for which it is possible to create appropriate solutions that can be applied in the real-world. The shift towards design as enquiry recognizes the value of more abstract, theoretical pursuits that might allow the field of design to build a body of codified knowledge in the ways that other scientific disciplines have developed. In short, it is about opening up new kinds of questions and possibilities for the field of design through reflective engagement in the design process.

More recently, in line with these developments in the field of design, scholars in a range of other fields including the social sciences and humanities have also begun to incorporate more experimental methodologies into their work through such approaches as inventive methods [40], critical technology praxis [41], critical making [42], and research-creation [43, 44]. Once again, these emergent practices are about thinking through design rather than designing per se. For example, drawing on the arts, cultural probes—often conceived of as small packets of materials that are used to inspire designers—have been used in design and HCI [45, 46] to engage specific groups of users around important issues in their communities and as boundary probes to bridge conversations among interdisciplinary groups [47].

While HCI and design have incorporated methodologies such as cultural probes, material speculation, making and prototyping as ways of using objects and things as ways of thinking, futuring and storytelling, it is important to note that there are a number of critiques of the widespread proliferation of these approaches that are relevant to consider. For example, Lilly Irani has argued that many of the prototypes and “demos” that are created at hackathons never get built and that what is produced is actually an “entrepreneurial citizenship celebrated in transnational cultures that orient toward Silicon Valley for models of social change,” [48]. Similarly, Fred Turner has traced the ways in which Silicon Valley's enthusiasm around prototypes can be traced to Puritan teleology and scientific progress [49]. Lindtner et al. [50] discuss the ways in which amateur DIY making communities in China mobilize visions of ubiquitous computing in the development of products at hackerspaces, hardware start-ups and incubators in order to reveal the ways in which these prototypes, components, and materials are not neutral emblems of innovation but rather imbued with national politics.

2.2 Sensing the Future through Speculative Storytelling, Design Fiction, and Feminist Fabulation

Similarly, design futures [51, 52], speculative design [53], and design fiction [54] approach design as a practice that serves to pose critical questions about alternative possible futures rather than to create solutions to problems as is the case with human-centered design. Since HCI (and design research more broadly) are still largely dominated by a more positivist scientific traditions, many of these more speculative modes are still not well understood, often relegated to the status of “art” or “craft” rather than computing or design. One of the reasons for this exclusion is the lack of well understood evaluation methods around speculative design. Put simply, how do we know when it is done well? Our work thrives at these interdisciplinary intersections that seek to complicate the categories of what counts as a scientific knowledge practice as well as what makes an exciting creative practice. The fields of HCI and interaction design have been slowly incorporating some of these methods into their practice as evidenced by workshops and articles at major conferences including CHI, DIS, NordiCHI, and CSCW in the last few years. As such, we believe that it is important to offer a brief review of that literature here in order to continue to expand the understanding of such practices. In addition, since there are relatively few texts that outline how to do speculative work; instead, it is essential to take inspiration from creative domains such as art, film, and literature.

Speculative design and design fiction have been applied to projects on a variety of topics including awareness about sustainability [55] among other topics. While there are many distinct but related design practices including experiential futures, design futures, and speculative fiction, they share a similarity in that:

Most design futures strive to create a rich, textured, often first-person immersion in a credible alternate world through the use of multiple media and storytelling techniques. The best examples also seek to evoke the everyday richness of life in a “thick” way, going beyond the obvious layers to explore more subtle “scents and sounds” of an alternative future in more emotional, evocative ways, [51].

Furthermore, drawing on the fields of literary theory and semiotics, Markussen and Knutz argue for a poetics of design fiction as a mode of enquiry and knowledge-building that “Would consist in giving a formal account of, for instance, various techniques for prototyping possible futures, the role of utopias and dystopias in design research experiments and the types of knowledge that may result from practicing design fiction” [56].

In order to learn more about the ways in which design fictions might count as knowledge-building rather than artistic practice as is common with RtD, Blythe created a project in which scholars participated in writing design fictions about possible research projects:

The fictions take the form of “imaginary abstracts” which summarize findings of articles that have not been written about prototypes that do not exist. It is argued that framing concept designs as fictional studies can provide a space for research focused critique and development [57].

Along these lines, Haraway's notion of “speculative fabulation” [58] is also relevant here in that she discusses the ways in which “worlding” happens in part due to material artifacts. Specifically, she writes, “These knowledge-making and world-making fields inform a craft that for me is relentlessly replete with organic and inorganic critters and stories, in their thick material and narrative tissues. The tight coupling of writing and research—where both terms require the factual, fictional, and fabulated; where both terms are materialized in fiction and scholarship—seems to me to be built into SF's techno- organic, polyglot, polymorphic wiring diagrams.” Furthermore, she draws on Marilyn Strathern's ethnographic work on gender, elaborating that “It matters what matters we use to think other matters with; it matters what stories we tell to tell other stories with; it matters what knots knot knots, what thoughts think thoughts, what ties tie ties. It matters what stories make worlds, what worlds make stories.”

It is important to note that, for the most part, in the speculative approaches mentioned above, it is designers and scholars that are doing the majority of the creative and imaginative work that is required to bring alternative possible futures to life through storytelling and the creation of artifacts. Another strand of futuring, which we describe in the next section, brings together PD approaches together with speculative approaches. Since, for the most part, these practices come from different traditions (i.e., PD vs. the industrial design lineage of speculative design), there are many projects that are participatory and not speculative and also many projects that are speculative but not participatory. For us, paying attention to these diverse lineages is important for the development of our own approach.

2.3 Public and Community Engagement in Speculative Futures

One of the most common critiques of speculative design approaches is that they are rarely used for the purpose of public engagement. For example, DiSalvo argues that these interventions are usually situated in exhibitions rather than used for the purpose of social and public engagement [59]. Beyond the making of products and services, DiSalvo et al. [60] discuss the idea that designers should design for and about matters of concern [61]. They (along with Binder et al. [62]) are moving HCI into a space in which it engages more directly with publics and creates objects that are meant as forms of public engagement.

A public orientation takes a step back from the common design imperatives of providing solutions or initiating change. Adopting a public orientation in HCI sets the articulation of issues and giving form to problematic situations as the primary design objective—the purpose of design becomes to identify the qualities and factors of a condition and make those qualities, factors and their relations experientially accessible. [60] Design interventions that express matters of concern, convey lived experience, perceived consequences and desired futures [60].

While embracing public engagement is a new direction for HCI, science communication has been working on engagement for two decades [63–65]. Public engagement with science and technology (PEST) has been done in the fields of science communication and science studies. The public engagement paradigm has been largely associated with citizen juries and consensus conferences [66, 67]. While there have been many critiques of the voracity and sincerity with which policy makers and scientists have embraced these forms of engagement [68], they remain an important step in fostering the relationship between science, publics, and governments. However, such endeavors often privilege those who are already somewhat knowledgeable in science or technology or who are well versed in logical discourse as a form of deliberation. Davies et al. [9] build, in part, on feminist critiques of deliberation [69] that suggest traditional forms play to, and even hide, traditional power structures. Logical discourse leaves out many voices, and does not often provide structures for creative solutions, but rather, occur as debates (often with foregone conclusions) about existing solutions or policies. Not all knowledge or understandings of science exist and can be explained as logical discourse. To offer the public other ways of expressing multiple knowledge about science, Davies, and colleagues suggest seeking opportunities to engage users through material, affective, narrative, and other means of opening conversations.

These intersections between speculative and participatory approaches have been described as “collaborative speculation” [70], “participatory speculation” [71], and “speculative civics” [72]. Light describes the ways in which speculative artifacts and processes can work together to “seed” omni-directional imaginations of the future as “embedded forms of engagement that promote transformative agency” in workshops that are meant to expand democratic participation in futuring. Gerber emphasizes the need for collaborative imagination around public safety in order to move away from individualistic visions because “enacting radical systemic change is a long, multi-generational process”. Another related project on autonomous distributed energy systems uses a series of speculative objects in combination with performative engagements and debate as a way of engaging publics in questions about the value (and values), control and ownership embedded in different business models for home appliances as well as energy systems more broadly [73].

As the previous section on public engagement makes clear, speculative design, design fiction, and design futures have been criticized for being overwhelmingly white, male, able-bodied, wealthy and Western fantasies of utopia and dystopia—the exclusive domain of designers science fiction authors, and other experts [6, 74, 75]. As a result, there have been a number of projects that engage directly with communities—Black and indigenous people—for the purpose of imagining collective alternatives to normative narratives around futures using participatory workshops and games [76–79]. These workshops often produce images, sketches, zines, and artifacts of the imagined futures. Another approach, speculative enactments more directly engages with the improvised and unscripted performance of imagined futures that “constitute an effort to meaningfully enact elements of possible futures with participants” by creating “conditions for genuine social interactions to unfold, amidst elements of speculation” through “stage setting and intervening in existing everyday routines” [80].

2.3.1 Prototyping the Past.

In order to understand the projects that we present below, one of which engages directly with the history of technology and work, it is important to consider the ways in which the field of HCI has engaged with history as a resource for thinking about and designing future technologies. Though nascent, HCI has already begun to develop a body of work that engages with history [81] in a number of ways including archival research and historical methods; critical histories; design methods that engage with historical precedent; and, research on the social implications of design. One approach that leverages historical knowledge, Jentery Sayers's “prototyping the past” integrates rapid prototyping with computer numerical control (CNC) technology for the purpose of “remaking technologies that no longer function, no longer exist, or may have only existed as fictions, illustrations, or one-offs” [82]. This approach understands the ways in which both the meanings as well as the materials of technologies are intertwined and historically situated, integrating “history into the social, cultural, and ethical trajectories of design” by making objects that fill “absences in the historical record” and offering alternative explanations and relations [82]. Another example “humanistic engineering” collaboratively reverse engineers historical artifacts including wireless technologies, electronic musical instruments and analog computing as a mode of “doing history” in order to “stage situations that allow us to experience a specific process, moment, or flow” [83]. This approach is highly “exploratory, experiential and performative” and typically shared through workshops and documented on websites as well as in code repositories such as GitHub. One key feature of this research is the focus on small-scale technologies that can be more easily built and experimented with in order to “reenact and reimagine” historical technologies as well as their social, cultural and material contexts.

Another relevant example is Kat Jungnickel's “Bikes and Bloomers” project brings to life historical Victorian women's bicycle-wear in order to better understand the role that women played as inventors of specialized garments [84]. By remaking these garments as well as wearing them, Jungnickel is able to use design as a mode of inquiry that allows for a deeper engagement in the ways in which bicycles allowed for new modes of transportation while at the same time shaping and shifting social norms, cultures of use and the possibilities of clothing as expressions of what it meant to be a women during that time period. By introducing this example into the HCI field, we can ask different questions about the nature of computing, textiles, labor, and gender.

Similarly, Bjorn and Rosner's Atari Women project documents the overlooked contributions of women in the field of video games beginning in the 1970s in order to use the politics of design to challenge historical understandings [85]. The project engages both with historical research as well as with the use of design to make a series of artifacts using the Atari aesthetics in order to tell the stories of women in the computer gaming industry. Their approach, “intertextual design” is unique in that it “uses esthetic and discursive material from a particular historical era as design inspiration for a contemporary design practice, seeking to change the future by re-working the past” [85]. Along these lines, the Making Core Memory project investigates the work that women did to hand-weave early forms of computer information storage known as “core memory” [86]. This project illustrates the ways in which craft and computing are deeply entangled, creating a feminist design intervention in the form of an electronic quilt.

Lastly, recent work on “infrastructural speculations” takes into account “complex and long-lived relationships of technologies with broader systems, beyond moments of immediate invention and design” [87]. Such approaches focus on “lifeworlds” within speculative design in order to more clearly focus on social, cultural, political, economic, and environmental concerns. Building on theory from science and technology studies, researchers present a series of design tactics that situate speculative design artifacts in multiple worlds and ecologies, from multiple perspectives and with multiple stakeholders. In particular, history is considered in two ways: (1) as a source of aesthetics, practices and technologies; and, (2) as a site for the design of failed technologies from the perspective of the future. These examples illustrate the ways in which historical research is being engaged experimentally in the HCI field as well as the ways in which it has value for centering the narratives of previously marginalized or vulnerable groups.

A deeper engagement with history will attune the HCI field to the ways in which the nature of what it means to be human has been defined over time and how that might shape computing and design in the future. For example, for the last four hundred years, humanity in the HCI field has been defined according to a rational, liberal notion of a discrete individual subject—often, male, white and able-bodied. Currently, there are many calls to unsettle, decenter and destabilize these notions from a variety of perspectives including posthumanism, actor-network theory, object-oriented ontology and transhumanism. Many of these scholars argue for a more-than-human understanding of the ways in which we are in relation with both technological systems as well as environmental systems. However, research in gender studies, Black studies, critical disability studies and indigenous studies has argued that these attempts to displace the human have only served to further colonize scholarly discourse. Instead, they seek to re-invent the notion of what it means to be human around a kind of radical humanism based on intersectional feminist approaches such as design justice [88–90]. While such discussions are too vast and complex to go into detail in this article, they are one clear example of why historical context is essential for the development of the HCI field.

2.3.2 Counterfactual Thinking, Counterfactual Artifacts, and Counterfactual Actions.

In conversation with these varied approaches to integrating design, technology and history in the field of HCI and related areas, our project engages with history as a speculative and generative space for the creation of counterfactual artifacts. Counterfactual thinking, and the artifacts that result from exploring the counterfactual, are ways of offering new opportunities for engaging with the world. It is a generative process that invites imagined accounts, artifacts, and narratives that might exist within the realities of a particular historical, contemporary, or future context. In other words, creating space for imagined aspects of real circumstances provides new ways of understanding and making sense of those circumstances. Counterfactual thinking can be found in many experimental design approaches that draw on design fiction, speculative design, PD, and science fiction.

In HCI, counterfactual artifacts have been used in order to interrogate the relationships between human and nonhumans. Drawing on possible worlds theory in literature and counterfactual artifacts, Wakkary et al. [91] introduce the concept of material speculation, “which uses physical design artifacts to generate possibilities to reason upon.” Specifically, the counterfactual artifact:

Opens up speculation on the artifact but on its conditions as well. When encountering a material speculation, potential reasoning would include not only “what is this artifact” but also “what are the conditions for its existence” (e.g., including the systemic, infrastructural, behavioral, ideological, political, economic, and moral). Material speculation probes the desirability of the truth condition of the proposition and the conditions bound to it [37:101].

Specifically, Oogjes and Wakkary use “Videos of Things” in order to speculate, anticipate, and synthesize the ways in which technologies mediate relationships [92]. In another example, Morse Things, researchers use material speculation to design counterfactual artifacts, which are defined as “a realized functioning product or system that intentionally contradicts what would normally be considered logical to create given the norms of design and design products” [93]. In the case of Morse Things, researchers study Internet of Things objects in order to understand the ways in which our relationships with digital things unfold over time. The use of counterfactual artifacts is a way of “doing philosophy through things” using an RtD approach in line with post phenomenological commitments, which include a focus on empirical work, studying the nature of human-technology relations and technological mediation [94]. The purpose of these experimental approaches is to ask new questions about the nature of human-technology relations rather than focusing on a functional approach to user research and/or interaction design predicated on the central role of the user, their needs and potential solutions. With respect to postphenomenological approaches, the Tilting Bowl project investigates human-technology relations through the philosophical concepts of embodiment, alterity, hermeneutic, or as a background relation in order to give greater depth to notions of interaction design [95].

Within PD, “counterfactual scripting” has been developed in order to more accurately engage with the ways in which design processes can embrace “a more pluralistic view about the past and future” based on a case study of participatory spatial planning [96]. According to Huybrechts and Hendriks, counterfactual history or “what if-history” draws on an analytical philosophical approach that integrates literary theory with history for the purpose of making arguments about alternative possible worlds. In their project, they created “minimal rewrites” of significant historical turning points in a city in Belgium in order to project future possibilities. These rewrites were prototyped in the form of a fictional newspaper at a PD workshop. They argue that the use of counterfactual scripting allowed the project to be more inclusive by bringing publics together around important issues through the use of situated design interventions.

Another example of counterfactual thinking is in the field of science fiction. One well-known example is Philip K. Dick's 1962 alternate history book The Man in the High Castle, the post-World War II United States has been divided between Germany and Japan [97]. The book's original subtitle is telling: “an electrifying novel of our world as it might have been.” As an example of this alternative history, in the opening scene of the second season of the Amazon TV series by the same title, a teen boy in uniform with a Nazi patch on his armband gets off a bus in front of Fritz Julius Kuhn High School in Long Island, New York. According to United States Federal Bureau of Investigation records, Kuhn was a naturalized pro-Nazi leader who became head of the German American Bund in 1937 and was deported back to Germany in 1945.¹

While this specific scene does not appear in the book, it is an example of the ways in which historical “facts” become the material of imagined “fictions” that make up possible alternative worlds. Furthermore, these fictions often re-emerge in visions of scientific and technological progress and innovation toward “the future”, which are deployed by experts, politicians and others. The lines between fact and fiction are, and have always been blurred, regardless of the current political climate of “alternative facts”.

In the original book, Robert Childan is dining at the home of a young Japanese couple, Betty and Paul Kasoura, in San Francisco, living in the divided United States, when they encounter an “alternative present” in which “Germany and Japan lost the war” in a banned fiction book called The Grasshopper Lies Heavy. This encounter prompts the characters to ponder the role of fiction, science, futures and alternative possible presents as well as revealing their own beliefs and biases. Would the world be better or worse off? Which nations, groups or individuals would hold power? Whose values, ethics or cultures were worthy of preservation and why?

Dick's telling of an alternate history in this novel is compelling precisely because of the ways in which it opens up the past as a generative and imaginative space rather than one dominated by “facts” waiting to be discovered. This approach invites a critical reflection on past events as well as a means of considering alternative possible futures by asking “How might things be otherwise?”, a question that is particularly relevant to scholars and, in particular, historians of science and technology. Such alternative, or fictional, histories create rich opportunities for scholars to think critically about the histories of science and technology and as well as the possible ways of doing historical work about science and technology.

3 The Workshops

Below we illustrate two projects that have allowed us to develop the counterfactual actions approach—a game-based workshop and a performance-based workshop—in order to speculatively intervene in histories and futures of human-technology relations in conversation with the concepts that we introduced above. Both projects draw on speculative methods in combination with public engagement in discourses around the future of technologies. The game-based workshop was an invitation-only or closed workshop for labor activists, designers, and scholars while the second was an open activity that attracted families at a festival-style event. In both cases, the majority of the participants were unfamiliar with the use of speculative approaches for thinking about the social implications of technology. We believe that situated, contextualized, embodied, experiential, and performative approaches hold great promise in that they leverage somaesthetic forms of knowledge and first-person research.

3.1 Reimagining Work

The first project that we would like to analyze is Reimagining Work, which was part of the Open Society Foundation's Future of Work Inquiry, a two-year learning project that “engaged a diverse group of academics, activists, economists, and corporate, governmental, labor, and technology leaders to address some of the biggest questions about the transformation of work and what work might look like in 30 years” [98]. The Open Society Foundations's inquiry was predicated on the assumption that, for better or worse, new technologies will change the nature of work in the future. Their hope was to intervene in these changes to help protect workers from being left out or marginalized. As in many cases of the adoption of new technologies, we believed that an examination of the past might help to contextualize and understand our current thinking about the future. Thus, this project asks the following questions: How might engaging with the historical contexts and situations around technology build agency among worker advocacy organizations? How might history be mobilized for the purpose of experimentation with alternative possible futures? How might design facilitate new alliances across scholars, technologists and worker advocacy organizations? How might fiction offer modes of experimenting that can resist the constraints of everyday advocacy work?

In order to answer these questions, we developed a four-hour workshop in order to engage stakeholders from the social and economic justice community in Chicago. Prior to the workshop, as part of our ongoing research on the topic, we conducted qualitative research on the future of work using the following methodologies: the analysis of over 275 mainstream media articles; fourteen semi-structured, hour-long qualitative interviews; and, the creation of a PD workshop. We conducted interviews with activists from organizations that served diverse communities of workers, ranging from telecommunications workers to domestic workers. While the majority of the activists and organizers were based in Chicago; two were based elsewhere. Eleven of these interviews were conducted prior to the workshop in order to inform its design and three were conducted after the workshop in order to evaluate its results.

These interviews focused on three concepts related to the work organizers were doing in their respective organizations, and the attitudes they thought they were seeing from their organizations and constituents. The first line of questioning focused on their perceptions of the current and future state of labor practices in the United States and included questions about challenges and opportunities facing the labor force and their organizations. For the second theme, which was technology, interviewees were asked about their own relationship to technology, the impact they felt it had on their constituents, and work they were doing to manage that impact. Finally, interviewees were asked about how their organizations planned for the future, and how they and their constituents talked about the future. The purpose of these interviews was to gain a better understanding of pressing issues related to the role of technology in the future of work as well as the kinds of philosophies and values that shaped the work of labor activists in order to inform the design of a workshop. Another reason for the interviews was to reach out to labor activists in order to build relationships with individuals that might be interested in participating in the workshop as well as to get recommendations of additional labor activists and organizations that might want to be interviewed or participate in the workshop. This outreach and interview phase was important to ensure that we had a wide range of groups represented including those that worked on behalf of women, immigrants, youth, formerly incarcerated people, African-Americans, and Latinos in the Chicago area.

The purpose of the workshop was: (a) fostering the examination of new and emerging technologies with an historical lens; (b) using the tools of critical play, speculative design, and reflective practice to engage activist communities and designers in meaningful collaborations, and (c) developing our understanding of related concepts in HCI including prototyping the past, counterfactual artifacts and material speculation. The workshop was designed to help stakeholders examine paradigmatic shifts in labor throughout history to better understand the current changes impacting the labor force. We set out to devise a way to build on various stakeholders’ (such as labor activists, tech activists, designers, historians, and other scholars) understandings of the past by creating an open, playful forum, that, like the cultural probes described above, would help the players express their ideas and beliefs about the relationship between labor and technology.

We organized the workshop into two sections. The first, a game, was meant to help participants examine some of the ways emerging technologies transformed work throughout history and to begin to imagine how they might do so in the future. The second part of the workshop, the prototyping activity, was meant to provide opportunities for them to develop creative responses to some of the challenges. Discussion throughout the day helped participants reflect on their experiences and to suggest future directions for the work.

Approximately twenty-five people including labor activists (8), technology activists (2), designers (9), funders (3), and scholars (3) attended the workshop. Four of the workshop participants were interviewed as part of the project—two before the workshop and two after the workshop. Workshop participants were not paid. One of the authors participated directly in the workshop while the other primarily facilitated and observed. The workshop had two parts. The first part enlisted participants in a board game designed by the authors using critical game theory [99, 100] and reflective design practices like cultural probes [45, 46]. The game was designed for up to four teams of two to four players. Two games were played in tandem with one of the authors participating and observing each game, while a graduate student observed and reported to the authors her impressions of game play. The games each had twelve players total, with three players on each team. One of the paper's authors [removed for review] participated on a team while the other [removed for review] observed and answered questions (she was the thirteenth player at the table). The game was an exercise in thinking speculatively about both the past and the future and it represented a timeline that spanned five eras, beginning with ancient Greece and Egypt, and ending in the year 2050. Each space on the board was within an era and featured a specific point of view. For example, a team might land in “Era 2”, on a space labeled “Child factory worker” and might draw a card that asks them to create a list of demands. Or later in the game, they might land on a space that identifies them as a robot worker, and tasks them with sending a postcard from their vacation.

The game allowed workshop participants to explore historical and present technologies, socio-economic conditions, and labor realities in order to open up discussions around the way in which technologies shaped and were shaped by social, economic, political and cultural contexts. It also allowed labor activists to collaborate with scholars, designers and technologists around the creation of counterfactual histories that might allow for alternate relationships, outcomes and possibilities that might benefit workers. However, we were specifically interested in how this engagement might allow for the design of future technologies that might embed the philosophies of labor activists. As a result, the second part of the workshop tasked groups to work together to design an object, prototype, experiment, or platform that drew on the most unexpected or counterintuitive moments during the game play. In order to do this, participants reviewed the ideas that they had come up with as part of imagining counterfactual histories as well as how these ideas expressed their values and used them as a basis for the creation of the prototypes. Groups were asked to specify who they were designing for including demographic characteristics such as race, class, gender, economic status, and industry.

Each team from Game A joined with a team from Game B to complete the prototype. Each of the four prototype teams had six members, and each team had forty-five minutes to create their prototype using materials that were in the room. We provided low-fidelity supplies such as cardboard boxes, blue tape, markers, and twine, to encourage the groups to create objects that more closely resembled sketches than polished prototypes. To better understand the impact of the workshop, we paused between activities and at the end of the workshop to ask questions and invite reflections on participants’ experiences. These brief focus groups were recorded and analyzed along with the interviews that guided the development of the workshop.

The game, which spanned 3,000 years in history, invited participants to invent counterfactual actions—such as worker campaigns, schedules and uniforms—around the intersection of technology and labor throughout human history. The purpose of using history as a speculative space was to support a kind of long-term thinking by taking participants out of their day-to-day constraints and realities, which are structured by short-term demands, deadlines and campaigns. By inviting participants to create fictional artifacts and prototypes as part of the game play, participants could experience a sense of agency around what it might mean to engage in the designing technologies that might incorporate their values, ethics and concerns. In this way, the game-based workshop explored ways of engaging social and economic justice organizations in the PEST.

As we designed the workshop, we focused on what was similar and what was different about past changes in the labor force with changes that we see as looming factors in the next major shift in the socio-technical relationships between technology, labor and the future of work. We were interested in moving beyond the utopian and dystopian narratives that so frequently frame discussions about the future of work to open a discussion about possible new directions for labor advocates. To situate current shifts in the future of work related to the introduction of new technologies, we devised a game that would offer opportunities to consider past shifts in the relationship between labor and technology. The game was developed to help participants explore the past without having to become experts on the historical relationships between technology and work. This focus on the past was tricky because we did not want to spend the time “educating” the participants about what the past was like. Instead, we wanted to give them some information and let them rely on their own knowledge as well as their imaginations. So, the tasks had to be fairly whimsical. The game asked teams of players to imagine counterfactual histories and to complete a task to express something about those histories during each turn. The goal was to reach the end of the board first, or, as we explained in the rules of the game, to get to the future first. The board itself was a timeline and was approximately five feet long (See Figure 1). It was divided into five eras: Ancient Egypt and Greece; the era of steam power in Europe in the 18^th and 19^th centuries; the assembly line in the early to mid 20^th century in the US and England, the current era of automation, and a future era around 2050.

Fig. 1.

Upon entering a new historical era on the board, each team would be able to open a dossier on that time period. The dossiers were meant not to be a comprehensive history lesson, but to provide some sense of the time, and some inspirational material for the task. The dossiers contained photographs and drawings, advertisements, graphical information, quotes, and short texts about labor and technology in that era. For example, the second era included a timetable of Lowell's Mills, a photo of children working in a mill, an advertisement, and an excerpt from Lord Ashley's 1942 report on mining conditions [101] (See Figure 2). Each era had six spaces that identified an individual from that time period. For example, in the first era, a team might land on a pyramid builder in Ancient Egypt, or on a domestic slave in Greece. Once a team landed on a space (through a roll of the die), they drew an “Action Card” which asked them to collaboratively develop a response or solution.

Fig. 2.

An example of a turn might be to draw the card “plan a collective action with fellow workers” while on a space marked “pyramid builder” in Ancient Egypt on the board. Other “Action Cards” included the following: Create a list of demands; Draft a memo from the head of the company; Create an image that represents the relationship between workers and technology; Draw a mechanism by which labor could be measured. Describe the mechanism; Create an alternative way of engaging goods and services; Create a job application; Create a schedule for regular time away from work or leisure time; Create a corporate logo; Design a worker's uniform; Design the mechanisms by which a worker's schedule is determined; Describe a meal break or lunch break; Describe how a worker travels to and from work; Send a postcard from a workers vacation; Draw one instance of a worker interacting with a technology from the time period and industry; Develop a new purpose for a technology that exists in this time period; and, Bring a technology from another time period into this one and adapt it for a new use. The aim of the action cards was for teams to use their imaginations and to develop alternatives to the historical, contemporary, and future scenarios, not necessarily to accurately represent history.

Each team was equipped with “Action Sheets” to use as worksheets for their task. These sheets would be an active site for speculation about counterfactual histories, as participants recorded their answer to the task provided on their “Action Card”, which are understood as “What If” questions. For example, the “Action Sheets” asked participants to think about the following questions: Who are the workers? Who do they work for? When is this happening? Where is this happening? Participants were asked to think of an idea, describe the idea and think through the implications of their idea.

The teams had 6 to 10 minutes to complete their task. They had, at their disposal, the information from the dossier of the era in which they had landed, as well as their own imaginations. We emphasized our hope that the game would help explore counterfactual histories to bring to life new and interesting dimensions of historical labor struggles. As such, we wanted the teams to share their answers with each other. Though it proved to be time consuming, the opportunity to learn what prompts other teams received and how they responded to them enriched the game, as well as the discussion and activities that followed.

A brief discussion after gameplay revealed what participants felt they gained from participating. There was an obvious value in new information about historical labor struggles. Many participants marveled at evidence of strikes in Ancient Egypt. They also enjoyed the way some of the primary sources in the packets provided new insights. For example, many were interested in thinking about time as a regulatory technology.

Another participant suggested that the counterfactual nature of the game promoted creativity as well as critical thinking about the future.

Listening to some of the answers during the game gave a nice angle on how to use existing technologies or current technologies from those times in a subversive way. Just to imagine a coal rail as a channel of dissemination of worker information, that was great. It really makes me think hard about what we're doing—what I'm doing in my industry—what ways to kind of use that in the same way.

The counterfactual histories helped some of them feel that they could stretch their imaginations about the way we use current technologies for activism, while they helped others reflect on the constructed nature of history. For example:

I really liked the term counterfactual. It was humorous and fun but also helps break open some more creative energy. And there are counter histories. So it's sort of thinking about what history are we paying attention to. Are we looking at the social history of conditions of work? Are we looking at industrial history and new forms of production and celebration of technology? So I think maybe it's about just being critical about what are we looking at as history, what constitutes history, and that's what's perhaps helpful. Are we taking inspiration from this history or from this history when we're thinking about current conditions?

Still others found a sense of solidarity with workers and labor activists throughout history.

There was something kind of empowering about seeing that throughout all of these ages, you know, it is a process of workers responding to the conditions that they find themselves in, and even in our little team of two or three people, in some small way we were creatively trying to respond to a challenge of our work situation.

Finally, there was comfort in the symmetry between what we don't know about the past and can't predict about the future.

Still so much in the past that we don't know there's that we don't know how it exists, like Stonehenge. And then there's so much in the future that we don't know what it will look like. So we still sit between these two worlds of not having all the answers even of things that have already occurred. And then by giving ourselves permission and being okay of not being in control of what the future might look like.

The conversation also served as a platform for the group to provide feedback for future iterations of the game. These focused largely on the length of the game and the time investment required, though alternative structures and formats were also discussed. Some participants suggested workers would find value in playing the game; however, gameplay was over two hours, which, for a worker would likely be two uncompensated hours.

The game and discussion were followed by a prototyping activity, for which we combined groups from each of the two parallel games. We gave them the following instructions:

Review your action sheets

•

What was the most unexpected or counterintuitive thing you came up with during the game?

•

How can that be relevant today?

Design an object, prototype, experiment, or platform

•

Who are you designing for (race, class, gender, economic status, industry)?

•

Represent your idea using the materials in the room

Each group came up with a unique design.

Relationship Model
Model of relationship between people and technology. Developed from one of the action cards (presumably “Create an image that represents the relationship between workers and technology.” As they described their model, they took it from its base and tried to stand it up. It would not stand on its own, which, they said was the point. Having them all “requires us all to be mindful and observant, or the game falls into the danger zone.”

LSCU
This group designed the “Little Sister Counter-surveillance System,” (LSCU, or Little Sister C's U) a system that monitors employers (See Figure 3(a) and (b)). This group put “sensors” around the room (and even on participants’ and organizers’ backs) to monitor the working and environmental conditions. The group envisioned LSCU as a way of warning workers when their physical and environmental conditions were sub-standard, or when they were being monitored without their knowledge. It was also meant to eliminate issues like wage-theft by keeping employers accountable.

Fig. 3.

Healthcare Robot
This healthcare worker's assistant robot (service robot) was designed to do some of the tasks associated with healthcare practitioners like nurses and aides. The robot would draw blood and do vitals while the worker was able to spend time speaking with the patient. The aims were to facilitate a more personal experience for the patient while helping support the worker.

Artificial Emotional Intelligence
The last team designed an artificial emotional intelligence that worked with wearables. The team suggested that using the device would prevent awkwardness and accidental offenses.

In most cases, the groups began with discussion, but the discussion led to more discussion rather than solutions. Organizers intervened and suggested that groups begin building something rather than continuing to deliberate. This prompted several groups to move forward without specific plans and to describe what they were doing as they were doing it. This act of making as a way of reflecting was fascinating and an important finding from this work; it planted the seeds for the next project. How could we take this kind of experience, in which the material nature of a design process sparked thought and reflection rather than a new product? How could it apply to a broader audience?

3.1.1 Contributions and Hopes.

The primary purpose of our project was to challenge the uncritical technovisions, myths and sociotechnical imaginaries around discussions about the future of work in order to raise the question of “How might it be otherwise?”. We sought to introduce—albeit in a less didactic way through a game—notions of the social shaping of technology and the ways in which human-technology relations have developed throughout history. As a result of this design intervention, we aimed at setting the conditions for building agency among advocacy organizations around the possibility that technologies could be designed differently. We also wanted to facilitate new connections between advocacy organizations, scholars (including historians of science and technology) and technologists for the purpose of infrastructuring new publics with a focus on the ways in which human-technology relations were being discussed and designed with respect to the future of work.

Through the game, we wanted to move beyond rational, verbal debates on issues and towards more playful, experimental, creative engagements and actions. In particular, we were interested in ways of moving discussions, concerns and questions about existing and future technologies into an exploratory space. By focusing on a historical context, we were able to remove the everyday constraints that faced worker advocacy organizations. These constraints included limited time, short temporal horizons and lack of resources in addition to the pragmatic and policy-oriented nature of their work. By using a game that was organized around specific historical contexts, we sought to create a space for participants to explore counterfactual actions that allowed for situated, experienced, embodied and performative ways of exploring their expertise and imagination. An alternate politics around the future of work requires new visions, myths and narratives around the role of technology. Our project sought to use history in order to seed these new visions (while also remaining aware of the frictions in human-technology relations).

3.1.2 Reflections and Frictions.

Despite our ambitions for the project, and what we believe was a truly enjoyable and valuable workshop based on our follow-up interviews, we would like to highlight a number of challenges: (1) establishing trust; (2) awareness of speculative approaches; (3) political orientation of the project; (4) potential for long-term impact; and, (5) lack of ties between social and economic justice organization and civic technology organizations. First, as academic design researchers, we faced difficulties in building trusting relationships with the economic and social justice organizations. In advocacy work, it is often necessary to establish credibility, for example, through prior work in grassroots activism. Second, while speculative approaches are not new, at the time that we conducted the project, we were not aware of any similar workshops or events that engaged advocacy organizations working on social and economic justice. Thus, experimental methodologies such as speculation, games and fiction were not familiar to the stakeholders that we engaged in the project. While the use of experimental design approaches to engage in discourses around automaton and the future of work is an enjoyable, hands-on mode of engagement, the implications of such methods in public policy are still unclear. Third, while we believed that we had a clear orientation towards the value of creating alternatives to existing technovisions around the future of work (specifically, those that would benefit the most vulnerable workers), we did not necessarily make these as explicit as would have been necessary to communicate our politics. As a result, in the final hour of the workshop, participants asked clarifying questions about the nature and purpose of our work. This brings to mind the earlier discussion about the ways in which future methodologies are often apolitical and/or framed as neutral and universal. Efforts must be made in the future to communicate more clearly the political stakes of such approaches. There remains a fundamental tension between researchers and participants around the degree to which the technologies of automation can be reshaped in response to the demands of social and economic justice organizations and their constituents due to the inherent inequities around power, resources, and agency. There are also persistent power imbalances as well as between (and among) university researchers and those advocating for legal protections and worker rights on the ground. Fourth, while we hoped that the project would result in more meaningful collaborations and connections with social and economic justice organizations, we have not been able to foster those relationships based on the workshop alone. Such long-term trusting relationships require a great deal of investment and commitment to the community, which is difficult without sustained funding and face-time working alongside community based organizations. Lastly, at the time of this project, there were not yet strong links between social and economic justice organizations and civic technology organizations. Over the last several years, with continued efforts by advocacy organizations, growing awareness of poor and declining work conditions (made increasingly stark during the pandemic) and growing tech worker organizing, we believe that the links across these advocacy efforts have strengthened. But, at the same time, the power of technology corporations to determine the conditions under which vulnerable people work has also increased, which has only underscored the urgency of finding ways to shift narratives around technology and the future of work.

3.2 Future Design Studio

The second project that we would like to analyze is the FDS. We developed FDS as a part of Emerge 2015, Arizona State University's festival of futurism in art, science, design, and engineering. The FDS was one of 10 “visitations of the future” featured at Emerge, along with a keynote presentation by RadioLab's Jad Abumrad. FDS was a two-part project developed as a way to engage audiences at Emerge in a conversation about technologies of the future through speculation. The first part was the Studio itself where anyone could come and build a low-fidelity prototype of an artifact from the future. The second, the Future Design Players, was an improvisational performance that explored some of the artifacts through long-form improvisation.

3.2.1 The Studio.

We designed the studio to be a fictional design space. Our aim was to provide an open space that had performative or theatrical qualities: a set for participants (it was open to the public) to play the role of designer, or to make believe. This theatricality would help to guide their experience and provide a starting point, so we provided fortune cookies with open-ended questions meant to spark design ideas. These fortunes were in the kinds of questions often asked in cultural probes: oblique (not directly to the point), and whimsical.

There were 15 available prompts. None of these prompts asked users to design a specific object, but rather, they asked a broad question about cultural activities: How do we walk our dogs in the future? What does a first date look like in the future? And How do we regulate temperature and control our environment in the future? were among the fortunes. We began the day with 500 fortune cookies, and ended with less than 50, while approximately 85 artifacts (include several taken home by participants) were created. Many people chose fortune cookies but declined to create artifacts.

To better understand the artifacts these prototypes represent, we asked participants to fill out an “abridged user manual.” (see Figure 4) The manual asked users to note the fortune they received, and to write a description and instructions for using their artifacts. Once the artifacts and user manuals were complete, participants took them to the “future artifact digital archivist.” The archivist photographed the prototype, affixed the photo to the user manual, and scanned the manual for later analysis. Participants kept their user manuals but most left the artifacts to be used in the improvisational performance. The visit to the archivist was designed to provide some kind of closing ritual and continue to build the aura of the fictional design space. Since not everyone who created a prototype also created a user guide, the FDS yielded 69 complete prototypes with user guides, about a dozen prototypes without accompanying guides, and several unfinished prototypes as well. Since these proved to be too difficult to parse without the supplemental information, only the complete prototypes with user guides were included in our analysis.

Fig. 4.

3.2.2 The Artifacts.

In collaboration with researchers from Arizona State University, we analyzed the completed user manuals (See Figure 4), which included images of the artifacts. Each manual was openly coded by two of the authors, and ongoing discussions helped refine the coding process. In the end, the authors agreed on a set of codes, and the manuals were recorded by one author. We identified four categories for the user manuals we examined: iterative, aspirational, cynical, and reactionary. It is important to note that there were many border cases, and some of these categorizations might easily be contested.

We categorized thirty of the artifacts as iterative. This category comprised iterations of existing technologies, new ways of doing things we do already, or ways of serving existing cultural phenomena. For example, a doggie treadmill answered the prompt asking how we walk our dogs in the future; headphones embedded in the ear were created for a prompt asking about music in the future; and a robot that picks up and sorts trash was created to help improve the environment. These artifacts were usually consumer electronics products and many of them involved bodily implants, like screens embedded in arms or holographic elements that replaced smartphone screens. In these cases, the new technologies often performed the same functions as their precursors.

Another thirty of these fell into our aspirational category. They represented transformational ideas or technologies aimed at changing sociotechnical systems in which these artifacts would be embedded. For example, a weather station that transformed global climate or a system to sustainably grow food in one's own backyard. Though they were often Aspirational artifacts sought some sort of transformation in how we do things. This was not always grounded in social good, though that often played a role. Like the iterative artifacts, some of these contained language that seemed to be borrowed from marketing campaigns, or proprietary brand names, like the Google.

A handful of the designs (six in all) were either cynical or reactionary. Objects that indicated an extreme pessimism about humanity were dubbed cynical, while those that resisted the concept of technological advancement “designing” existing human abilities or old technologies, like pen and paper, were coded as reactionary. We found the remaining manuals (three) lacked sufficient information to code into one of the four categories.

Many of the artifacts in both the iterative and aspirational categories had catchy names, like “Modul-ear” or model numbers, like Fanbox 5000. Many also included marketing language or referred to separately sold parts or components, indicating that their concept of the future was somehow tied to marketing tactics like planned obsolescence and iterative improvements in design.

Some interesting themes emerged across the responses to certain prompts. For example, “how do we regulate our environment or control the temperature?” saw the creation of artifacts to accomplish several different things. First, some participants designed iterative ways to control rooms and buildings, replacing existing thermostats with something they saw as superior. Others attempted to control personal environments through force fields built into wearable technologies like bracelets. Still others aimed for global systems to regulate the weather and heal environmental damage.

The future of fandom also inspired a variety of artifacts. One participant saw fandom as a way to own a piece of one's favorite famous person. The Fanbox 5000 promised to provide a miniature replica of your favorite star created with their DNA. Another saw an opportunity to create new ways to emulate the object of one's affection: a sleeve that changes its owner's appearance to match the celebrity they are following. Still another participant opted to create a device for listening/watching favorite artists or sports teams. These are different conceptions of fandom: possession, emulation, consumption.

3.2.3 The Future Design Players.

Because we wanted to maintain the playful spirit of the studio while drawing out some of the more serious implications of designed objects, we chose to use improvisation as opposed to a more traditional deliberative format. The improvisational performance allowed for humor, for the unexpected, and for surprisingly deep exploration of the themes.

To create this humorous forum for continued deliberation, we auditioned and rehearsed with a group of actors trained in log-form improvisation. Long-form differs from the more familiar short-form, which consists of a series of predetermined games often resulting in a sequence of thematically or narratively unrelated scenes. Rather than short, disconnected games and sketches, long-form work is best described as an improvised play in which the theme or narrative is revealed through connections made between characters and ideas over the course of a single or a series of connected scenes. Performers aim at finding human truth in the moment, rather than to seek opportunities for humor [102]. We chose a long form-approach for its emphasis on patterns, themes and connections.

The aim of the Future Design Players was to be able to open up a deeper conversation about the assumptions and values embedded in participants’ artifacts and user manuals. To prepare for such conversations, the performers incorporated thinking from leading scholars in the social studies of science and technology, such as Winner [103] and Bijker [104]. Discussions surrounding seminal articles in the social construction of technology and the politics of artifacts began the rehearsal process and helped shape the final structure for the performance.

The structure of the improvisation was built on an existing structure for a long-form opening called “invocation.” The original invocation asks the audience to suggest an object, and then goes through a series of statements about that object that build to a crescendo, at which point the players move into scene work. Our structure began with our own version of the invocation. Once the audience selected their favorite from among the artifacts the performers chose, the performers began to chant about the object as a Greek chorus, each taking turns providing following a four-stage pattern:

IT IS: Initial statements describe the item and its functions.
MADE: These secondary statements begin to build a history of the item. Who made it, why they made it, what materials, resources, and industries were used to create the object.
LOVED BY/LOATHED BY: These statements examined the stakeholders and included broad statements about industries and organizations as well as accounts of individual relationships with the object.
IT IS: The return to the initial statement created a kind of symmetry; however, this instance offered the performers a chance to more deeply explore the nature of the item. “It is” statements made at the end of this opening probe more deeply into the place the item takes in culture and begin to develop the social and cultural implications of the object.

This opening was proceeded by open ended, free form sketches, and then by a structured closing, similar in nature to the opening, but using “it changed” and “it did not change” phrases to explore the impact of the artifact they had explored over the past 30 minutes. The performance concluded with a discussion.

3.2.4 The Performance.

The Future Design Players each selected an artifact and were given printouts of the user guides for their choice. They had several minutes to read over their user guides and continue warming up before the performance began. When it began, they each introduced their artifact and asked the audience to applaud for their choice. The winning artifact was the “Date-bot 2040X,” an artificial intelligence that helps alleviate both loneliness and social awkwardness. The bot was an answer to the question What does a first date look like in the future? The robot is described in the user manual as “companion contraption: highest order—empathy; design—flawed (artificially); but endearing to all human forms seeking mates.” Though the robot was designed to kiss, most of the functions the creator describes have more to do with emotional rather than physical intimacy. The future design players began with the opening described above to help them establish the world in which a Date-bot 2400X exists.

To do this, the actors used the opening (it is, made, loved/loathed by) to explore who makes Date-bot (factory workers who “don't quite understand what this device is used for”), who designed it, and how it is used. For example:

It has a mode for awkwardness if you'd like to practice.
It is loved by those who have finally learned to be less robotic themselves by loving a robot, they can love a human.
It is loathed by your grandmother who just wants you to find a nice Jewish boy.

Once the stage was set, the actors played out scenes in which Date-bot was used as a relationship coach, a companion, a babysitter, and even a US President (whose bot status is questioned by the media). Date-bots fell in love with humans and with other Date-bots. Maid-bots, Friend-bots, “Bachelor-bots,” and even Dog-bots made appearances, questioning human reliance on artificial intelligence for physical as well as emotional labor.

The scenes explored gender norms, existential loneliness, and prejudice:

I saw you on a date with a Date-bot…we're in the kind of place where people don't use bots.
I'm sorry, it's a part of my past that I'm trying to leave in my past.

It also drew on family dramas and coming of age stories:

Whoa. Are you the Date-bot that taught my dad to date?
Your father? I must have worked!

One subplot saw a bot president blaming faulty programming for illegal or improper behavior:

Any wrongdoing is regrettable, and a result of bad programming.
Mr. Presbot, if you go back to your factory reset, will we have a different administration?

The closing (It changed/did not change) contained some of the most insightful exchanges about the relationship between Date-bot and dating. Here is a sampling of the closing proclamations:

The date bot did not change the desire for human contact.
The date bot expanded what it meant to have a relationship.
The Date-bot did not change the truly awkward.
The Date-bot did not change our fear of the unknown.
The Date-bot changed the first date.
The Date-bot did not change all the other dates.

Though most of the performance was lighthearted and fun, it painted a picture of a world where we trust artificial intelligence more than we trust humans and began to delve into some of the ramifications. Subsequent versions of FDS were developed as pedagogical tools and professional development workshops. The process was refined so that participants also engaged in parts of the performance, pushing them even further, to unpack the unexamined and unintended consequences of what we choose to design and what we choose to buy. See for example, Wigner, Halpern, and Record, 2018 [105].

The audience for Emerge was exploring other visions of the future that included online surveillance of ordinary citizens, utopian cities, and landscapes of the distant future through hacked cameras. These interventions and the design prompts worked together to guide participants toward design for their everyday lives, which are often bombarded with messages about consumption. It makes sense, then, that the artifacts were largely consumer products that people envisioned being largely available to US citizens in the future. The Date-bot, along with many of the other artifacts, was the kind of technology that would be widely available in the US despite its steep price tag. That ubiquity was hinted at in some of the early opening and scene work but could have been more explicitly explored. Future incarnations might see both design prompts and improvisational performances geared toward specific discussions about technology and culture.

3.3 Discussion

Based on an analysis of these two very different projects, we developed the counterfactual actions approach, defined as situated, embodied and performative modes of participatory engagement. While our approach shares some similarities to other relevant examples that we have mentioned above, we argue that there are a number of aspects that distinguish it as well as prove its relevance for HCI researchers as well as for future application in other domains. Rather than arguing that our projects are two ideal cases, we argue that they present a research trajectory that draws on notions of patchwork [1, 106] in which disparate projects can be joined together in order to define and characterize an approach such as counterfactual actions.

Our approach has five key themes—histories, speculation, participation, relationality and situatedness—that make it valuable for informing new directions in HCI research, which we will describe in detail below. A key to our approach is in bringing together these five themes in order to build the counterfactual action approach. While many of the projects that we have described in our literature review engage with one or more of these themes, none of them fully engages with all five themes. Furthermore, since our original empirical projects were conducted and published in an earlier article, there has been a growing interest in the themes that we have outlined, which we will illustrate through references to more recent design precedents than those described in the literature review. In short, since our initial projects, the relevance of our approach for HCI research has grown as it allows researchers to expand and experiment with new methodologies that are uniquely equipped for understanding emerging relationships between humans and machines.

3.3.1 Deep Engagement with Historical Facts is Essential for Relevant Design Futures.

First, related directly to the theme of this special issue, the counterfactual actions approach engages with historical facts (albeit through game-play). As we have argued above, while many approaches to design futures within HCI research focus on uncritical, linear understandings of the futures as directly tied to technology and progress, we seek to trouble these associations by engaging participants in the making of alternative histories and futures.

For example, as one Future of Work participant cautioned:

I think our culture tends to over-valorize technology as always creating good and progress. And I think this kind of exercise helps to break that down. But the distance between those extremes is great and to translate what we did together into real world—I think there's a lot of challenges to bridge that gap.

And, in follow up interviews, another participant working on advocacy for healthcare workers, emphasized:

There's real value in sort of seeing and thinking these things and seeing the progression and the interaction... it's potentially very beneficial, not only historically, but also concretizing…oftentimes, we don't do the work of sitting through and not only examining what has occurred historically, but also sort of using these processes to project [forward]. And I think that, so from the from, particularly in, in an area such as healthcare, where there's a lot of potential trepidation, [there are] huge, huge implications for the workforce. And oftentimes, these issues are not necessarily talked about in a sort of communicated in a transparent or in an effective manner. But then sometimes we need processes that sort of break down the barriers that have existed in terms of them. And I don't differ depending on where he's talking about that prevents those sorts of things from happening. And I think that this is clearly a mechanism for doing so that involves the workforce, I think is really crucial, but does not necessarily always been the norm, when we talk about sort of, sort of decision-making processes, solutions are starting to roll out rollout of particular indicate technology. But I think that there's clear value in that content.

Much research about the “future of X” whether focused on work, food or transportation in HCI is ambivalent about and, often, ignorant of historical context. In particular, HCI and design programs are often ahistorical and/or missing the social, cultural, political, economic and environmental contexts. Furthermore, in recent years, prominent schools have eliminated their historians and/or critical theorists. By using history as a speculative space for the use of counterfactual actions, we illustrate the value of engaging with historical context. While the Reimaging Work project engages explicitly with historical context, for comparison, the FDS project engages temporally with public engagement in futures. Though FDS is focused on futures, the process by which the performers give life to those futures is rooted in counterfactual actions and thinking. In order to make these futures meaningful for audiences, the actors treated them as pasts. As one performer noted, when building worlds around the artifacts, they must “look at it from an outside perspective instead of immersing ourselves immediately into it. As archeologists, as people beyond that time…and once we had those ideas, to immerse ourselves into it, to experience that culture.”

We believe that situating HCI in historical contexts, presents and futures will allow a more reflexive design practices. Lastly, we caution that HCI research is predominantly focused on Western understandings of modernist, technological progress, and temporality. One limitation of our approach is that we also embrace a linear understanding of time. For example, the layout of the Reimagining Work game is highly linear. While we considered and experimented with alternative notions of time—such as simultaneous and circular notions—we found that they were difficult to prototype and develop in this version of the game. We believe that non-linear time and other non-Western notions of time, history and ancestry could result in new understandings of time for the field of HCI.

3.3.2 Speculation is Tightly Coupled with Considerations about Social Consequences.

While many speculative design projects claim to allow audiences to consider the ethical implications around the development and use of technology, very few live up to their promise. In addition, such projects are often overly biased towards thinking about imaginative new uses for technology rather than considering their harms. In cases where social implications are considered, projects often turn towards dystopias imagined by a white, male, able-bodied and wealthy designer rather than the harsh realities that are experienced by billions of people everyday. As the performers developed their process for FDS, they often discussed walking this line between envisioning the dystopian futures or failing to adequately address social consequences. Reflecting on the process, they found that developing complex histories for these objects and the worlds they inhabit made their visions of the future more human. One performer described this as, “Trying to figure out the humanity behind it.” she continued, “There is an assumption that people in the future suddenly know everything…the future is the self, realized, full of genius magical people. It was nice to realize we had that preconceived notion and also break it. Oh yeah, people can be just as confused with incredible forms of technology [as] we are now.”

One new project within HCI research that engages both with speculation as well as with a more robust understanding of the legal and ethical considerations around technology is Casey Fiesler's recent work [107–109].

As one of our Future of Work participants, working on immigration rights advocacy, reflected:

…in the game, the whole thing on on, you know, bits and pieces of labor organizing throughout history, whether that's, you know, with slave revolts in the, in ancient Egypt, to the industrial age, and then even picturing into the future, what, you know, what does? You know, what does collective action at the workplace look like…50 years from now. So I think…that kind of imaginative exercise allows our folks who can begin to see new possibilities in the present as well.

They expanded on this, drawing attention to the tension between artifacts and actions that we have elaborated in this article:

…the game I thought was, was useful, it's something that our folks could probably enter into and play some version of it and get and come out with insights and ideas. On some level, the fact that the prototyping was focused around a thing [emphasis added] …could be limiting as well…what if instead of, you know, prototyping…instead of looking at an object, you know, what would, you know, what would a process of organizing look like? [emphasis added]…These are the questions that I think a lot of organizers are dealing with, you know, how do you organize fast food workers? And, you know, some people are trying some experiments with that, or, or day laborers or domestic workers or things like that is to say, you know, even, yeah, 10 years from now, like, what, what new technologies, could you imagine that would support or enhance the organizing work that you're doing?

At the same time, they continued, the speculative artifacts themselves created some interesting trajectories: “The prototypes that we came up with were actually interesting…this idea that came out of our group was an artificial emotional intelligence. Like, I've actually been doing some poking around on artificial intelligence, since then, because it felt like such a compelling idea to me.”

Again, while most speculative work focuses on futures, our work engages with “what ifs” in historical settings, making up alternative histories and thinking through the possible implications of these counterfactual actions as part of game-play. Tying speculative artifacts more directly to counterfactual actions is one way to achieve more relevance for the purpose of social change and social consequences.

3.3.3 Participation and Community-based Futuring are Necessary for Impactful Design Futures.

While not necessarily unique in HCI given the acceptance of PD within HCI research more recently, our approach is specifically oriented towards engaging stakeholders and publics in conversations about the political choices that we make as a society around the development and use of technologies. While many speculative design projects consider the designers as the experts that design a set of artifacts that allow for engagement with possible futures, our projects specifically sought to engage participants in the practice of futuring itself by allowing them to engage in counterfactual actions.

Yet, upon reflection, our engagement with stakeholders and publics may not have gone far enough in depth nor in making a more long-term commitment. For example, in the Reimagining Work project, while we designed the game in response to a set of interviews with social and economic justice groups, we did not have strong enough relationships with them at that time to engage them in choices about whether a game would be valued and/or what the game might allow for. In this sense, we created the conditions for design futuring without explicit input from the stakeholders that we wished to engage. Similarly, in the FDS performance, we created the idea for a theatrical engagement without the input of participants. As such, for the purpose of these projects, we achieved a participatory engagement but, perhaps, not an actual PD of the projects.

One of our Future of Work participants strongly emphasized the value of our process, saying that the workshop could be used in a variety of advocacy settings:

Yeah, so I actually came back and I did say that I thought we should be using them…Yeah. So absolutely. I absolutely see the potential benefits of using so even the activity that we went through, I absolutely see the potential benefits of doing so…there's clear value. And so one of things we talked about also was involving the worker perspective and things of that sort. I think there's clear value in the work that we are having individuals go through that exact same process that we have no question. I have no question about that. And I've actually recommended that, I think that we can do that, both in the context of I mean, there are there upcoming things that we have that might be larger scale things, and they're also smaller venues that we can make use of the process...I think that the value in sort of an exit sort of an exercise, this is one that we went through is it sort of, it provides the space for people to engage in a different manner than they traditionally have done. Even if it's a facilitated discussion or conversation in the labor management contract, right. There's certain things that always said in the labor management context…I think that the exercise that we went through, provides a space for individuals to engage perhaps in different ways in which they had in the past, or perhaps there are other barriers for people to engage in the imaginal, which could be most fulfilling to them, right. So I think that this provides the opportunity to, because of the nature of the design process provides the opportunity to engage in a different way. It provides a nice quality for workers to engage in different ways.

3.3.4 Understanding and Engaging with Socio-technical Systems Requires a More Relational Approach.

The HCI field is still predominantly focused on making tools, artifacts, things and prototypes as solutions to problems (and this impulse is shared in the design and futures fields). While there is a growing community of researchers interested in critical approaches such as the sub-community on critical social computing, the focus on making objects as discursive tools for the purpose of asking questions is still not the norm. While we believe that artifacts, things and prototypes (or, more accurately ‘provotypes’) are valuable, there is an opportunity to bring them to life through actions, events and situations to emphasize their relationality. Because we are committed to broadening debates about human machine relations within the context of complex socio-technical systems, we believe that a relational approach is essential. Barad's notion of intra-actions signifies that things and people together shape agencies in dynamic ways [110]. This is captured by a number of projects mentioned above in the literature review. As such, the focus on making things or artifacts is not necessarily static or fixed but the shift to actions allows for a deeper exploration of identities, relations and processes in the context of socio-technical systems.

Both of our projects illustrate ways of participating in and creating actions. For example, in the Reimagining Work project, the use of “Action Cards” suggests the opportunity to create counterfactual actions such as worker campaigns in historical contexts. The focus on actions also references the ways in which humans not only have relationships with individual artifacts but, rather, are entangled in broader and more complex socio-technical systems. Through game-play, we sought to focus on the ways in which things become relational through being embedded in actions. In our interviews for the Reimagining Work project [111], we sought to understand the ways in which (different) technologies could be seen, on the one hand, as impenetrable “black boxes” as well as, on the other hand, easily used for activism in social media campaigns on labor issues such as the “Fight for 15”. This contradiction led us to want to embed agency and action into the game in a more deliberate way since activism is often tied to action (rather than discourse). As we disagree with the binary opposition between discourse and action and, rather, embrace a praxis that integrates critique with creation, our game sought to integrate discourse and action through game-play. This necessary focus on relationality has been reinforced more recently in discourses around the posthuman [112], the more-than-human, the decentering of the human [113], other than human [114] and entanglement in HCI research [115]. Rather than viewing things and artifacts as fixed entities as previous HCI research has typically done, we embrace a phenomenological understanding of the agency and dynamism of things in relation to humans in the context of wider socio-technical systems. We believe that our projects do this quite successfully, informed by the field of science and technology studies. Here, recent scholarship has emphasized the co-performance of relations between people and things with respect to artificial intelligence, which helps to dispel a range of myths about the deterministic agency of AI [116].

3.3.5 A Relational Approach is Predicated on Situated, Performative and Embodied Engagement.

In order for an approach to be relational, there must be a kind of responsiveness embedded into the process. Suchman's original notion of situated action, based on her studies at Xerox PARC, refers to the ways in which digital technologies that are designed around psychological models and plans are often in conflict with the ways in which situated actions play out in everyday human-computer interaction [117]. Suchman uses the metaphor of canoeing through rapids in order to illustrate the ways in which context-specific and embodied responses are essential to understand the relationship of humans to the world and, in particular, to our technological systems. Building on this, recent work in HCI has engaged with literature on infrastructures from science and technology studies [118, 119]. Specifically, Jack et al build on Nguyen's concept of infrastructural action [120] to develop their understanding of infrastructure as creative action defined as “the resourceful, ad hoc and imaginative development of homegrown infrastructures and the work of integrating new tools into older infrastructures and cultural practices” [119].

We share this orientation and believe that it is an essential component for design futuring. Our approach to counterfactual actions is similarly a situated, performative and embodied engagement. Counterfactual actions—like other experiential approaches such as role-playing and bodystorming [121]—require that participants put themselves into new situations in order to explore what it might feel like to be a different person in a different time or place or, even, as themselves in a different situation whether historical, present or future. For the purpose of the Future of Work project, this meant participating in and imagining historical identities, relations and processes from the perspective of their present role as worker advocates. This emphasis on their existing professional identities, ties our approach to another important strand of emerging HCI methodologies around somaesthetics, first-person research and autoethnography. For FDS, this embodied engagement meant imagining futures through specific imagined artifacts, and then exploring how those artifacts shape the imagined, counterfactual futures in which they exist. As one of the performers noted, they were engaging in a “collaboration with the objects that had been created. Trying to find potential context for them and how they would come about and effect…potential realities.”

Specifically, counterfactual actions take up engagement with embodied interaction and embodied design [122, 123], somaesthetics [124] and first-person approaches in design [125], which draw on the knowledge that can be gained from engaging the body [124, 126, 127]. One such approach is the creation of biographical prototypes [128] based on first-person accounts by disabled people in order to recognize their contributions to the field of design in terms of the ways in which they draw on their own experience to make something work. Another relevant lineage of this work is tied to social practice theory [129] and, in particular, Kuijer et al.’s study of bathing, which was “integrated into a Generative Improv Performances (GIP) approach, entailing a series of performances by improvisation actors with low- fidelity prototypes in a lab environment” [130]. Specifically, they argue that GIP shifts the focus away from professional designers and towards actors that are skilled at navigating complex situations for the purpose of working out more sustainable use of resources. This is perhaps similar to Cefkin's [131] lab-based engagements with ethnographic futures around autonomous vehicles, which mobilizes the space of the lab as a site for ethnographic understandings about cultural practices rather than their more common use for psychology experiments. Lastly, counterfactual actions draw on notions of praxis in which theories can be experienced in everyday life and which are closely tied to the work of social justice and liberation [132, 133]. We argue that finding ways of bringing situations to life in more embodied and performative ways will help to build less deterministic and more complex and dynamic ways of understanding human-technology relations.

By moving from artifacts to actions, we support ways of experiencing human-technology relations in more embodied, situated and performative ways. This approach builds on recent interest in somatic approaches within design such as autoethnography and first-person research. As one Future of Work participant noted:

I was struck by, especially in the [prototyping activity], how we started off in our group just talking and we just kept talking around each other until you actually came and intervened and said you must get up and do something and it wasn't until we actually started physically moving that we were—we weren't’ developing our ideas, we were just putting stuff together. And for a linear thinker that just like how did that happen. We're used to brainstorming or you know you have to battle out the ideas before you get to any kind of action, and this was actually, doing a physicality that actually helped me think through a problem.

Here, the reference to getting up and physically moving as part of the thinking process is striking as it is a direct reference to the value of a nondualistic somaesthetic approach. Rather than placing an overemphasis on the artifacts that were created, we seek to focus attention on the embodied actions of the participants. Counterfactual actions, as creative rather than dialogic, help to move away from the continued overemphasis on dualistic, rational, mind-focused modes of knowing and towards experimenting with the role of the body as a generative source for HCI.

4 Design Implications, Limitations, and Future Directions

This research has a number of significant implications for the field of HCI. Specifically, the five key themes that are integral to the counterfactual actions approach offer rich directions for the field. Specifically, the interdisciplinary literatures that we have drawn together around histories, speculation, participation, relationality, and situatedness invite deeper engagement. While when considered individually these themes are familiar to HCI researchers working at the intersection of HCI and related fields such as science and technology studies and anthropology, one can easily argue that they are still at the margins of the field [90]. The vast majority of HCI research is still focused on more positivist, linear, extractive and quantitative understandings of what it means to make scientific knowledge, how to do that and with whom to do that. Our approach brings together five important themes that underpin a critical orientation to HCI (and design research more broadly) that still has great potential to inform research, pedagogy, and practice. While there are many research projects around the world that draw on similar approaches, these approaches are still not well defined and/or supported in HCI curricula and/or practice as they are likely to require interdisciplinary and/or transdisciplinary knowledge. For example, there are only very few curricula that are specifically oriented around design futures specifically, which brings these various strands of knowledge and practice around histories, speculation, participation, relationality, and situatedness together into an essential pedagogy for the future. And, even more challenging, doing any of these things well in a single semester is difficult. As such, we believe that we are only at the very beginning of developing more imaginative, generative, and performative modes of engaging with important questions around the ways in which technology might be shaped in the future.

In particular, we believe that our approach is most relevant to HCI researchers interested in participatory and community-based futuring methodologies, which are relatively nascent developments in the field. As technology continues to play an essential, and even an existential role in the future of society, it is vital to continue to find ways of critically engaging the public with the ethical and political stakes around these choices. We argue that the counterfactual actions approach can be used to create the conditions through which to develop and experience a more relational understanding of the ways in which people can engage with the politics of infrastructures and socio-technical systems as well as with each other. We believe that counterfactual actions are highly useful for exploring the social consequences—both first order, second order and third order implications of the worlds that we are building with and through technologies as well as with the historical context around specific moments in history as well as first-person experiences of the present. By asking how the world might be otherwise, and performing those actions through games, theater and other experiential modes, we believe that we can broaden the conversation around the politics of our future technologies and build greater capacity for intervention and agency in these choices, albeit in a modest way.

4.1 Limitations and Recommendations

As mentioned earlier in this article, there are two main limitations of our projects that we would like to reiterate for the benefit of future HCI research. First, like many other speculative projects, we were not able to find a way to embrace an alternate, non-linear understanding of time in these projects. While indigenous and decolonizing methodologies [134] suggest that vastly different understandings of time are possible, we could not reconcile this with the design of the game and/or performance. Yet, we believe that a pluriversal [135] approach to design futuring and/or a creative reimagining of the world with a broader perspective on worldviews is an important goal that will likely challenge understandings of power, ownership and redistributions of wealth and natural resources.

Second, we do not promise any specific emancipatory outcomes or social changes from these projects. Rather than PD of a particular technology, it is more accurate to understand them as participatory engagement workshops that put the issue of technology's relationship to histories and futures up for discussion as a central concern. While we could have expanded the ways in which we engaged participants—for example, engaging them much earlier in the design of the project, that was not possible given the relatively small amount of funding, the relatively short timeline and lack of pre-existing relationships with participants and community organizations—again highlighting the patchworked nature of our process. Participatory approaches require resources, time and trusting relationships that cannot quickly be materialized. Despite this, we believe that we accomplished something meaningful that has helped to inform our own research since these two projects were conducted. Technology is often coupled with modernity and progress in linear, deterministic assumptions about the direction of and decisions about what is good for whom. We believe that this meta-discussion is of great value to building equitable and just futures enabled, when appropriate and relevant, by technology. Given the complexity of both the issues that we engaged as well as the counterfactual action approach that we suggest, we believe that these projects are an important step towards bringing design futuring into HCI research in a meaningful way.

4.2 Future Directions

We believe that the counterfactual actions approach is likely to be applicable in a number of domains in HCI beyond the specific projects that have been mentioned here. For example, recent research has illustrated that somatic engagements in art, music and theater are highly beneficial for people with trauma and post-traumatic stress disorder (PTSD) [136, 137]. This suggests that HCI projects that are engaged with histories and futures -related to chronic illness, disease and disability in healthcare settings might find particular value in moving beyond verbal articulation of explicit knowledge or 2-dimensional visualizations and towards the more situated, embodied and performative counterfactual actions approach that we have described in this article. Furthermore, there are a wide range of human experiences that draw on the affective dimensions of all of the five senses—sight, sound, touch, taste, and smell as well as perhaps lesser-known senses such as bodily movement and position. For example, many future food projects engage directly with taste and smell such as Christy Spackman's work on sensory labor [138]. Lastly, since our projects were conducted, there has been a growing tradition of work that engages more directly with theater both with trained actors as well as researchers and publics. We believe that such collaborations are essential for a deeper understanding of the ways in which computing is and will be experienced in the future. For example, Affect Theater and Feminist Theory Theater are two notable examples [139, 140] that emphasize responsiveness and processes for experiencing theory and data in new ways.

5 Conclusion

This article engages with history as a speculative space for the purpose of working towards more critically, socially, and politically engaged futures in the field of HCI. Drawing on related work around prototyping the past and the use of counterfactual artifacts to understand human-technology relations, we develop counterfactual actions as a situated, embodied and performative approach that moves beyond the design of things and towards engaging with infrastructures and socio-technical systems. Drawing on participatory engagement, we illustrate the results of two projects that have allowed us to develop our approach. Rather than focusing exclusively on making artifacts and things, counterfactual actions place the focus on collectively exploring new identities, relations and processes. Ultimately, we believe that this form of imagining will allow for the development of new narratives about the relationship between computing and society, expanding people's ability to assert greater agency and control (rather than merely passive users or oppressed workers). We believe that these experiences allow participants to view technology not as a deterministic force but rather as a malleable, relational and changeable creation of humans. In short, our engagement with history in this way allows for the possibility of worlds—past, present and future—that might be designed otherwise.

Footnote

See https://vault.fbi.gov/fritz-julius-kuhn. Accessed on March 28, 2017.

References

[1]

G. Günel, S. Varma, and C. Watanabe. 2020. A manifesto for patchwork ethnography. Society for Cultural Anthropology 9 (2020).

Abstract

1 Introduction

1.1 Techno-visions and the Problem of the Future

1.2 Labor History, Worker Advocacy, and the Future of Work

2 Speculation and Fiction, Fabulation and Fabrication

2.1 Connecting Interdisciplinary Approaches to Critical and Generative Research

2.2 Sensing the Future through Speculative Storytelling, Design Fiction, and Feminist Fabulation

2.3 Public and Community Engagement in Speculative Futures

2.3.1 Prototyping the Past.

2.3.2 Counterfactual Thinking, Counterfactual Artifacts, and Counterfactual Actions.

3 The Workshops

3.1 Reimagining Work

3.1.1 Contributions and Hopes.

3.1.2 Reflections and Frictions.

3.2 Future Design Studio

3.2.1 The Studio.

3.2.2 The Artifacts.

3.2.3 The Future Design Players.

3.2.4 The Performance.

3.3 Discussion

3.3.1 Deep Engagement with Historical Facts is Essential for Relevant Design Futures.

3.3.2 Speculation is Tightly Coupled with Considerations about Social Consequences.

3.3.3 Participation and Community-based Futuring are Necessary for Impactful Design Futures.

3.3.4 Understanding and Engaging with Socio-technical Systems Requires a More Relational Approach.

3.3.5 A Relational Approach is Predicated on Situated, Performative and Embodied Engagement.

4 Design Implications, Limitations, and Future Directions

4.1 Limitations and Recommendations

4.2 Future Directions

5 Conclusion

Footnote

References

Cited By

Index Terms

Recommendations

On Speculative Enactments

Expanding Modes of Reflection in Design Futuring

Engaging the General Public in Speculation on Human-Building Interaction Futures at a Pop-up Event: A Design Fiction Approach

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

HTML Format

Login options

Full Access

Share

Share this Publication link

Share on social media

Affiliations