ARTICLE
https://doi.org/10.1057/s41599-023-02060-8
OPEN
The concept of “interaction” in debates on
human–machine interaction
1234567890():,;
Sebastian Schleidgen
1 ✉,
Orsolya Friedrich1, Selin Gerlek2, Galia Assadi
3
& Johanna Seifert1
The concept of “interaction” is central to debates on Human–machine interaction (HMI). At
the same time, however, it is vague and ambiguous: “interaction” is understood in different
ways within and between the scientific disciplines involved in debates on HMI. Ultimately,
this makes it difficult to reasonably debate questions of ethics, politics, engineering, and the
sciences regarding HMI. Against this background, we elaborate and analyze the different
meanings and dimensions of the term “interaction” in the disciplines and discourses relevant
to debates on modern HMI. For this purpose, we introduce a four-dimensional SMPC model,
according to which “interaction” is specified with a view to its subjects, modes, purposes, and
contexts. With this, we aim to provide a basis for a fruitful intra- and particularly interdisciplinary discourse on HMI.
1 FernUniversität in Hagen, Hagen, Germany. 2 University of Amsterdam, Amsterdam, The Netherlands. 3 Evangelische Hochschule Nürnberg,
Nuremberg, Germany. ✉email: sebastian.schleidgen@fernuni-hagen.de
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2023)10:551 | https://doi.org/10.1057/s41599-023-02060-8
1
ARTICLE
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-023-02060-8
Introduction
ecent developments in artificial intelligence (AI), machine
learning (ML), neuro- and self-tracking technologies, or
social robotics have increasingly prompted debates on the
conditions for successful Human–Machine Interaction (HMI), as
well as its potential implications and consequences for engineering, the sciences, ethics, and politics. While there is a strong
tradition of (interdisciplinary) exploring the necessary conditions
for interaction between humans and computers in informatics,
engineering, the humanities, and the sciences, these disciplines
were primarily concerned with questions regarding appropriate
user interfaces for a long time, i.e., with possible ways of adequately and effectively transferring data and information between
humans (understood as users) and computers with the goal of
solving certain problems. Hence, the primary focus was on
developing visually, haptically, and linguistically adequate input
as well as output devices for effectively using computers (referred
to as Human–Computer Interaction (HCI))1.
However, since the 1980s, following, e.g., the development of
the first Brain–Computer Interfaces (BCIs), augmented and virtual reality, ML, or ubiquitous computing, there has been a tendency away from this focus on useful devices toward a more
sophisticated understanding of interaction, often referring to
some kind of dialogue or communication between humans and
machines in a broad sense (i.e., neither are humans only understood as users, nor is the focus solely on computers anymore).
This often is referred to as HMI and ultimately resulted in the still
ongoing efforts to simulate essential characteristics and conditions of human communication in machines. Even though there
is by far no agreement on what these are exactly (for instance,
consciousness, intelligence, or embodiment) and how they could
effectively be simulated, the perspectives of sociology, philosophy,
psychology, and cognitive science as well as media studies and
communication science increasingly came into play in the
development of machines capable of interacting with humans.
It is this background against which Janlert and Stolterman state
that “interactivity is one of the most commonly mentioned and
prominent characteristics of digital artifacts” (Janlert and
Stolterman, 2017: 107; cf. Rafaeli, 1988; Bucy, 2004; McMillan,
2005). At the same time, however, the authors note that although
there is a colloquial idea of what “interaction” means, and, hence,
what the term refers to, the concepts’ usage, especially in scientific
debates, is still vague and ambiguous (Janlert and Stolterman,
2017: p. 105; cf. Bucy, 2004). It can be noted that “interaction” is
often used to denote “mutual or reciprocal action or influence”
(Merriam-Webster Dictionary, 2022). According to this understanding, “interaction” refers to something (e.g., certain actions)
taking place between two or more entities (in most cases:
humans) with a view to some purpose or goal (within a certain
context), and with any of these entities undertaking an active role
(in the process of interacting). Given this characterization, the
reason for the term “interaction” being vague and ambiguous
becomes obvious: what exactly takes place in interactions in what
contexts between what kinds of entities with a view to what
purposes or goals can be interpreted in many different ways.
However, a vague and ambiguous concept of interaction makes it
difficult to reasonably debate questions of ethics, politics, engineering, and the sciences when it comes to HMI. Strictly speaking,
even (fundamental) questions like “is interaction (in this situation)
actually happening? Is it good or bad? What does this mean for
future design processes?” are difficult to answer (Janlert and
Stolterman, 2017). This issue is further complicated by the fact that
ethical and philosophical debates on HMI often refer only implicitly
to the concept of interaction without analyzing or explaining it.
Against this background, in the following, we elaborate and
analyze the different meanings and dimensions of the term
R
2
“interaction” in the disciplines and discourses relevant to debates
on modern HMI. This helps to highlight similarities and differences in disciplinary understandings and generate conceptual
clarity for subsequent normative debates on HMI. For this purpose, we, first, introduce a four-dimensional model of interaction
as a basis for analyzing the different meanings of “interaction”.
Second, we present some important terms related to the concept
of “interaction”, i.e., the concepts of interactivity, interactability,
and interactiveness. Third, we elaborate on the most prominent
meanings attached to the concept of interaction in the disciplines
essential to the discourse on HMI. This will, fourth, be followed
by an analysis of their key elements with a view to our fourdimensional model of interaction.
With this, we do not claim to elaborate a single correct definition of “interaction”. Rather, we aim to give an overview of the
currently most prominent usages of “interaction” as well as their
implications and presuppositions with the aim of providing a
basis for a fruitful intra- and particularly interdisciplinary discourse on HMI between, e.g., philosophers of science, social scientists, linguists, engineers, or designers, that also captures novel
aspects of interaction in emerging AI-based technologies. Not all
approaches presented here make explicit reference to HMI.
Nevertheless, they are relevant to the debate insofar as they
provide frequently used background assumptions in debates
about HMI.
A four-dimensional model of interaction
Recall that, according to the understanding of “interaction” as
mutual or reciprocal action or influence, the term refers to
something (e.g., certain actions) taking place between two or
more entities with a view to some purpose or goal (within a
certain context). Hence, interaction may be understood as a fourdimensional concept referring (1) to certain subjects (i.e., to the
question: who interacts?), (2) to modes of interaction (how do
these subjects interact?), (3) to purposes of interaction (why, or:
for what reasons is interaction taking place?), and (4) to certain
contexts (where, or: under what conditions is interaction taking
place?). In the following, we refer to this as the SMPC model of
interaction, with regard to which we analyze the different
meanings of “interaction”. Ultimately, given its formal characterization, the reason for the vagueness and ambiguity of
“interaction” can be further specified: it is the four dimensions of
“interaction” that are interpreted differently intra- as well as
interdisciplinary.
“Interaction” and related terms
The understanding of “interaction” as mutual or reciprocal action
or influence has been criticized for not considering essential
aspects of interaction, especially when it comes to HMI. Svanæs
(2000), e.g., has pointed out that “being interactive” may not only
denote some sort of ongoing process, but also some kind of
potential or disposition: humans (as well as, for instance, computers) would even be considered interactive in situations where
they were not actively involved in mutual or reciprocal action
(humans) or not in use (computers and machines). Against this
background, authors often refer to “interactivity” instead of
“interaction” (Rafaeli, 1988; Svanæs, 2000; Kiousis, 2002; Bucy,
2004; Bucy and Tao, 2007), using it “(a) as a general term for the
phenomenon of [or disposition for] interaction, and (b) as a term
for ongoing interaction” (Janlert and Stolterman, 2017: p. 112). In
the following, we are primarily interested in the different meanings and dimensions attached to processes of ongoing interaction
(usage (b) of “interactivity”), whereas we understand usage (a) as
delineating the conditions principally necessary for processes of
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2023)10:551 | https://doi.org/10.1057/s41599-023-02060-8
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-023-02060-8
interaction and hence, as being necessarily presupposed in any
reference to “interactivity” in the sense of (b).
Janlert and Stolterman seem to have something similar in mind
when introducing the term “interactability” to denote the
“intrinsic quality of an artifact or system that allows for interactions with a user” (Janlert and Stolterman, 2017: p. 112) and,
hence, to specify usage (a) of “interactivity”. According to the
authors, “interactability” delineates the specific properties or qualities of specific objects (i.e., computers or machines) in specific
scenarios, which (to a certain degree) enable interaction with these
very objects in these very scenarios. In contrast, we understand
usage (a) as designating the principal conditions that entities
(computers, machines, or humans) must satisfy to be involved in
interaction processes at all. Nevertheless, the term “interactability”
captures an important aspect: the necessity of specifying these very
conditions in view of specific scenarios and specific entities. We,
thus, understand “interactability” as delineating the possibilities
and limitations of specific computers or machines to interact or, in
other words, the degree of a specific computer’s or machine’s
disposition for interaction (i.e., usage (a) of “interactivity”) and,
hence, the conditions for concrete processes of interaction with
this very computer or machine (usage (b) of “interactivity”). This,
however, presupposes some conceptual point of reference, i.e., an
understanding of usage (a) informing of what it means that
somebody or something has a disposition for interaction. The fact
that Janlert and Stolterman miss this important point might be
due to their more technical and, hence, application-oriented
approach to the questions of the meaning of “interaction” in HMI.
They point, however, to another important concept in HMI
scenarios: as “interactability” only specifies the degree of a specific
computer’s or machine’s disposition for interaction, a high degree
of interactability does not necessarily result in actual interaction
between humans and computers or machines. What is lacking,
therefore, is a term denoting “an artifact’s or system’s propensity
to engage users in interactions” (Janlert and Stolterman, 2017: p.
113), i.e., a term that refers to the sense or extent to which a
computer or machine stimulates its human counterpart to interact
with it at all or to maintain an ongoing interaction. The authors
propose the term “interactiveness” to denote this phenomenon.
To sum up, “interaction” is intricately connected to the terms
“interactivity”, “interactability”, as well as “interactiveness” (Fig. 1).2
Interactivity
Disposition
for
Interaction
(Conditions
for
Interaction)
(Specific)
InteractionEnabling
Properties
(Interactability)
(Ongoing)
Process of
Interaction
Propensity to
Interaction
Engagement
(Interactiveness)
Fig. 1 Interactivity, Interaction, Interactability, and Interactiveness.
Relation of terms connected with “interaction”.
ARTICLE
In what follows, against the background of our SMPC model,
we are primarily interested in analyzing the different meanings of
“interaction” as a process, i.e., in the sense of usage (b) of
“interactivity”. Where appropriate, we will, nevertheless, introduce the meanings of “interactability” and “interactiveness” as
implied in the different accounts of “interaction”.
The concept of “interaction” in disciplines essential to the
discourse on HMI
“Interaction” in informatics and computer science. In informatics and computer science (as the main reference point of HMI
debates), several stages of addressing interaction between humans
and computers or machines must be distinguished. Following
Charles Babbage’s concept of the Analytical Engine in 1837
(Bromley, 1982), Ada Lovelace’s first computer program in 1842
(Charman-Anderson, 2015), or the development of the first
punch-card-based data processing system by Herman Hollerith in
1889 (Heide, 2009), the focus was mainly on questions of construction and effectiveness of algorithms and computer
predecessors.
This did not change significantly until the development of
Konrad Zuse’s first digital computer Z3 in 1941 and the growing
need to cope with the constantly increasing amount of data and
information (e.g., in research and the sciences) with the help of
calculating machines (Bush, 1945). Subsequently, questions
regarding appropriate user interfaces aiming at accessible and
understandable ways of one- or two-way data and information
transfer between users and computers became increasingly
virulent. This was mostly referred to as questions of HCI. In
the following years, the development of the first handwriting
recognition devices (Dimond, 1957), the first graphical computer
system DAC-1 (Krull, 1994), and the computer game Tennis for
Two (Gold, 2004) were marking milestones in this regard.
Up to this point, questions of appropriate user interfaces for
data and information exchange between users and computers
were solely discussed regarding users transferring pre-formulated
problems to computers, getting them processed and the results
returned (e.g., transferring handwritten records or design
sketches to be returned in digitized form). In the 1960s, this
changed fundamentally when Joseph Licklider in his seminal
paper Man-Computer Symbiosis brought up the vision of
computers not only being used for processing pre-formulated
problems but also for the development of new (technical)
problems:
[…] many problems that can be thought through in
advance are very difficult to think through in advance. They
would be easier to solve, and they could be solved faster,
through an intuitively guided trial-and-error procedure in
which the computer cooperated, turning up flaws in the
reasoning or revealing unexpected turns in the solution.
Other problems simply cannot be formulated without
computing-machine aid. Poincaré anticipated the frustration of an important group of would-be computer users
when he said, “The question is not, ’What is the answer?’
The question is, ’What is the question?’” One of the main
aims of man-computer symbiosis is to bring the computing
machine effectively into the formulative parts of technical
problems. (Licklider, 1960: p. 5).
In this regard, Licklider’s vision implied some kind of division
of labor between user and computer. As such, it is often
understood as having introduced the very idea of interactive
systems (focusing on questions of interactability). This led to a
decisive shift in considerations on user interfaces: ways of
accessible and understandable data and information transfer, as
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2023)10:551 | https://doi.org/10.1057/s41599-023-02060-8
3
ARTICLE
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-023-02060-8
well as questions regarding the possibility of and conditions for
interactive communication and dialog between users and
computers, increasingly got into the focus of research and
development (cf. Licklider, 1960).3 Licklider described the basic
features of such interactive interfaces as follows:
Certainly, for effective man-computer interaction, it will be
necessary for the man and the computer to draw graphs
and pictures and to write notes and equations to each other
on the same display surface. […] With such an inputoutput device, the operator would quickly learn to write or
print in a manner legible to the machine. […] He could
correct the computer’s data, instruct the machine via flow
diagrams, and in general interact with it very much as he
would with another engineer, except that the „other
engineer“ would be a precise draftsman, a lightning
calculator, a mnemonic wizard, and many other valuable
partners all in one. (Licklider, 1960: p. 9).
In the subsequent years, a number of innovative interfaces were
developed taking up these ideas, e.g., Ivan Sutherlands interactive
drawing program (Sutherland, 1964), the first ever Virtual Reality
System (Sutherland, 1968), the NLS (oN-Line System) introducing the computer mouse (Barnes, 1997), the RAND tablet with
its GRAIL (GRaphical Input Language) system (Ellis et al., 1969)
or several haptic interfaces (Brooks et al., 1990). This was
followed in the 1970s by the development of, for instance, the first
WYSIWYG (what you see it what you get)-based word processing
software BRAVO (Newman, 2012), the concepts of responsive
environments and artificial reality (Krueger, 1977; 1983), or the
first ever data glove (Sturman and Zeltzer, 1994). In the 1980s,
commercial computer systems emerged including more sophisticated WYSIWYG-based software (Johnson et al., 1989; Perkins
et al., 1997) as did new interaction concepts as, for instance,
multi-touch input devices and touch screens (Buxton et al., 1985;
Lee et al., 1985; Buxton, 2010).
A somewhat different step in the history of interactive
technologies, however, was the development of the first BCI in
1988, which aimed at enabling locked-in patients to communicate
with their outside world (Farwell and Donchin, 1988) and, thus,
added a further aspect of interaction between users and machines.
Yet another aspect appeared in the 1990s when the first
approaches to augmented reality emerged (Thomas and David,
1992). Moreover, growing miniaturization in microelectronics
allowed for the development of new forms of humans interacting
with machines, exemplified, for instance, by so-called embedded
systems, ubiquitous computing (Weiser, 1991), or new interfaces
for mobile devices. In the early 2000s, body language as yet
another feature of interaction came to the fore and led to the
development of gaming technologies like the EyeToy, the Wii
Remote or Kinect (Nowogrodzki, 2018), as well as to the first
approaches of using human hand gestures in interactions with
machines (Maes and Mistry, 2009). Furthermore, research in the
field of brain–computer interfaces was intensified with the aim of
using the human body itself as an interface for interacting with
machines (Velliste et al., 2008). From now on, interaction was not
necessarily understood as a direct and explicit exchange of data
and information anymore. This is one aspect leading to the
increasing use of the concept of HMI instead of HCI.
In recent years, research in informatics and computer science
had a strong focus on AI, especially on the development of neural
networks and approaches of ML (Simeone, 2018), leading to
groundbreaking developments in, for instance, online search and
automated image recognition systems, social media algorithms, as
well as concepts and first applications of, e.g., autonomous
vehicles (Schwarting et al., 2018), medical decision support
systems (McKinney et al., 2020), or systems for determining the
4
probability of criminal recidivism (Biddle, 2022) and thus to
novel ways of HMI.
In summary, the analysis and investigation of interaction
between humans and machines in informatics and computer
science can roughly be divided into three stages: in the first stage,
beginning in the 1840s and ending around the 1950s, the focus
was primarily on questions of engineering, construction, and
effectiveness of algorithms and computer predecessors.
The second stage (focusing on HCI), starting in the 1950s, was
dominated by approaches to develop appropriate user interfaces
for data and information exchange between users and computers,
focusing on the successful transfer of pre-formulated problems to
computers to be processed and the results returned. At this stage,
informatics and computer science were primarily concerned with
questions of interactability and interactiveness, i.e., with the
disposition of computers to interact with users in view of their
objectives. Interaction was understood as a direct and explicit
exchange of data and information and followed purely epistemic
objectives (cf. Dix et al., 2003: p. 126; Norman, 1984; 2013) (i.e.,
computers were understood as cognitive devices “that extend or
supplement human cognitive functioning by performing information processing tasks” (Brey, 2005: p. 384). Regarding our
SMPC model of interaction, users and computers were understood as asymmetrical subjects of interaction (referring to the
question of who interacts), interacting in the mode of exchanging
data and information (referring to the question of how interaction
takes place) for the purpose of solving certain user-centered
problems (referring to the question of why interaction takes
place), mostly in complex technical and mathematical contexts
(referring to the question of where interaction takes place),
This changed fundamentally in the third stage of informatics’
and computer science’s investigation of interaction, beginning in
the late 1980s and resulting in the reference to HMI instead of
HCI: From now on interaction was not only analyzed from a
user-centered perspective anymore. Rather, issues of interactive
and responsive communication and dialogue between humans
and machines came to the fore. No more were informatics and
computer science solely concerned with questions of computers’
disposition to interaction in view of the user’s goals and
intentions, but also with issues of human disposition to
interaction with computers and machines as well as with
(ongoing) processes of interaction. Furthermore, HMI now
increasingly adhered to an ontic logic (i.e., “computers [and
machines] simulate environments and tools to engage these
environments” (Brey, 2005: p. 384). Analysis of such new forms
of HMI thus involves investigating “the meaning of […] actions,
[…] goals and […] intentions. (It cannot be ‘just the data’.)”
(Müller, 2011: p. 4). This is since in the third stage
[n]ew interactive environments are responsive, active,
sensitive, and in a constant dialog with people in the
environment. The environments themselves are in some
sense becoming more agential and goal driven. Because
interactivity is understood here as requiring agency of some
sort, interactivity is not only about being reactive and
responsive but also about pushing reality in a certain
direction. (Janlert and Stolterman, 2017: 118).
The disposition to as well as (ongoing) processes of such new
forms of interaction, according to Janlert and Stolterman, depend
on several parameters such as agency, pace or time, independence, receptivity, predictability, and enforcement, which now
come into focus of informatics and computer science. With a
view to the four dimensions of the SMPC model, users and
computers were increasingly understood as symmetrical subjects
of interaction, interacting in the modes of communication and
dialogue for the purposes of developing and solving certain
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2023)10:551 | https://doi.org/10.1057/s41599-023-02060-8
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-023-02060-8
problems as well as stimulating and changing environments
within a broad range of different contexts.
“Interaction” in game theory. Another important reference
point for debates about HMI is game theory. Dating back to the
proof of the min-max theorem by John von Neumann (1928),
modern formal game theory serves manifold fields of application,
e.g., in economics (Laffont, 1997), sociology (Swedberg, 2001),
philosophy (de Bruin, 2005), biology (Tomlinson, 1997), and
particularly in HMI (Li et al., 2019). As a mathematical theory, it
models decision-making situations in which rational participants
interact with each other. Such interactive decision problems or
games involve “two or more individuals making a decision in a
situation where the payoff to each individual depends (at least in
principle) on what every individual decides” (Webb, 2007: p. 61).
The primary goal of game theory is to derive rational decision
behavior in such decision problems, i.e., social conflict situations.
With a view to our SMPC model, interaction subjects are
generally understood in terms of rational decision-makers, who
interact in the mode of adopting strategies. That is, interaction
subjects implement plans of action that they (a) have for every
circumstance they can observe in a given decision problem, and
that (b) have some effect on at least one other interaction subject.
The purpose of interaction consists of maximizing rewards
depending on the specific type of decision-making problem, i.e.,
the specific context (Osborne, 2011: p. 4). As regards HMI, game
theoretical approaches are of particular interest, because they may
help to understand fundamental processes as well as outcomes of
(strategic) interactions between humans and machines and, thus,
can be used to answer questions of, e.g., control or mutual
“understanding” in such interactions (cf., e.g., Li et al., 2019).
“Interaction” in sociology. Apart from mathematical game
theory, both sociological and philosophical concepts of interaction are often referred to with a view to the understanding and
designing of HMI. In sociology, interaction is one of the key
concepts being used to explain a broad variety of social phenomena on different societal levels. Its fundamental role in
sociological theory derives from the fact that concepts of interaction can explain heterogeneous phenomena located on different
societal levels (individual, organizational, and societal), like, e.g.,
interpersonal relations, the structuring, and reproduction of social
situations and institutions as well as societal cohesion and
transgenerational transfer of knowledge. For a long time, the use
of “interaction” was, however, restricted to relations between (at
least two) humans. However, with the rise of Science and Technology Studies (STS), “interaction” started to cover phenomena of
both human–human interaction and HMI.
As the concept of interaction is crucial for every sociological
theory, a variety of definitions can be identified throughout the
history of sociological thinking (Bales, 1950; Hall, 1966; Goffman,
1967; Parsons and Ebinger, 1968). Theories differ regarding the
preconditions of interaction or interactivity, which an entity must
satisfy to be recognizable as a potential interaction partner. These
preconditions implicitly or explicitly express different ideas of
humans (or subjectivity). The variety of theories can be roughly
categorized according to six criteria. 1. Intelligence: Some theories
assume that only rational beings are capable of interaction and
argue that interaction requires intelligence or the potential to
mutually recognize interactive counterparts (Müller, 2011). 2.
Intentionality: another type of thinking connects interaction to
the ability of mutual understanding of goals and intentions,
presupposing human beings as beings characterized by goals and
intentions (Müller, 2011). 3. Embodiment: according to this
line of thinking interaction is closely tied to embodiment (and
ARTICLE
intentionality). Therefore, only purposeful moving beings (and,
hence, beings with a body and goals) can meet the prerequisites of
interaction (Vannini, 2016). 4. Adaptability: some theories stress
the importance of a shared common meaning and linguistic
understanding, which they deem necessary for the potential to
adapt one’s own behavior to the behavior of potential interaction
partners (Krappmann, 1998). 5. Symbolic exchange: in the
tradition of symbolic interactionism, interaction is defined by
mutual exchange of interpretation, which is regarded as the
ground of socialization (Blumer, 1986). 6. Intensity and
friendliness: in his Formal Theory of Interaction, Herbert A.
Simon (1952) develops a mathematical formulation of George C.
Homan’s approach to human interaction as presented in his
seminal theory of human group behavior. Simon defines
interaction over time as a function of the level of friendliness
among group members (over time), the amount of activity carried
on by members within a group (over time) as well as the amount
of activity imposed on the group by its environment (over time).
Hence, interaction is formalized as I ðt Þ ¼ a1 F ðt Þ þ a2 Aðt Þ
(where F(t) is a function denoting friendliness over time, A(t) is a
function denoting the activity imposed on a group over time, and
a1 as well as a2 are variables specifying the level of friendliness, or
the amount of activity, respectively).
Regarding our SMPC model and the question of interaction
subjects (who interacts?), the sociological concept of interaction
for a long time was restricted to humans. With the upcoming of
Bruno Latour’s Actor-Network-Theory (ANT) (Latour, 2005) and
STS, however, the concept became broader with also concerning
interaction between humans and machines or rather between
human and non-human entities. Latour (2005), as well as Castells
(2010) thus changed the theoretical perspective towards a
relational perspective, arguing that the physical and social world
should be understood as a complex arrangement of intertwined
relations between different entities, which do not have to be
thought of as structurally identical to be attributed agency and
interactivity. Following this line of argument, Latour and Castells
rewrite the history of humans and technology by pointing out
that the relations between humans and technological systems,
humans and animals as well as humans and their natural
environment are of the same importance for sociological analysis
as are interpersonal relationships. Following this line of thinking,
Werner Rammert (2002), a protagonist of German STS, raised the
question of technological agency by following a functionalist
approach rather than a materialistic one. By focusing on the
function of technology, Rammert demonstrates the ability of
technology to act and interact on different societal levels thereby
shaping situational as well as institutional and societal structures.
Regarding the question of how interaction is taking place,
different modes of interaction can be distinguished. Whereas
Jensen (1998) argues that interactivity and interaction are bound
to mutual adaption of behavior and action, Blumer (1986)
emphasizes the importance of mutual exchange of interpretation.
From the latter point of view, human interaction is mediated by
using symbols and signification, by interpretation, or by
ascertaining the meaning of others’ actions. The tradition of
symbolic interactionism accentuates the importance of symbolization and distinguishes two types of interaction by introducing
the difference between symbolic and non-symbolic interaction.
The former occurs if an interaction partner has already
interpreted the actions of their counterpart, i.e., interaction takes
place in mutual role-taking as well as interpretation of behavior.
On the contrary, the latter is exemplified in spontaneous, reactive
responses to another’s actions. Interdependence of action is also
crucial for Starkey Duncan (1989), who argues that a process is
interactive if at least two interdependent individuals, where
interdependence is identified as a state of reciprocal awareness,
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2023)10:551 | https://doi.org/10.1057/s41599-023-02060-8
5
ARTICLE
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-023-02060-8
i.e., every interaction partner is a) aware of the presence of the
other and b) assumes that the other is aware of his/her presence.
Regarding modes of HMI, Philip Hayes and Reddy (1983) point
out that interaction takes place if fragmented input can be parsed
and combined, if abilities and limitations, actions and motives
can be explained, and if a dialog can be started and perpetuated
by keeping track of the focus of attention.
With a view to the purposes and contexts of interaction (why is
interaction taking place? Where is interaction taking place?), the
concept of interaction points to the construction and reproduction of social structures on different societal levels. Hence,
interaction is a multi-stable process, whose purpose is dependent
on the specific context in which interaction is taking place.
Following Bahr and Stary (2016), interaction primarily has to be
considered as a process of exchanging material and immaterial
goods between acting parties (biological or technical entities)
embodied in a certain context. This emphasis on context, which is
shared by ANT, for instance, opens the analytical perspective for
the importance of differentiation and the impossibility of
generalizing hypotheses about possible outcomes of interaction.
The range of possible outcomes includes the creation of order and
meaning (symbolic interactions) as well as the construction of
organizational and societal structures established through behavioral and cognitive schemes that result from repeated interactions and influence future events as patterns and expectations.
Hence, the specific characteristics of each interaction, context,
and partner are to be identified, as the very same interaction may
lead to different results depending on the conditions of its context
(e.g., an interaction between a robot and a resident of a nursing
home may lead to another result than the interaction between a
customer of a supermarket and the same robot). These differences
must be taken into consideration when developing or evaluating
concrete instances of HMI.
Summing up the different theoretical strings, sociological concepts
of interaction can be differentiated based on the preconditions, which
interaction partners must satisfy. While most theories exclusively
reserve the concept of interaction for human exchange processes and
their effects, with the rise of ANT and STS, a different approach was
taken, which is of particular relevance for the analysis and evaluation
of HMI. By extending the range of possible interaction partners to the
realm of machines (in the broadest sense of this concept), ANT and
STS enable using “interaction” as a heuristic concept to explain the
technologically enabled conditions of modern sociality. Therefore,
these approaches seem especially fruitful for understanding modern
societal structures and social relations, whereas interaction concepts
derived from the interpersonal sphere should only be cautiously
transferred to contexts of HMI, since their preconditions often are
not satisfied by technical systems.
“Interaction” in philosophy. In philosophy of technology, the
development of the concept of interaction stretches from the preindustrial age through industrialization to the present day: in the
pre-industrial age, the focus was on humans in the “mirror of
their machines” (Meyer-Drawe, 1996), and thus on their comparison. During industrialization, this focus shifted to the social
consequences of an increasing integration of machines into
everyday life: henceforth, it was no longer primarily about
human–machine comparison, but about human–machine interaction (Gehlen, 2007; Müller and Liggieri, 2019). With the rapid
development of new types of technologies, philosophers of technology were increasingly asking questions about concrete interactions between humans and machines. This focus is reflected by
situated and contextual considerations of individual settings of
HMI, which take modes and contexts as central aspects of
interaction (how does interaction take place? Where does
6
interaction take place?). This becomes particularly clear in recent
approaches in the field of post-phenomenology as well as in
technoscience. Both Peter-Paul Verbeek and Karen Barad, as
contemporary key thinkers of the two fields, deal critically and
productively with the notion of interaction by focusing on the
aspect of the “between” in HMI.
In contrast, Shaun Gallager, a well-known thinker of
enactivism, introduces his Interaction Theory (IT) in which he
sheds light on the very context of concrete interactions based on
the thesis that every form of interaction can only be determined
by its situatedness. Finally, approaches like Luciano Floridi’s
analytically oriented Philosophy of Information must be mentioned, which focuses on HMI from an ethical and epistemological perspective. In the following, these four prominent positions
and research fields are presented and discussed in more.
Verbeek, who is one of the leading thinkers from the ranks of
post-phenomenologists, is particularly concerned with the
phenomenon of “human technology mediation” (Verbeek,
2005). Thus, he does not directly focus on the concept of
interaction, but rather on a critical examination of concrete
situations in which certain relations between humans and
technologies are revealed. For Verbeek, interaction therefore
names only one of several possible relations between humans and
technology. With this, he draws attention to the fact that in
concrete practical settings, it is through interaction that the
involved entities first appear as what they are: humans and
machines “are not pre-given entities but rather […] mutually
shape each other in the relations that come about between them.”
(Verbeek, 2015: p. 28). In this respect, post-phenomenological
analyses of HMI are concerned, first, with the mode of interaction
and, second, with the concrete, practical context of HMI.4
In the field of technoscience, it is above all Karen Barad (2007)
who is concerned with a critical discussion of the relationship
between humans and technologies (or more generally: objects).
Her starting point is a critique of the subject position within
philosophical traditions. To support this, she develops the
concept of intra-action—a term that is used to replace the
concept of “interaction”, which presupposes pre-established
bodies that participate in action with each other. In contrast,
intra-action understands agency not as an inherent property of an
individual or human to be exercised, but as a dynamism of forces
in which all designated entities are constantly exchanging and
diffracting, influencing, and working inseparably. Intra-action
thus “[…] acknowledges the impossibility of an absolute
separation or classically understood objectivity, in which an
apparatus (a technology or medium used to measure a property)
or a person using an apparatus are not considered to be part of
the process that allows for specifically located ‘outcomes’ or
measurement” (Stark, 2016). Rather, “[…] ’individuals’ would
only exist within phenomena (particular materialized/materializing relations) in their ongoing iteratively intra-active reconfiguring” (Barad and Kleinman, 2012: p. 76).
In this respect, Barad, like Verbeek, on the one hand, focuses on
the “in-betweenness” of humans and machines, while, on the other,
she is interested in the concrete ways in which agents appear in
interactions. With a view to our SMPC model, here, too, the focus on
mode and context is at the center, which highlights the accent certain
contemporary philosophies of technology place on HMI when
dealing with the benefits and limits of the notion of interaction.
Although Shaun Gallagher does not focus on HMI explicitly,
he nevertheless defines interaction in such a way that it can be
used for the analysis of HMI. Gallagher’s concept of interaction is
situated in an enactivist perspective, implemented in what
Gallagher—in distinction from Theory-Theory and Simulation
Theory—calls IT. For him, “IT emphasizes the importance of
context and circumstance, and the role of communicative and
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2023)10:551 | https://doi.org/10.1057/s41599-023-02060-8
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-023-02060-8
narrative practices” (Gallagher, 2020: p. 100]. Hence, Gallagher—
in analogy to Verbeek and Barad—points to the specific mode
and context of interaction, insofar as the mode is expressed in the
focus on communicative and narrative practices and the context
in the focus on the situatedness of interactions: interaction comes
into view as “a mutually engaged co-regulated coupling between
at least two autonomous agents”, distinguishing between “(a) the
co-regulation and the coupling mutually affect each other and
constitute a self-sustaining organization in the domain of
relational dynamics”, and “(b) the autonomy of the agents
involved is not destroyed, although its scope may be augmented
or reduced.” (Gallagher, 2020: p. 99).
In contrast to this, Luciano Floridi’s analytical account of
interaction is, at least, two-fold: first, in his Ethics of Information,
he states that “interactivity means that the agent and its environment
(can) act upon each other” (Floridi, 2013: p. 140). With a view to our
SMPC model, he understands agents, i.e., autonomous, adaptable,
and situated systems as interacting with their environment to
transform or produce certain effects upon each other via
information exchange. Hence, according to Floridi, not only human
agents may interact, but also artificial agents (insofar they are
autonomous and adaptable), whereby both can act as moral agents
(“if and only if they are capable of morally qualifiable action”
(Floridi, 2013: p. 147)). This, of course, has important ethical
implications for at least certain contexts of interaction.
Apart from this ethical perspective on interaction, in his Logic of
Information, Floridi (2019) presents a second understanding of
interaction aiming at providing us with some epistemic criterion for
existence. In this regard, he defines internal interaction as the
interplay and interlock of mathematically describable structures, and
external interaction as our relationship (as agents) with such
structures. Hence, external interactions would provide us with some
metaphysical criterion for existence (“ghosts do not exist […],
because there is no LoA [level of abstraction] at which you can
interact with them” (Floridi, 2019: p. 94)). In this understanding,
according to our scheme, “we” (as agents) interact with mathematically describable structures in our environment via information
exchange to account for facts or beliefs about certain facts.
In summary, in the fields of (post-)phenomenology, technoscience, and enactivism, an awareness of the limits and
possibilities of the notion of interaction is currently emerging.
In these fields, thinkers turn to the concept of interaction in a
fruitful and productive way, embedding it in practical and
situated contexts to address particular modes and contexts of
interaction. Against this background, central yields are, for
example, the critical reflection on the relationship between
humans and technology or machines, which highlights a mutual
dependence between humans and non-humans or technology
and emphasizes the concrete context in which “interaction”
appears as a meaningful concept. The analytical tradition, on the
other hand, aims at developing a fully fledged concept of
interaction regarding the four dimensions of the SMPC model,
which differ from an ethical and epistemological perspective.
“Interaction” in psychology and cognitive science. In psychology, the term “interaction” is used and studied in different subdisciplines (e.g., social psychology, differential psychology,
developmental psychology, ergonomics, industrial, or organizational psychology), resulting in a wide range of differing understandings. In general, as regards the first and fourth dimension of
our SMPC model (who interacts? Where does interaction take
place), psychological theories refer to two (or more), variables,
psycho-physiological or interpersonal states, constructs, systems,
environmental conditions, persons or behaviors interacting in
specific social contexts (Dix et al., 2003; Bolis and Schilbach,
ARTICLE
2020). However, psychological theories focus more on potential
modes and purposes of interaction (how does interaction take
place? Why does interaction take place?) as well as on possible
psychological consequences of interaction.
The general question of how interaction takes place is answered
in different ways, e.g., in the sense of being closely related to each
other, of exchanging information, of interdependency, of reciprocal steering or control, of a joint influence on something third, or
of collective transformations of the world (Hasson and Frith, 2016;
Bolis and Schilbach, 2020). In general, interaction is understood as
a dynamic process that relates to continuous mutual adaptation, a
dynamic coupling of interacting parties, and a related development
of complementary behavior (Hasson and Frith, 2016).
A wide variety of proposals exist in psychological literature
regarding concrete modes of interaction (e.g., by gesture, facial
expression, communication of content, cognition, emotion,
intimacy, etc.). As regards HMI, cognitive aspects or mental
states (thinking, learning, remembering, attention, perception,
planning, decision-making, etc.) play a particularly important role
in the context of psychology and cognitive science (Sharp, 2019;
Cross and Ramsey, 2021): theories of cognition (e.g., mental
models, information processing, distributed cognition, embodied
interaction) are deemed highly important for studying, developing and designing interactive machines (Sharp, 2019). The same
holds true for emotions, which are another important component
of the psychological understanding of interaction and of research
in psychology related to possible modes of interaction: on the one
hand, emotional responses of users to machines or their design
significantly influence concrete HMIs. On the other hand,
interaction with machines can be used to influence people’s
emotions in various ways (cf., e.g., persuasive technologies)
(Sharp, 2019).
Modes of social interaction, e.g., in collaboration, communication, and coordination, are also highly important with a view to
HMI. In psychology, social interaction is generally understood as
a two-way process. The preconditions for such processes,
however, not only complicate the development of HMI, but also
the psychological understanding of human interaction with
machines. The most prominent approaches to describe the
preconditions of social interaction (e.g., common ground,
perspective taking, and Theory of Mind (ToM)) assume that
humans are equipped with direct and implicit knowledge of other
humans (regarding, e.g., certain capacities of other humans, their
physiological needs, etc.). A key difficulty for HMI in comparison
to human–human interactions results from the fact that at least
some aspects of social interaction between humans are never
made explicit (Krämer et al., 2011). Furthermore, even if all
aspects of human–human interaction could be made explicit,
implementing the underlying rules or knowledge would not
suffice to establish successful mentalization in machines and,
hence, to allow developing forms of HMI that are similar to
human–human interaction (Frith and Frith, 2003; Krämer et al.,
2011).5
Another model for unfolding interaction that is difficult to
apply to machines is expressed in social penetration theory
(Krämer et al., 2011; Fox and Gambino, 2021). According to this
account, a stepwise reciprocal self-disclosure is necessary to
develop relationships. However, when machines share information, such information is not based on personal values,
experience, or self-image (Fox and Gambino, 2021). Thus, from
a psychological point of view, there is much to suggest that the
understanding of social interaction, i.e., the how of interaction
between humans, is not fully transferable to the understanding of
interaction between humans and machines.
For the psychological conceptualization of interaction between
humans and machines and for the question of how interaction
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2023)10:551 | https://doi.org/10.1057/s41599-023-02060-8
7
ARTICLE
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-023-02060-8
with machines takes place, it may therefore be more important to
understand, recognize, and implement different types of interactions. For instance, users may give instructions to a system
(typing commands, gesturing, etc.) (instructing type of interaction), enter dialogs with systems (conversing type), move through
(virtual) spaces (exploring type), or respond to system-initiated
interactions (responding type) (Sharp, 2019).
A central purpose of human–human interaction (why is
interaction taking place?) is to form, maintain and shape
relationships. In psychology, this aspect has also been addressed
regarding HMI, both as a design goal and with a view to the
psychological effects of relationships between humans and
machines. The motivation for forming relationships generally is
located in the human need to belong (Krämer et al., 2011; Fox and
Gambino, 2021): humans as social beings seek the company of
others, interact with others, and form various forms of
interactional relationships (family, couple relationships, friendship, etc.). Against this background, interactions with machines in
principle may also lead to relationships between humans and
machines. More interesting from a psychological perspective,
however, is the question under which exact conditions HMI may
lead to the formation of relationships. Conditions that are
discussed in this context are, among others, a certain degree of
attractiveness, similarity, or reciprocal liking (Krämer et al., 2011;
Fox and Gambino, 2021).
Other aspects regarding possible purposes of interaction are
captured by social exchange theory, equity theory, or investment
theory (Krämer et al., 2011; Fox and Gambino, 2021). A basic idea
of social exchange is that people need to exchange different goods
(tangible or abstract and with diverse functions) and cooperate to
survive. Cost-benefit considerations play a crucial role here.
Accordingly, humans would prefer relationships offering more
advantages than disadvantages or costs (compared to other
relationships), relationships in which they have invested more
compared to others, or relationships in which partners are equal in
the exchange process. The longer and closer a relationship is,
however, the less equality seems to be a priority. These
considerations are also applied to human interactions with
machines, analyzing how people use cost-benefit trade-offs to enter
satisfying interactions and relationships with robots, for example.
To summarize, according to the psychological understanding,
interaction is not an exclusively human phenomenon. Thus, not
only do humans interact with each other, but also, for instance,
with certain human features (behavior, gestures, cognition,
emotions, etc.), systems, environmental conditions, or machines.
Furthermore, interaction types play an important role in
describing and understanding different modes of interaction. In
this context, the cognitive, emotional, and social features of
humans are decisive for the psychological description and
analysis of HMI modes. If, however, social interaction between
humans and machines is understood by reference to theories of
human–human interactions, the limits of transferability quickly
become apparent.
“Interaction” in media studies and communication science.
With the entry of the personal computer into everyday processes,
media and communication studies experienced a radical upswing. In
the 1980s, for example, the field of German Media Theory was
established, which—under the auspices of Friedrich Kittler—identified the operations of transmission, storage, and processing as basic
media functions (Kittler, 1993). While German media studies
developed out of the humanities (more precisely, literary studies)
and have a historical and theoretical focus, communication studies
are more empirically oriented, i.e., they study the use and application
of media technologies and conduct media research. Especially in the
8
latter approach, concepts of interactivity (referring both to processes
of interaction and to their necessary conditions) play a central role.
Since Sheizaf Rafaeli’s (1988) account of reactive communication, for
instance, interaction has been understood as a way of thinking about
communication, where it was originally assumed as an attribute of
face-to-face conversation (Isotalus, 1998; Rafaeli and Sudweeks,
1998) which then was extended to mediated communication settings
and, as such, must be distinguished from social interaction (Bucy,
2004). In the latter regard, interaction in media studies and communication science “is used as a broad concept that covers processes
that take place between receivers on the one hand and a media
message on the other” (Jensen, 1998: p. 188). In this context, Bucy
(2004) distinguishes two main types of interaction concepts: first,
concepts that focus on human interaction with certain (e.g., online)
content, and address the control that users exercise over its selection
and presentation (cf., e.g., Steuer, 1995; McMillan, 2002; StromerGalley, 2004); and second, interaction processes that involve personto-person conversations mediated by technology (cf., e.g., Massey
and Levy, 1999). Furthermore, some approaches seek to combine
these two types by focusing on the aspect of control in mediated
person-to-person interaction processes (cf., e.g., Williams et al., 1988;
Neuman, 1991). Common to these message-centered approaches is
their strong focus on users, or more generally, their focus on human
senders and recipients. With a view to our SMPC model of “interaction”, in message-centered approaches, users and media are
deemed asymmetrical subjects of interaction with a focus on users
(referring to the question of who interacts), who select, present, and
control certain content (referring to the question of how interaction
takes place) for the purpose of communication (referring to the
question of why interaction takes place) in mediated contexts
(referring to the question of where interaction takes place).
In contrast, structural approaches consider the technological
attribute or media feature, which allows users to talk to other
users, engage with or manipulate media, or influence its content.
Here, interactivity is located as a property of technology or media.
In this regard, Jensen, e.g., states that interactivity would be “a
measure of a media’s potential ability to let the user exert an
influence on the content and/or form of the mediated communication” (Jensen, 2008: p. 129). With a view to this, Jensen
distinguishes three principal ways of defining interaction in
media studies and communication science: first, approaches that
define interactivity through prototypic examples (referring, e.g.,
to the telephone, audio conferencing systems, or email). Second,
approaches defining interactivity through certain criteria deemed
necessary for a reciprocal dialog between users and systems. And
third, understandings of interaction as a continuum that can be
present in varying degrees with reference to 1 to n dimensions
(covering, for instance, the degree of choices available, the degree
of modifiability, the quantitative number of the selections, and
modifications available, or the degree of linearity or nonlinearity). With a view to our SMPC model, in structural
approaches users and media are understood as asymmetrical
subjects of interaction with a focus on media and technology,
which enables users to influence certain content for the purpose
of communication in mediated contexts.
Apart from structural approaches, perceptual approaches (cf.,
e.g., McMillan, 2002) consider user perceptions as the unit of
measure: the degree of interactivity, which is now presumed to
have variable effects, is reflected in the extent to which users
subjectively experience interactivity. As for the SMPC model, in
perceptual approaches, users and media are deemed asymmetrical
subjects of interaction with a focus on users and their experience
of interaction with media and technology for the purpose of
communication in mediated contexts.
Kiousis, for instance, combines a structural and perceptual
approach when postulating that “[i]nteractivity can be defined as
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2023)10:551 | https://doi.org/10.1057/s41599-023-02060-8
Scientific discipline
Subjects of interaction: who
interacts?
Modes of interaction: how does
interaction take place?
Purposes of interaction: why
does interaction take place?
Informatics/Computer
Science
(a) first stage:a user and computer
(user-centered, asymmetrical)
(b) second stage: user and
computer (user-centered,
asymmetrical)
(c) third stage: human and machine
(symmetrical)
rational decision-makers
(a) via information transfer
(b) via information transfer through
user interfaces
(c) via communication/ dialog
(a) to solve pre-formulated
problems
(b) to solve pre-formulated
problems
(c) to develop and solve
problems, to simulate and
change environments
to maximize rewards
Sociology
traditional approaches: humans and
humans
modern approaches: humans and
humans, humans and non-human
entities (e.g., machines)
Philosophy
(a) post-phenomenology: depend
on mode and context
(b) technoscience: arise from
modes and contexts of intraaction
(c) enactivism: autonomous agents
(d) analytical philosophy: da) ethics:
agents and environments
db) epistemology: agents and
mathematical describable
structures
-variables
-psycho-physiological or
interpersonal states
-constructs
-systems
-environmental conditions
-persons
-behaviors
(a) message-centered approaches:
user and media (usercentered, asymmetrical)
(b) structural approaches: user and
media (media-centered,
asymmetrical)
(c) perceptual approaches: user and
media (user-centered,
asymmetrical)
-via mutual adaption
-via mutual exchange of
interpretation through symbols
and signification
-via mutual awareness
-via perpetuated dialog
(a) via mutual shaping in specific
relations
(b) via exchanging, diffracting,
influencing, working
inseparably
(c) via mutual co-regulation and
coupling through
communicative and narrative
practices
(da) via information exchange
(db) via information exchange
-via continuous mutual adaptation
-via dynamic coupling and a related
development of complementary
behavior
Game theory
Psychology/cognitive
science
Media studies/
communication science
aAs
via adopting strategies
(a) via selecting, presenting, and
controlling content
(b) via influencing content
(c) via user experience
-to exchange material or
immaterial goods
-to create order and meaning
-to construct organizational
and societal structures
Contexts of interaction:
where does interaction
take place?
(a) in technical and
mathematical contexts
(b) in technical and
mathematical contexts
(c) in a broad range of
different (e.g., scientific,
or social) contexts
in specific decision-making
problems
in specific social contexts
(a) depends on mode and
context
(b) depends on mode and
context
(c) to constitute selfsustaining organizations
(da) to transform or
produce certain effects
upon each other
(db) to account for facts
or beliefs
-to form, maintain and shape
relationships
(a) in specific and concrete
practical settings
(b) in certain phenomena
(c) in dynamic relational
situations
(da) in specific contexts
(db) in an environment
(a) to communicate
(b) to communicate
(c) to communicate
(a) in mediated contexts
(b) in mediated contexts
(c) in mediated contexts
-in specific social contexts
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-023-02060-8
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2023)10:551 | https://doi.org/10.1057/s41599-023-02060-8
Table 1 Interaction in the disciplines.
regards the first stage, the elements of human–computer interaction must be understood as mostly ideal goals, which we found to motivate the overarching questions of engineering, construction, and effectiveness of algorithms and computer predecessors.
ARTICLE
9
ARTICLE
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-023-02060-8
Interactivity
Disposition for
Interaction (Conditions for
Interaction)
(Specific) Interaction-Enabling
Properties (Interactability)
informatics/computer
science
sociology
philosophy
enactivism
analytical philosophy
psychology/cognitive
science
media studies/
communication science
(Ongoing) Process of
Interaction
informatics/computer
science
game theory
sociology
philosophy
psychology/cognitive
science
media studies/
communication
science
Propensity to Interaction
Engagement (Interactiveness)
informatics/computer
science
second stage
third stage
sociology
ANT
STS
philosophy
enactivism
analytical philosophy
media
studies/communication
science
structural approaches
Fig. 2 Interactivity, interaction, interactability, and interactiveness in the disciplines. Relation of terms connected with “interaction” and their reference
in the disciplines.
the degree to which a communication technology can create a
mediated environment in which participants can communicate
(one-to-one, one-to-many, and many-to-many), both synchronously and asynchronously, and participate in reciprocal message
exchanges […]. Regarding human users, it additionally refers to
their ability to perceive the experience as a simulation of
interpersonal communication and increase their awareness of
telepresence” (Kiousis, 2002: p. 372).
In traditional communication science, however, structural
approaches were discussed quite critically. Ha and James (1998:
p. 461), for instance, state that “[i]nteractivity should be defined
in terms of the extent to which the communicator and the
audience respond to, or are willing to facilitate, each other’s
communication needs” and, hence, claim message-centered
approaches as the only plausible accounts of interaction.
Schumann et al. justify this claim by postulating that “[u]
ltimately it is the consumer’s choice to interact, thus interactivity
is a characteristic of the consumer and not a characteristic of the
medium. The medium simply serves to facilitate the interaction”
(Schumann et al., 2001: p. 45).
While all these perspectives have a strong focus on human
users and social interaction, German Media Theory focuses more
on technology-immanent interaction processes, with human
users being only one actor among others. Bernhard Siegert, for
example, examines the sign practices used for representations in
media systems and thus investigates the beginning of electric
media (Siegert, 2003). Friedrich Kittler, on the other hand, in his
seminal study Aufschreibesysteme 1800/1900 examines “the
network of techniques and institutions […] that allow a given
culture to address, store, and process relevant data” (Kittler, 1985:
10
p.519), focusing on technologies as diverse as the letterpress, the
phonograph, or the gramophone. Here, it is not the human user
who is at the center of interaction processes, but the diverse
operations of medial mediation.
Conclusion
The terms of interaction and interactivity are used in many ways
in debates on HMI. Ultimately, however, this results in the
concept of interaction being vague and ambiguous, which makes
it difficult to reasonably discuss questions of ethics, politics,
engineering, and the sciences regarding HMI. Against this
background, we analyzed the different meanings attached to
interaction in the scientific disciplines relevant to debates on HMI
to provide a basis for a fruitful intra- and particularly interdisciplinary discourse on HMI. For this purpose, we introduced
the SMPC model, which alludes to interaction as a fourdimensional concept referring (1) to certain subjects (i.e., to the
question: who interacts?), (2) to modes of interaction (how do
these subjects interact?), (3) to purposes of interaction (why, or:
for what reasons is interaction taking place?), and (4) to certain
contexts (where, or: under what conditions is interaction taking
place?), and is intricately connected to the terms of “interactivity”,
“interactability”, as well as “interactiveness”. In view of this, our
analysis showed a broad range of understandings of interaction in
the disciplines of informatics and computer science, game theory,
sociology, philosophy, psychology, and cognitive science as well
as media studies and communication science (Table 1). Moreover, manifold positions regarding the connected terms became
obvious (Fig. 2).
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2023)10:551 | https://doi.org/10.1057/s41599-023-02060-8
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-023-02060-8
This variety of understandings trivially results from the different research subjects and foci of the disciplines. Therefore, it
seems impossible to develop one correct definition of interaction
or interactivity. At the same time, this highlights the need for a
basic understanding of the respective meaning(s) used in concrete
debates on issues of HMI. For, after all, the underlying understanding of interaction influences, e.g., normative analyses or the
development of recommendations for dealing with HMI. Even if
no definitive understanding can be found to be consistently used
across the disciplines, it is useful to be aware of the different
disciplinary approaches to the phenomenon of interaction (in
HMI). In this way, misunderstandings or (normative) decisions
made on the wrong conceptual basis can be avoided.
Data availability
Data sharing is not applicable to this article as no datasets were
generated or analyzed during the current study.
Received: 21 November 2022; Accepted: 24 August 2023;
Notes
1 ANT actor-network-theory, AI artificial intelligence, BCI brain–computer interface,
HCI human–computer interaction, HMI human–machine interaction, IT interaction
theory, ML machine learning, SMPC-model (of interaction) Subject-Mode-PurposeContext-model (of interaction), STS science and technology studies, ToM theory
of mind.
2 Janlert and Stolterman have been criticized for having left out the key discussions of
agency, affordance, and possibility spaces in developing their taxonomy. In our
opinion, however, each of these concepts can in principle be used consistently within
the framework of this taxonomy (affordance and possibility spaces, for instance, refer
to the terms of interactability and interactiveness).
3 Of course, Alan Turing (1950) had already speculated about the conditions,
possibilities, and limitations of human–machine communication and dialogue.
4 While Don Ihde is also an important reference in discussions about human-technology
relations, we do not relate to his usage of schemes of these relations, since our
taxonomy tries to find a basis to compare the concepts of interaction in different
disciplines.
5 Nevertheless, there is already research on implementing concepts such as ToM in HMI
(Krämer et al., 2011).
References
Bahr GS, Stary C (2016) What is interaction science? Revisiting the aims and
scope of JoIS. J Interact Sci 4(2):1–19. https://doi.org/10.1186/s40166-0160015-5
Bales RF (1950) Interaction process analysis. Addison-Wesley, Boston/MA
Barad K, Kleinman A (2012) Intra-actions. Mousse Magazine 34:76–81
Barad K (2007) Meeting the universe halfway. Quantum physics and the entanglement of matter and meaning. Duke University Press, Durham/NC
Barnes SB (1997) Douglas Carl Engelbart: developing the underlying concepts for
contemporary computing. IEEE Ann Hist Comput 19(3):16–26. https://doi.
org/10.1109/85.601730
Biddle JB (2022) On predicting recidivism: epistemic risk, tradeoffs, and values in
machine learning. Can J Philos 52(3):321–341. https://doi.org/10.1017/can.
2020.27
Blumer H (1986) Symbolic interactionism: perspective and method. University of
California Press, Berkeley/CA
Bolis D, Schilbach L (2020) ‘I interact therefore I am’: the self as a historical
product of dialectical attunement. Topoi 39(3):521–534. https://doi.org/10.
1007/s11245-018-9574-0
Brey P (2005) The epistemology and ontology of human-computer interaction.
Minds Mach 15(3-4):383–398. https://doi.org/10.1007/s11023-005-9003-1
Bromley AG (1982) Charles babbage’s analytical engine, 1838. IEEE Ann Hist
Comput 4(3):196–217. https://doi.org/10.1109/85.728228
Brooks Jr FP, Ouh-Young M, Batter JJ et al. (1990) Project GROPE. Haptic displays
for scientific visualization. ACM SIGGraph Comput Graph 24(4):177–185.
https://doi.org/10.1145/97880.97899
Bruin B (2005) Game theory in philosophy. Topoi 24:197–208. https://doi.org/10.
1007/s11245-005-5055-3
ARTICLE
Bucy EP (2004) Interactivity in society: locating an elusive concept. Inf Soc
20(5):373–383
Bucy EP, Tao C-C (2007) The mediated moderation model of interactivity. Med
Psych 9(3):647–672. https://doi.org/10.1080/15213260701283269
Bush V (1945) As we may think. Atl Monthly 176(1):101–108
Buxton W (2010) A touching story: a personal perspective on the history of touch
interfaces past and future. Sid Symposium Digest of Technical Papers
41(1):444–448. https://doi.org/10.1889/1.3500488
Buxton W, Hill R Rowley P (1985) Issues and techniques in touch-sensitive tablet
input. Proceedings of the 12th Annual Conference on Computer Graphics
and Interactive Techniques. pp. 215–224
Castells M (2010) The information age: economy, society, and culture. Vol. II: the
information age. Blackwell, Malden/MA
Charman-Anderson S (2015) Ada Lovelace: victorian computing visionary. Ada
User J 36(1):35–41
Cross ES, Ramsey R (2021) Mind meets machine: towards a cognitive science of
human–machine interactions. Trends Cogn Sci 25(3):200–212. https://doi.
org/10.1016/j.tics.2020.11.009
Dimond T (1957) Devices for reading handwritten characters. Proceedings of the
International Workshop on Managing Requirements Knowledge. pp.
232–237
Dix A, Finlay J, Abowd GD et al. (2003) Human-computer interaction. Pearson
Education, London
Duncan S (1989) Interaction, face-to-face. Oxford University Press, Oxford
Ellis TO, Heafner JF, Sibley WL (1969) The GRAIL Project: an experiment in manmachine communications. Rand Corp Santa Monica, Santa Monica
Farwell LA, Donchin E (1988) Talking off the top of your head: toward a mental
prosthesis utilizing event-related brain potentials. Electroencephalogr Clin
Neurophysiol 70(6):510–523. https://doi.org/10.1016/0013-4694(88)90149-6
Floridi L (2013) The ethics of information. Oxford University Press, Oxford
Floridi L (2019) The logic of information: a theory of philosophy as conceptual
design. Oxford University Press, Oxford
Fox J, Gambino A (2021) Relationship development with humanoid social robots:
applying interpersonal theories to human-robot interaction. Cyberpsychol
Behav Soc Netw 24(5):294–299. https://doi.org/10.1089/cyber.2020.0181
Frith U, Frith CD (2003) Development and neurophysiology of mentalizing. Philos
Trans R Soc B Biol Sci 358(1431):459–473. https://doi.org/10.1098/rstb.2002.1218
Gallagher S (2020) Action and interaction. Oxford University Press, Oxford
Gehlen A (2007) Die Seele im technischen Zeitalter: sozialpsychologische Probleme
in der industriellen Gesellschaft. Vittorio Klostermann, Frankfurt/Main
Goffman E (1967) Interaction ritual: essays on face-to-face interaction. Aldine,
Chicago/IL
Gold J (2004) Object-oriented game development. Pearson Education, London
Ha L, James EL (1998) Interactivity reexamined: a baseline analysis of early
business web sites. J Broadcast Electron Media 42(4):457–474. https://doi.org/
10.1080/08838159809364462
Hall ET (1966) The hidden dimension. Doubleday, Garden City/NY
Hasson U, Frith CD (2016) Mirroring and beyond: coupled dynamics as a generalized framework for modelling social interactions. Philos Trans R Soc B
Biol Sci 371(1693):20150366. https://doi.org/10.1098/rstb.2015.0366
Hayes PJ, Reddy DR (1983) Steps toward graceful interaction in spoken and
written man-machine communication. Int J Man-Mach Stud 19(3):231–284.
https://doi.org/10.1016/S0020-7373(83)80050-9
Heide L (2009) Punched-card systems and the early information explosion,
1880–1945. Johns Hopkins University Press, Baltimore/MD
Isotalus P (1998) Television performance as interaction. Nordicom Rev 19(1):175–184
Janlert L-E, Stolterman E (2017) The meaning of interactivity—some proposals for
definitions and measures. Hum-Comput Interact 32(3):103–138. https://doi.
org/10.1080/07370024.2016.1226139
Jensen JF (1998) ‘Interactivity’: Tracking a new concept in media and communication studies. Nordicom Rev 19(1):185–204
Jensen JF (2008) The concept of interactivity-revisited: four new typologies for a
new media landscape. Proceedings of the 1st International Conference on
Designing Interactive User Experiences for TV and Video. pp. 129–132
Johnson J, Roberts TL, Verplank W et al. (1989) The xerox star: a retrospective.
Comp 22(9):11–26. https://doi.org/10.1109/2.35211
Kittler F (1985) Aufschreibesysteme 1800/1900. Wilhelm Fink, München
Kittler F (1993) Draculas Vermächtnis. Technische Schriften. Reclam, Stuttgart
Kiousis S (2002) Interactivity: a concept explication. New Media Soc 4(3):355–383.
https://doi.org/10.1177/146144480200400303
Krämer NC, Eimler S, von der Pütten A et al. (2011) Theory of companions: what
can theoretical models contribute to applications and understanding of
human-robot interaction.? App Artif Intell 25(6):474–502
Krappmann L (1998) Interaktion. In: Endruweit G, Trommsdorff G (eds.) Wörterbuch der Soziologie. UVK Verlag, Konstanz, p. 250
Krueger MW (1983) Artificial reality. Addison-Wesley, Boston/MA
Krueger MW (1977) Responsive environments. Proceedings of the June 13–16,
1977, National Computer Conference. pp. 423–433
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2023)10:551 | https://doi.org/10.1057/s41599-023-02060-8
11
ARTICLE
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-023-02060-8
Krull FN (1994) The origin of computer graphics within General Motors. IEEE
Ann Hist Comput 16(3):40–56. https://doi.org/10.1109/MAHC.1994.
298419
Laffont JJ (1997) Game theory and empirical economics: the case of auction data.
Eur Econ Rev 41(1):1–35. https://doi.org/10.1016/S0014-2921(96)00017-7
Latour B (2005) Reassembling the social: an introduction to actor-network-theory.
Oxford University Press, Oxford
Lee S, Buxton W, Smith K (1985) A multi-touch three dimensional touch-sensitive
tablet. ACM SIGCHI Bull 16(4):21–25. https://doi.org/10.1145/1165385.
317461
Li Y, Carboni G, Gonzalez F et al. (2019) Differential game theory for versatile
physical human-robot interaction. Nat Mach Intell 1:36–43. https://doi.org/
10.1038/s42256-018-0010-3
Licklider JC (1960) Man-computer symbiosis. IEEE Trans Hum Mach Syst
1(1):4–11. https://doi.org/10.1109/THFE2.1960.4503259
Maes P, Mistry P (2009) Meet the sixthsense interaction. Available via https://www.
ted.com/talks/pattie_maes_pranav_mistry_meet_the_sixthsense_interaction
Massey BL, Levy MR (1999) Interactivity, online journalism, and English-language
web newspapers in Asia. J Mass Commun Q 76(1):138–151. https://doi.org/
10.1177/107769909907600110
McKinney SM, Sieniek M, Godbole V et al. (2020) International evaluation of an
AI system for breast cancer screening. Nature 577(7788):89–94. https://doi.
org/10.1038/s41586-019-1799-6
McMillan SJ (2005) The researchers and the concept: moving beyond a blind
examination of interactivity. J Interact Advert 5(2):1–4. https://doi.org/10.
1080/15252019.2005.10722096
McMillan SJ (2002) Exploring models of interactivity from multiple research traditions: users, documents, and systems. In: Lievrouw LA, Livingstone S (ed)
Handbook of new media. Sage, Thousand Oaks/CA, pp. 162–182
Merriam Webster Dictionary (2022). Interaction. Available via https://www.
merriam-webster.com/dictionary/interaction
Meyer-Drawe K (1996) Menschen im Spiegel ihrer Maschinen. Wilhelm Fink,
München
Müller VC (2011) Interaction and resistance: the recognition of intentions in
new human-computer interaction. In: Esposito A, Esposito AM, Martone
R et al. (ed) Toward autonomous, adaptive, and context-aware multimodal interfaces. Theoretical and practical issues. Springer, Heidelberg,
pp. 1–7
Müller O, Liggieri K (2019) Mensch-Maschine-Interaktion seit der Antike: Imaginationsräume, Narrationen und Selbstverständnisdiskurse. In: Liggieri K,
Müller O (eds.) Mensch-Maschine-Interaktion. Handbuch zu Geschichte—
Kultur—Ethik. J.B. Metzler, Stuttgart, pp. 3–14
Newman W (2012) Design case study: the bravo text editor. Interactions
19(1):75–80. https://doi.org/10.1145/2065327.2065342
Neumann Jvon (1928) Zur Theorie der Gesellschaftsspiele. Math Annal
100:295–320. https://doi.org/10.1007/BF01448847
Neuman WR (1991) The future of the mass audience. Cambridge University Press,
Cambridge/MA
Norman DA (2013) The design of everyday things: revised and expanded edition.
Basic Books, New York/NY
Norman DA (1984) Stages and levels in human-machine interaction. Int J ManMach Stud 21(4):365–375. https://doi.org/10.1016/S0020-7373(84)80054-1
Nowogrodzki A (2018) The research hardware in your video-game system. Nature
553(7686):115–116. https://doi.org/10.1038/d41586-017-08968-x
Osborne MJ (2011) An introduction to game theory. Oxford University Press,
Oxford
Parsons M, Ebinger P (1968) Interaction: I. Social Interaction. In: Sills DL, Merton
RK (ed.) The international encyclopedia of the social sciences. McGraw-Hill,
New York/NY, pp. 429–441
Perkins R, Keller DS, Ludolph F (1997) Inventing the Lisa user interface. Interactions 4(1):40–53
Rafaeli S (1988) Interactivity: from new media to communication. In: Hawkins
RP, Wiemann JM, Pingree S (eds.) Advancing communication science:
merging mass and interpersonal processes. Sage, Thousand Oaks/CA, pp.
110–134
Rafaeli S, Sudweeks F (1998) Interactivity on the nets. In: Rafaeli S, Sudweeks F,
McLaughlin ML (eds.) Network and netplay: virtual Groups on the internet.
AAAI/MIT Press, Boston/MA, pp. 173–189
Rammert W (2002) Können Maschinen handeln?: Soziologische Beiträge zum
Verhältnis von Mensch und Technik. Campus, Frankfurt/Main
Schumann DW, Artis A, Rivera R (2001) The future of interactive advertising
viewed through an IMC lens. J Interact Advert 1(2):43–55. https://doi.org/10.
1080/15252019.2001.10722050
Schwarting W, Alonso-Mora J, Rus D (2018) Planning and decision-making for
autonomous vehicles. Annu Rev Control Robot Auton Syst 1:187–210.
https://doi.org/10.1146/annurev-control-060117-105157
Sharp HY (2019) Interaction design: beyond human-computer interaction. John
Wiley & Sons, New York/NY
12
Siegert B (2003) Passagen des Digitalen. Zeichenpraktiken der neuzeitlichen
Wissenschaften 1500–1900. Brinkmann und Bose Verlag, Berlin
Simeone O (2018) A very brief introduction to machine learning with applications
to communication systems. IEEE Trans Cogn Commun Netw 4(4):648–664.
https://doi.org/10.1109/TCCN.2018.2881442
Simon HA (1952) A formal theory of interaction in social groups. Am Sociol Rev
17(2):202–211
Stark W (2016, August 15). Intra-action. Available via https://newmaterialism.eu/
almanac/i/intra-action.html
Steuer J (1995) Defining virtual reality: dimensions determining telepresence. In:
Biocca F, Hillsdale MR (eds.) Communication in the age of virtual reality.
Erlbaum Associates, Hillsdale/NJ, pp. 33–56
Stromer-Galley J (2004) Interactivity-as-product and interactivity-as-process. Inf
Soc 20(5):391–394. https://doi.org/10.1080/01972240490508081
Sturman DJ, Zeltzer D (1994) A survey of glove-based input. IEEE Comput Graph
Appl 14(1):30–39. https://doi.org/10.1109/38.250916
Sutherland IE (1968). A head-mounted three dimensional display. Proceedings of
the December 9–11, 1968, Fall Joint Computer Conference, Part I. pp.
757–764
Sutherland IE (1964) Sketchpad: a man-machine graphical communication system.
Simulation 2(5):3–20. https://doi.org/10.1177/003754976400200514
Svanæs D (2000) Understanding interactivity: steps to a phenomenology of
human-computer interaction. Norges teknisk-naturvitenskapelige universitet,
Trondheim
Swedberg R (2001) Sociology and game theory: contemporary and historical perhttps://doi.org/10.1023/A:
spectives.
Theor
Soc
30(3):301–335.
1017532512350
Thomas PC, David WM (1992) Augmented reality: an application of heads-up
display technology to manual manufacturing processes system sciences.
Proceedings of the Twenty-Fifth Hawaii International Conference
on System Sciences II:659–669. https://doi.org/10.1109/HICSS.1992.
183317
Tomlinson IPM (1997) Game-theory models of interactions between tumour cells.
Eur J Cancer 33(9):1495–1500. https://doi.org/10.1016/S0959-8049(97)00170-6
Turing AM (1950) Computing machinery and intelligence. Mind 59(236):433–460.
https://doi.org/10.1093/mind/LIX.236.433
Vannini P (2016) Body/embodiment: symbolic interaction and the sociology of the
body. Routledge, London
Velliste M, Perel S, Spalding MC et al. (2008) Cortical control of a prosthetic arm for
self-feeding. Nature 453(7198):1098–1101. https://doi.org/10.1038/nature06996
Verbeek P-P (2015) Beyond interaction: a short introduction to mediation theory.
Interactions 22(3):26–31. https://doi.org/10.1145/2751314
Verbeek P-P (2005) What things do: philosophical reflections on technology,
agency, and design. Pennsylvania State University Press, University Park/PA
Webb JN (2007) Game theory. Decision, interaction and evolution. Springer,
London
Weiser M (1991) The computer for the 21st century. Sci Am 265(3):94–105.
https://doi.org/10.1038/scientificamerican0991-94
Williams F, Rice RE, Rogers EM (1988) Research methods and the new media. The
Free Press, New York/NY
Acknowledgements
This work was supported by the German Research Foundation (DFG) [Grant Number
418201802]. We thank Elena Meidert and Armin Gruber for their help with the extensive
research for this article.
Author contributions
All authors contributed to the conception and design, literature research, analysis, and
(analytical) interpretation, as well as to drafting and revising the manuscript. All authors
gave final approval of the paper as well as their agreement to be accountable for all
aspects of the work in ensuring that questions related to the accuracy or integrity of any
part of the work are appropriately investigated and resolved.
Funding
Open Access funding enabled and organized by Projekt DEAL.
Competing interests
The authors declare no competing interests.
Ethical approval
This article does not contain any studies with human participants performed by any of
the authors.
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2023)10:551 | https://doi.org/10.1057/s41599-023-02060-8
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-023-02060-8
Informed consent
This article does not contain any studies with human participants performed by any of
the authors.
Additional information
Correspondence and requests for materials should be addressed to Sebastian Schleidgen.
Reprints and permission information is available at http://www.nature.com/reprints
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.
ARTICLE
Open Access This article is licensed under a Creative Commons
Attribution 4.0 International License, which permits use, sharing,
adaptation, distribution and reproduction in any medium or format, as long as you give
appropriate credit to the original author(s) and the source, provide a link to the Creative
Commons license, and indicate if changes were made. The images or other third party
material in this article are included in the article’s Creative Commons license, unless
indicated otherwise in a credit line to the material. If material is not included in the
article’s Creative Commons license and your intended use is not permitted by statutory
regulation or exceeds the permitted use, you will need to obtain permission directly from
the copyright holder. To view a copy of this license, visit http://creativecommons.org/
licenses/by/4.0/.
© The Author(s) 2023
HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2023)10:551 | https://doi.org/10.1057/s41599-023-02060-8
13