On adaptive self-organization in artificial robot organisms
Serge Kernbach∗, Heiko Hamann† , Jürgen Stradner† , Ronald Thenius† , Thomas Schmickl† , Karl Crailsheim† ,
A.C. van Rossum‡ , Michele Sebag§ , Nicolas Bredeche§ , Yao Yao¶ , Guy Baele¶ , Yves Van de Peer¶ ,
Jon Timmisk , Maizura Mohktark , Andy Tyrrellk , A.E. Eiben∗∗ , S.P. McKibbin†† , Wenguo Liu‡‡ , Alan F.T. Winfield‡‡
∗ Institute of Parallel and Distributed Systems, University of Stuttgart, Germany, Serge.Kernbach@ipvs.uni-stuttgart.de
† Artificial Life Lab, Karl-Franzens-University Graz, Universitatsplatz 2, A-8010 Graz, Austria,
{heiko.hamann,juergen.stradner,ronald.thenius,thomas.schmickl,karl.crailsheim}@uni-graz.at
‡ Almende B.V., 3016 DJ Rotterdam, Netherlands, anne@almende.com
§ TAO, LRI, Univ. Paris-Sud, CNRS, INRIA Saclay, France, {Michele.Sebag, Nicolas.Bredeche}@lri.fr
¶ VIB Department Plant Systems Biology, Ghent University, Belgium,
{Yao.Yao, Guy.Baele, Yves.VandePeer}@psb.vib-ugent.be
k University of York, York, United Kingdom, {mm520, jt517, amt}@ohm.york.ac.uk
∗∗ Free University Amsterdam, gusz@cs.vu.nl
†† Materials and Engineering Research Institute (MERI), Sheffield Hallam University, s.mckibbin@shu.ac.uk
‡‡ Bristol Robotics Laboratory (BRL), UWE Bristol, {wenguo.liu, alan.winfield}@uwe.ac.uk
Abstract—Self-organization in natural systems demonstrates
very reliable and scalable collective behavior without using any
central elements. When providing collective robotic systems
with self-organizing principles, we are facing new problems of
making self-organization purposeful, self-adapting to changing
environments and faster, in order to meet requirements from
a technical perspective. This paper describes on-going work
of creating such an artificial self-organization within artificial
robot organisms, performed in the framework of several
European projects.
(a)
(b)
Keywords-adaptive system, self-adaptation, adaptive selforganization, collective robotics, artificial organisms
I. I NTRODUCTION
Adaptivity is a much-desired property of real-world systems, where the system itself can adjust its own functionality
or behavior to uncertainties and variations of the environment. The issue of adaptivity has been considered in the
theory of adaptive control (e.g. [1]), knowledge-based and
deliberative systems (e.g. [2]), situated [3] and embodied [4]
systems. Many different approaches are devoted to achieving
adaptivity: different learning techniques [5], behavior-based
[6], bio-inspired [7], evolutionary approaches [8] and many
other.
Considering multi-robot systems, such as collective,
swarm [9], reconfigurable and cellular robotics [10], we
should note that these systems utilize another principle of
control and organization: instead of one or several central controllers, collective systems undergo different selforganizing (SO) processes [11]. In particular, this work
addresses a new type of collective systems [12]: many single
swarm robots can aggregate into one multi-robot organism,
see Fig. 1. This system is an object of research in the
SYMBRION and REPLICATOR projects. Terminologically
we say the disaggregated robots are in swarm-mode, whereas
(c)
(d)
Figure 1. Examples of swarm- and organism-modes. (a)-(b) Demo
of concept, 2007: Real large-scale swarm of Jasmine robots and
topological model of an organism; (c)-(d) Prototypes, 2009: a
few robots in a swarm-mode and in a simple organism (images
c SYMBRION, REPLICATOR projects).
aggregated robots are in organism-mode. Here we observe
a new challenge. Robots in swarm-mode utilize SO phenomena as the main means of regulating functionality at
the collective level. Aggregation, decision making, energetic
homeostasis and other collective activities are created by
artificial SO through bio-inspired [6] or derived [13] local
rules. As shown by these and other works, the SO in the
swarm-mode provides efficient, scalable and very reliable
behavior. However, when measuring collective reactivity in
terms of how fast a collective system is able to process
information [6], we should remark, that SO remains a
relatively slow organizational process.
Considering organism-mode, we face two contradictory
requirements: we need decentralization, scalability and reliability provided by artificial SO, however we need much
faster and much more adaptive regulative functionality.
Making SO more self-adaptive and faster, while keeping
scalability and decentralism represents one of the main challenges here. This paper focuses on different SO processes
in the context of swarm- and organism-modes as well as
in the transition between them. SO is viewed as a means
toward an end − the ability of the self-organized robots to
come up with a competent and robust response to an open
environment under limited resources. In Sec.II this work
gives a short overview about different SO processes running
onboard and online in artificial organisms and introduces
these approaches in the following sections. In Sec. XII
we discuss several open problems and finally conclude this
work.
II. C OMMON
PICTURE OF DIFFERENT
SO
PROCESSES IN
ARTIFICIAL ORGANISMS
To represent different SO processes in artificial organisms,
let us consider a topological model of a planar (aggregated
organism on 2D plane) dog-like structure, shown in Fig. 2.
First, an organism represents an aggregation of indepenmacroscopic legs
Comm
and com on energy
municat
ion bus
sensors
controllers
energy
genome
sensors
controllers
energy
genome
sensors
controllers
energy
genome
sensors
controllers
energy
genome
sensors
controllers
energy
genome
sensors
controllers
energy
genome
sensors
controllers
energy
genome
sensors
controllers
energy
genome
sensors
controllers
energy
genome
sensors
controllers
energy
genome
sensors
controllers
energy
genome
sensors
controllers
energy
genome
Environment
sensors
controllers
energy
genome
sensors
controllers
energy
genome
sensors
controllers
energy
genome
macroscopic legs
Figure 2.
Topological model of a planar dog-like organism.
dent modules, each of which possesses sensors, actuators
(shown as active joints), internal controllers and an artificial
“genome”, which reflects configurations of a module. From
this viewpoint organism-mode is very similar to swarmmode with the difference that all robots are spatially fixed on
a grid, but with additional degrees of freedom through common buses and common actuation. Thus, it is expected that
SO phenomena, although a characteristic of swarm-mode,
can also be widely used in organism-mode. Considering
Fig. 2, we can distinguish five self-organization processes
running in the organism:
• Swarm-mode. As mentioned, swarm-mode is a classical application field of different SO phenomena. The
Sec. III gives an overview of using learning and evolving in swarms. Since the system will be in swarmmode only 15%-20% of the whole time, the relevance
of swarm-mode for an organism is relatively low.
• Developmental level. Developmental processes describe how the structure and functionality are “growing” from non-aggregated and not-differentiated modules to organisms with complex monofunctional actuation. More generally, during developmental phase,
which can be performed on-line and on-board but also
off-line and off-board, different adaptive mechanisms
can be evolved through evolutionary/learning processes,
as described in Sec. 3. Three examples of such approaches are described in Secs. V, VI and VII.
• Homeostatic level. All modules have different levels
of energy, different genome, different goals. Being
aggregated in the organism, from all modules should
emerge an internal homeostatic system, which maintains endogenous steady state and protects the whole
organism. Different SO processes are utilized on the
homeostatic level, as described in Sec. VIII.
• Cognitive level. All modules possess independent sensors and through aggregation there appear several effects like spatial distribution of sensors, overlapping
of functionality and increasing of redundancy. Sec. X
describes several cognitive processes, which use SO
effects.
• Level of macroscopic locomotion. Macroscopic locomotion represents one of the largest tasks for SO
processes in the organisms. For example, an organism,
shown in Fig. 2, has 4 legs, which have around 12 motors in active joints. All of them should be synchronized
in order to obtain an uniform macroscopic locomotion.
For a macroscopic SO-based control, such approaches
as synchronization of coupled oscillators or adaptive
hormone system, as described in Sec.XI can be used.
Despite the fact that all of these processes use SO phenomena on different levels, they all contribute to making
the whole system act as one entity, being thereby fully
decentralized with scalable functionality and behavior. In
Sec.XII we give a common view on different SO processes
and mention open problems in the organism-mode.
III. S WARM - MODE :
SELF - ORGANIZATION THROUGH
LEARNING AND EVOLUTION
While biological entities are implicitly required to survive
with a competitive advantage, self-organization in swarm of
robots is also assessed from the collective behaviors of all
individuals regarding the designer’s objective. Hence, both
environmental conditions and internal motivations define
an implicit objective function. In other words, swarms of
robots should be able to converge towards an efficient
behavioral strategy at both the individual and the population
levels that maximize the intended objective and comply with
the environment specific properties. This implies that each
individual within the swarm must be endowed with internal
variables, rewards and rules enforcing the desired behavior
in an environment- and self-driven fashion. On the one
hand, biological entities may rely on their instincts (curiosity,
fear, ...) to provide a set of basic behaviors to explore
the environment that are at least partially correlated with
the optimal survival strategy. On the other hand, designing
efficient SO artificial entities raises the question on how
to provide capabilities to address survival, or in a broader
sense: task optimization, in complex environments. This
raises two main challenges:
•
•
Defining self-driven rewards (curiosity, cognitive dissonance, . . . ) yielding a sufficient and safe exploration of
the policy space;
Defining a decentralized optimization process favoring
desirable policies on the individual level and communication rules enforcing convergence towards an efficient
behavioral strategy at the swarm level (i.e. emergence
of a collective behavior).
An integrated approach to the above challenges explored
within SYMBRION relies on the combination of Evolutionary and Machine Learning methods. Typically, criteria
derived from Information Theory can be used to measure
the new information gathered by an entity. A built-in instinct
(maximize the gathered information) provides the entity with
a “curiosity-like” bias, which should at least be partially
correlated with the optimal strategy (e.g. curiosity is a first
step towards finding energy sources in order to maximize
both autonomy and exploration). An evolutionary framework
driven by curiosity should provide a variety of behaviors,
bootstrapping the optimization process towards an efficient
swarm self-organization. Moreover, this kind of criteria can
be reformulated so as to cope with designer preferences,
standing for the Darwinian milieu. Setting the human back
into the loop makes it possible to shape some specific desired
behaviors related to the desired task to be solved (sparse
interactive optimization, supported by preference learning).
Such locally defined criteria make it possible to address
the behavior bootstrapping problem in complex environment.
Then, the swarm self-organization process can be reformulated as a decentralized optimization problem where the
swarm behavior may either be homogeneous or heterogeneous, and where the self-organization process is based on
local diffusion so as to reach an equilibrium in the DS. This
implies both the definition of specific optimization operator,
which can be defined as stochastic, such as evolutionary
operators, or deterministic, such as babbling algorithms, for
spreading behaviors and enforcing convergence. Moreover,
the particular setup imply that local explicit bootstrap (e.g.
curiosity fitness) or human-driven (e.g. shaped curiosity
fitness) criteria should be taken into account as well as
a global, not straight-forward, and implicit objective (e.g.
ensuring energy autonomy) should be addressed. While
explicit and implicit criteria may be correlated to some
extent, they might diverge in the final state (e.g. maximizing
curiosity may not be the best compromise to guarantee
autonomy and survival in an extreme environment).
IV. D EVELOPMENTAL L EVEL : ACHIEVING
ADAPTIVITY
THROUGH EVOLUTION
The role of evolutionary adaptive mechanisms is essential
in all forms and at all levels of SYMBRION & REPLICATOR projects, including individual robots in swarm mode
as well as in aggregated organisms. The main vision behind these projects includes that the controllers, or behavioral policies, undergo pervasive adaptation. Evolution
and learning are two pivotal components of the “adaptation
engine” facilitating this. To make such a system work one
basically needs good reward systems (to support selection)
and good evolutionary and learning operators (to support
variation). Inherently to the SYMBRION philosophy, these
mechanisms should work without any central control, and
even with some degree of self-adaptivity to regulate their
own parameters. In the system we obtain this way, one can
distinguish three levels of evolution, see 3. First, genetic
Organism Evolution
organism is unit of selection
Social Evolution
collective cross-fertilization
Genetic Evolution
robots are unit of selection
Figure 3. Achieving adaptivity through evolution: three levels of evolution.
evolution, concerning the artificial genomes in the individual
robots. At this level, the individual robots form the units of
selection and variation operators (mutation and crossover)
work on the genomes. Second, cultural or social evolution,
concerning the controllers directly, i.e., not the underlying
genomes encoding them. At this level, the individual robots
form the units of selection and variation operators work
on the controllers. It makes sense to see this mechanism
as a cultural or social process, where individual learning
plays the role of mutation (change of controller within
one robot) and individually acquired features are spread
over the group of robots through communication (crossfertilization or crossover of good ideas). At the third level
we find organism evolution, where artificial organisms form
the unit of selection and variation operators work on the
joint genome or controller of the organism.
While in most evolutionary applications, including evolutionary robotics, selection and variation is managed by
a central authority (the main EA loop or sometimes the
user), the SYMBRION system is inherently decentralized. In
particular, we will not have a master or oracle determining
which robots or organisms can mutate or transmit their
genes/controllers to others. Instead, the units of selection
will be autonomous, running all evolutionary and learning
operators onboard, in an online fashion. Hence we will
obtain a highly sophisticated version of embodied evolution
[14]. The resulting system will give rise to the emergence
of spatial, temporal and functional structures over time, thus
being inherently self-organizing.
One of the greatest challenges to this end is represented
by the reward functions. In general, we can use two types
of rewards here. On the one hand, self-driven rewards
that can be biased towards explorative behavior (curiosity),
social behavior (e.g., sex-drive or some “basic instinct” for
information sharing). On the other hand, we can use rewards
that are based on measurable task-performance, for example,
related to self-maintenance (energy level, system integrity),
the size of the organism the robot is part of, or even some
user defined task, like the number of red rocks collected or
the time needed to identify the exit of a given room.
(a) Embryo
(b) Specialising cells
(c) Structured neural network
V. D EVELOPMENTAL
LEVEL :
S HAPING
ROBOT
ORGANISMS AND CONTROLLERS BY VIRTUAL
EMBRYOGENESIS
Another approach to shape a robotic organism is to
simulate the self-organising processes, observable during the
process of biological embryogenesis [15]. In this approach
the robotic modules represent the single cells of an embryo.
The behaviour of a single module (e.g., allow docking,
switch to a predefined controller) is controlled by virtual
morphogenes, that diffuse throughout the whole robotic
organism. External influences (e.g. sensor inputs) or internal
influences (e.g., defined morphogene concentrations) lead to
the emission of other morphogenes. The conditions, under
which a morphogene emission or a robotic behaviour is
triggered, is coded in an artificial genome. By using artificial
evolution on this genome, it is possible to optimise the body
shape.
From the feedback-system, consisting of morphogene gradients, body-shape and robotic behaviour, a self-organised,
evolvable, bio-inspired process arrieses, that allows the development of different robotic body-shapes in a evolutionary
manner.
The process of virtual embryogenesis can also be used
for shaping the controller of single robotic modules [16]. In
this approach the growth of a virtual embryo is simulated to
shape the network topology of a neural network. The usage
of this approach seems to be advantageous for the shaping
of heterogeneous neural networks (Fig. 4) consisting of cells
Figure 4. Development of an artificial neural network in a virtual
embryo [16]. In Fig. 4(a) the embryo is depicted. white dots
indicate single cells. The shape of the embryo is influencing the
specialisation of cells (Fig. 4(b)) within the embryo, that later
develop into nodes of the neural network (Fig. 4(c))
with different functionalities, for example controlling cells
or teaching cells, as described in [17].
The self-organised process of virtual embryogenesis [16]
enables the adaptation of a (multi-) robotic systems as in an
evolutionary manner on both, the level of the single robotic
unit, and the level of the robotic organism.
SO
VI. D EVELOPMENTAL LEVEL :
THROUGH REAL ROBOT AUTONOMOUS
MORPHOGENESIS
A key requirement in the SYMBRION project is the transition from swarm to organism: autonomous morphogenesis.
This is the process by which firstly, one or more robots (in
swarm-mode) ‘decide’ that they need to self-assemble into
an organism (e.g. in response to a barrier which a single
robot cannot climb over) then, secondly, the robots selfassemble into the correct planar arrangement (as seen in
Fig. 2) and, thirdly, the robots in the 2D planar organism
assume the correct functionality (i.e. differentiate) according
to their position in the organism. These three steps we can
label as initiation, assembly and differentiation, respectively;
the key stages in autonomous morphogenesis. After these
three stages are complete the organism can lift itself from 2D
planar configuration to 3D configuration and, with respect
to locomotion, will function as a macroscopic whole, as
outlined in section XI. Consider these three stages.
1. Initiation. Prior to initiation the robots are operating
in swarm-mode, as outlined in section III. Initiation requires
that only one robot in the swarm makes the decision to start
the transition from swarm to organism − that robot then
forms the ‘seed’ robot for the new organism. If we take
the example of a barrier that needs to be crossed then a
single robot might determine this condition through, firstly,
its collision with the fixed barrier; secondly by running along
the barrier and colliding with another robot and, thirdly, by
then running along the barrier in the opposite direction and
colliding with a third robot. Following this sequence of cues
the robot needs to retreat from the barrier (in order to give
the organism room to self-assemble); stop; select the genetic
instructions for ‘barrier crossing organism’; then transition
to assembling, as outlined below. Note that this SO process
of initiation could allow several robots to make the same
decision to initiate at the same time − however, this problem
might be simply resolved by arranging that the signal to dock
(see below) also suppresses the initiation behaviour in those
robots close enough to see the signal.
2. Assembly. In this stage one or more of the robots
in a partially assembled organism will express its DNA
by signalling for other robots to dock with it, on certain
faces, in order to build the planar organism. Several relevant
approaches have been described in the literature; in one
approach aimed at self-assembling static structures (i.e. 3D
‘houses’ built from intelligent autonomous robot bricks)
each robot has a stigmergic rule set describing the whole
structure and decides what to be (i.e. wall, corner) according
to where it finds itself after randomly attaching [18]. While
this methodology is provably correct it has the drawback
of requiring a high level of random motion (i.e. energy) in
the swarm so that robots will, eventually, find themselves
in the right position for the growing structure. Another approach aimed at self-assembling planar mobile robot groups
proposes a script (SWARMORPH) that describes how the
structure self-assembles [19] the script is transferred into
each newly-docked robot which then interprets the script
to determine whether (and on which face) it should signal
for another robot to dock, or not if it’s at the end of the
chain. Since this approach actively signals for another robot
it (probably) requires less energy in the swarm to complete
the assembly process.
3. Differentiation. This stage is entered when the planar
structure is physically complete but each robot is in effect
an undifferentiated ‘cell’; it is in the right place in the
body of the organism but has no specialised function. In
the differentiating state each robot assumes a specialised
function: for instance: FootBot if it is at the end of a leg
structure and will be placed on the ground during walking;
LegBot if it is one of several robots joining a foot to a
knee or other joint or JointBot if it is required to bend at
the junction between leg or arm structures. It is possible that
differentiation will take place during assembly, in which case
this will not be a separate state; it is however important to
recognise that assembly and differentiation are key functions
required by each robot’s controller in organism-mode. The
simplest approach to differentiation is one in which the
function of each robot in the organism is also encoded in
the genetic instructions for building the organism.
VII. D EVELOPMENTAL LEVEL :
SO THROUGH GENE EXPRESSION
ADAPTIVE
This section discusses a new approach to deal with
artificial organisms and robotic co-evolution using an agentbased framework/model, a powerful simulation modeling
technique (see e.g. [20]).
Problem definition. As is commonly known, the genome
and the concept of gene expression play critical roles in
cell specification and morphogenesis. The genome encodes
a bottom-up developmental approach which drives the single
stem cell developing to complicated organisms with highly
specific functionality (see [21] for an example). In addition,
this extremely complicated process is performed without any
centralised control while interaction with the environment
allows for adaptation. These features of real-life genomes allow biological organisms to possess a robust self-organizing
ability as well as flexible adaptation by providing various
expression patterns. These features are also part of what we
plan to achieve in the development of an artificial organism.
We propose to build an agent-based framework which is
used to mimic a real-life genome and gene expression operations. This way, our framework can use a similar approach
as in biology to solve the adaptive self-organization problem
in our artificial organisms. This section discusses the general
idea of how our mechanism works in self-organization.
The robotic system (either in swarm or aggregated mode)
can hence gain good functional or cooperative patterns
through this self-organization process and the influences of
the external environment.
Methodology. The general idea is that of a society where
each member receives a particular identification according
to its performance in cooperation. This identification will
be used to guide the future cooperative operations. When
some basic cooperation/ sub-organism occurs in a group,
the partners in the cooperation/ sub-organism will compete
with each other for leadership and the winning partner will
activate the high level information (next stage development
knowledge; for example the templates of shape, the group’s
internal communication rules, . . . ) in its gene and build an
agent for processing high level computation. Lower level
cooperation can combine to higher level cooperation in a
similar way but only the winning partner will be involved
in this higher level combination because it can represent its
whole group as one entity in a higher level cooperation.
Implementation. An agent-based framework will be
loaded into each robot, along with an artificial genome.
Important knowledge concerning cooperation constraints,
possible templates, etcetera will be decomposed into a
robot’s genetic code. The encoding and decomposing rules
will be modeled according to Piaget stages of development
theories (see e.g. [22]). The goal is not to simply distribute
the knowledge into genes as the knowledge will be classified
corresponding to different stages in the whole development
process. Different stage developmental information will be
encoded in different levels of abstraction in the genome. In
the initial stage, for example, the system should focus on
one-to-one docking and communication operations, so there
will have to be a layer present in the genome which is used
to store this kind of knowledge. In a later stage, the system
needs to take care of more complex aspects like shape,
functionalities and physical constraints, the knowledge of
which will be stored in a higher layer in the genome.
In each robot, there is a controller which will be able to
develop various agents to read the different genes from the
robot’s genome (see Fig. 5). The controller’s functionalities
will be determined by the interaction of these self-developed
agents and the functional patterns of the controller will
emerge from the self-organizing process in swarm agents.
A controller can change its knowledge and functions by
developing new agents to read new genes in its genome.
Each robot shares similar genes in this design but according
to its specific environment, each robot will be able to activate
different genetic options and develop different functions during its lifetime. Initially, each robot has one basic controller
which is built using the initial stage information stored in
its genes. Further, each robot also has a certain cooperation
credit value from the moment the controller is built. This
credit value shows the usefulness of this robot to the system.
When a robot wants to cooperate with other robots, it
needs to pay some credit value to its potential partner.
When another partner accepts this value, the cooperation
will be able to proceed. If a potential partner refuses to
receive this value (for example the partner doesn’t need to
cooperate with others or the value is not enough to meet
the partner’s expectation) the cooperation will be canceled.
This way, the system will have an emerged identification
pattern which come from the self-organizing process of
robot’s cooperation.
Example. For example, the robot will be able to adjust its
cooperation price (credit value) according to its fitness status
when it is involved in cooperation. If the leading controller in
a cooperation can not afford the new prices of other robots,
the robots may decide to find a new leading controller in
the group. This ensures that only good cooperating groups
will be selected. This way, a system can develop through
several stages to achieve better shape and functionality until
Environment inputs
Evolutionary
operators
Agent for stage 1
Agent for stage 2
Initial
controller
Genome
Agent for stage 3
Master
controller
of level 3
Controller
of level 1
Master
controller
of level 2
Master
controller
of level 2
Master
controller
of level 2
Controller
of level 1
Controller
of level 1
Controller
of level 1
Evolutionary
operators
Genome
Controller
of level 1
Environment inputs
Initial
controller
Agent for stage 1
Structure diagram of the agent-based framework for
robotic cooperation.
Figure 5.
such complex cooperation is no longer required in the
given environment (for instance, the task is done). In the
development process, each robot controller can specify its
functionality by creating new agents and remove or replace
old agents. The way the system is doing this will depend on
the information in the genome and the environmental situation. Except for a few predefined constraints (for example,
for safety and avoiding simple mistakes), all functionalities
and strategies will emerge from the interaction of robots in
development processes. The formation process depends on
the environment. The integration conditions will be classified
into different levels of abstraction and stored in genes
independently.
SO
VIII. H OMEOSTATIC LEVEL :
ROBOTIC A RTIFICIAL I MMUNE S YSTEMS
WITHIN
Each SYMBRION robot will contain an individual artificial immune system (AIS) that is capable of identifying
and predicting when the robot will fall out of normal
operational conditions. This may be a fault that has occurred
within the robot caused by some mechanical problem, or an
environmental impact on the system. However, the AIS is not
limited to a single robot, but an artificial immune network
will be created between robots to allow for the sharing of
immunological information between units, and the communication between units to identify potential problematic units
and prevent them from joining the organism.
In our work, we make use of various immune system
metaphors for the creation of our artificial system, ranging
from the provision of an innate or pre-programmed type of
response that is typically static during the lifetime of the unit,
and an adaptive response that improves during the operation
of the unit. Each AIS has the potential to be unique on
every robot, as what each robot senses and performs during
it’s lifetime may well be different : and this will drive the
evolution of the adaptive AIS.
Of most interest to work in this paper is the adaptive
aspect of the AIS. We envisage a system that can maintain it’s own unique immune system through a combined
process of clonal selection (a way in which new artificial
detectors can be generated that takes into account heuristic
information) which allows us to develop a self-organising
memory structure for the AIS, based on stimulation and
death of error detectors in the AIS population. For interrobot immunity, we employ ideas from immune network
theory [23]. Immune network theory is now quite dated
within the immunological community, but has many useful
lessons for us to create self-organising memory structures.
A network occurs due to the ability of paratopes (molecular
portions of an antibody) located on B cells, to match against
idiotopes (other molecular portions of an antibody) on other
B cells. The binding between idiotopes and paratopes has the
e?ect of stimulating the B cells. This is because the paratopes
on B cells react to the idiotopes on similar B cells, as it
would an antigen. However, to counter the reaction there is
a certain amount of suppression between B cells which acts
as a regulatory mechanism. This interaction of B cells due
to the network, was said to contribute to a stable memory
structure, and account for the retainment of memory cells,
even in the absence of antigen.
This idea can be exploited in an artificial context through
the creation of an artificial immune network, typically of
robots that can be used to determine if other units within
a network may be heading towards some form of failure.
A simple binding system (or interaction) between units is
required to compare how similar states of the unit are: one
simple metric is Euclidean distance. This bind goes towards
stimulating a B-cell (or in this case a robotic unit), with a
strong bind indicating a strong similarity between the units.
This stimulation level of the unit is used to help regulate
the survival (or otherwise) of the robot, and help decide
if the unit should be allowed to join an organism: as if the
unit is potentially faulty, then joining the organism may well
be problematic. What emerges is a network of self-similar
robots supporting each other over time.
IX. C OGNITIVE LEVEL : S ENSORY-M OTOR F USION
M ODULAR ROBOTICS S YSTEM
IN A
A modular robotic system is a domain that crosses several/all robotics domains in that the system operates on
the basis of single autonomous robots, swarms of loosely
coupled multiple robots and tightly coupled multiple robot
collections in the form of complex robot-organisms. As a
result of this generality of the system, the robots employed
must be able to operate under all these circumstances
showing great adaptability to both environmental and system
changes. In REPLICATOR, the single robot cell has a degree
of autonomy that is not found in other modular robotic
systems[24][25][10]. Each robot is capable of locomotion
on a plane and is thus able to move around freely on its own
under many conditions. One of the robot designs is capable
of holonomic motion on a plane and the other is capable of
overcoming small obstacles and, if it finds itself flipped onto
its back, it can continue to move around upside-down. This
level of autonomy places a higher degree of importance on
the single robot cell within the system as a whole.
Sensory-Motor Coordination. From the perspective of
sensory-motor coordination, it is a difficult problem to
implement an architecture that is capable of operating under
these different modes of operation. Sensory-motor coordination is an essential quality for any embodied agent to
possess. It allows the efficient transfer of information from
motor actions to sensory sensations and to higher levels that
are more decoupled from the sensory-motor flow that can
integrate dynamics over time. Also, efficient implementation
of sensory-motor coordination can allow the robot to seek
out and utilise sensory information that is most relevant to
it under the current environmental conditions. Moreover, by
developing a proprioceptive type sensory system, the robot
agents can begin to ground sensory inputs to sensations that
link them with their environment. The relationship between
the robot and the environment in which it senses and acts
can be learned and exploited.
Sensory-Motor Fusion. Our approach to the sensorymotor fusion for REPLICATOR modular robotic system
consists in decentralising the system at the level of the
individual robot cell. The single robot cell is the only unit
that will not physically change during its operation, only its
interface with other units is a dynamic configuration. The
single robot is a natural level of decomposition as each
cell is capable of acting on and sensing the environment
immediately in its vicinity. Therefore any actions, sensations,
processes running on an individual robot will be inherently
relevant to that particular cell. Interfacing with other robot
cells in the system can arise as part of a self-organizing
process that can withstand perturbing fluctuations and yet
still undergo useful transformations.
Since it is impossible to know in which situation a robot
will find itself at any given moment in time, the system
that implements the sensory-motor coordination should be
adaptable to change and should work for a single robot, a
robot swarm and a robot organism. What is clear from this is
that the robot will operate either on its own or in conjunction
with others. Normally, sensory-motor coordination can be
achieved on a single robot by tightly coupling the sensory
and motor systems in order to exploit specific sensor flows
however, in a multi-robot multi-configuration system, this is
not enough. The single robot should be able to sense if it
is operating alone or in cooperation with others, i.e. if it is
physically connected to another robot cell or within communication range of another cell. By identifying these situations
a robot cell can adapt its sensory-motor coordination to suit
a particular context.
We propose a Recurrent Neural Network (RNN) that
takes inspiration from similar models used for language processing, recognising sensory-motor flows and for imitation
[26][27][28][29]. What is common to these models is that
they are able to articulate sensory-motor flows over extended
periods of time and that they do so in a dynamic and selforganising fashion. The benefit of using an RNN is that it
is capable of learning temporal sequences by automatically
fusing information from many sensors of possibly different
modalities and across varying timescales. For mobile robots
that operate in changing environments it is important that
they can extract and exploit the dynamics of the environment
and RNNs have been shown to be a suitable control representation for evolving this capability [30][31]. The drawback
of using neural networks can be that they do not lend
themselves well to the extraction of the rules on which they
operate. This drawback can be outweighed by their ability to
both generate new, but similar behaviours, on unseen input
data or to recognise and generalise unseen sensor patterns.
The idea of a Parametric Bias (PB) in the RNN model
is to allow an outside interaction with the model that can
change the way that it behaves according the learned/evolved
behaviours or sensory-motor patterns. It is through the use of
these PB nodes that the RNN is capbale of both recognising
and generating sensory-motor flows. This enables both a
bottom-up and top-down approach to building intelligent
systems. Another advantage of this approach is that multiple
models residing on separate robot cells can be dynamically
hooked together to produce a more complex system from
similar individual units. The dynamics of the system should
self-organise according to a distal fitness function or learning
signal that promotes stability within the system.
Allowing individual robots to encapsulate and process the
sensory-motor flow within a single robot cell enables the
distributed processing of sensory information that can be
shared through specific channels of docked robots cells or
through communication channels of more loosely cooperating swarming robots.
X. C OGNITIVE LEVEL : S ELF - ORGANIZATION
IN
COGNITIVE SENSOR FUSION
This section describes a self-organizing approach for
sensor fusion on modular robots. It follows the course
of problem definition, illustrative example, current work,
methodology, implementation and discussion.
Problem definition. The type of robots that are addressed
in this article are so-called modular robots. They can undergo drastic metamorphosis from say, a snake to a spider
form. This poses stringent requirements on the way sensor
data is processed and the nature of the sensor fusion architecture. The scientific challenge is coined meso-morphosis:
internal change alongside (body) metamorphosis. Or in other
words: how to survive from caterpillar to butterfly.
Leading example. Consider a robot snake with cameras
turned on at the head and the tail. First visual processing
might occur at the modules near the head and the tail.
Higher-level processing at subsequent modules. The sensor
fusion architecture is aligned alongside the body of the snake
robot. Then, the snake morphs into a spider robot. The body
may now contain the cameras and the legs the visual data
processing units.
Related work. There are so-called cognitive frameworks
that allow for awareness of not only the environment, but
also the body itself. Vernon et al. [32] have an excellent
overview on cognitive frameworks. They distinguish the
cognitivist approach from the emergent systems approach.
The former being defined as operations on symbolic representations. The latter as an umbrella term for dynamical,
connectionist and enactive systems. Their twelve distinctions
will not be reiterated over here, but an example will be given
in the context of sensor fusion. The cognitivist approach
would ask for primitives like “is there a hole in the ground
between me and robot X”. The designer defines a symbol
“hole” and the visual architecture to classify entities in the
field as a hole. An emergent approach would define a goal
like “go to robot” or “aggregate”. It needs an environment
with holes and it uses a reinforcement strategy to let the
robot acquire sufficient knowledge about holes to reach the
goal.
Methodology. In this section the emergent approach is
endorsed. Vernon et al. explain: “adaptation ... in emergent
systems ... entails a structural alteration or reorganization to
effect a new set of dynamics”. In essence, sensor fusion combines data from multiple sensors across several modalities
into representations that can be used in subsequent stages.
For that reason, data processing units, or filters, are used
in topological configurations. Filter output is fed into the
input of next “layer” filters in a hetero-hierarchical set-up.
For emergent cognitive sensor fusion three components are
needed in this methodology for self-organized sensor fusion:
1) A network of primitive filters that perform sparse
coding, auto-associative coupling, saliency detection,
etcetera;
2) A reconfigurable topology of those filters and metainformation that guides reconfiguration;
3) A search process that finds those topologies that correspond to cognitive notions like attention, anticipation
as in existing cognitive models.
Implementation. All three points of the methodology are
addressed in this section:
1.) The type of filters that have been implemented are
2D feature maps as described by Itti and Koch [33]. An
orientation filter decomposes an image in patches that only
respond to lines with a specific orientation. This can be
done for color, intensity and even other modalities. The
feature filters contribute to an overall saliency map. Now,
the horizontal line filter might need to be downscaled in
importance with respect to color and intensity. The desired
weights of the feature maps (also a weight of zero) is almost
impossible to tell in advance and might differ per task
and robot morphology. Hence, a self-organized system that
comes up with a proper filter topology is what is required
over here.
2.) For a reconfigurable topology a developmental engine
as described in section V is used. It is an implementation of
the gene regulatory network by Bongard [34]. This type of
engine takes a genome as input and has a graph as output.
The graph is in Bongard’s work interpreted as the body of an
artificial organism or as a neural network. For sensor fusion
the graph will be interpreted as a topology between filters
and connections between them.
3.) Comparison of the developed sensor fusion topology
with cognitive models is necessary to describe its level
of cognition. The basal ganglia analogue in Shanahan’s
robot [35] intercepts the recommendations of the saliency
based system and modifies them using an internal simulation
mechanism. In a world of power outlets and other robots,
detecting the first might become biased (attention) when the
robot is running low on batteries.
Discussion. The current implementation is at the second
step, however, it is already worthwhile to look forward to
step 4 and beyond. There are two additional steps necessary
to obtain full-fledged meso-morphosis:
4.) Post-development use of the gene regulatory network.
The network level is then able to make changes from spider
to snake form without the need for a new genome: online
self-organization;
5.) Online self-organization that preserves cognitive capabilities. To put it simply: the snake should be able to
remember things from its life as spider.
XI. L EVEL
OF MACROSCOPIC LOCOMOTION :
ARTIFICIAL HOMEOSTATIC HORMONE SYSTEMS
An option to generate SO processes, that are leveraged
to control complex multi-modular robot organisms, is the
mimicry of hormone systems. Our approach is based on
differential equations, that model the production, flow, and
reduction of hormones, and on structures of compartments
containing hormone concentrations that create the embodiment of the hormone system, see [36], [37]. The compartments establish a certain locality either within the robot or
within the robotic organism. Hormone productions, that are
triggered by sensor input, will establish a decreasing gradient
Figure 6. Schematic representation of a hormone concentration; in
a first phase, an artificial organism of two robotic modules moves
governed by an oscillating hormone concentration; in a second
phase, an environmental change (obstacle) disturbers this limit
cycle behavior; in a third phase, a third module connects to the
organism and another limit cycle is established.
within the robot/organism from the position of the sensor
to the opposite side. Homeostatic processes are intrinsic to
such systems and will be formed automatically. Thus, the
hormone system can be interpreted as a SO dynamic system.
Given a static sensor input or unchanged periodic sensory
stimuli for longer periods the system will always converge
to an attractor/equilibrium (e.g., fixed point, limit cycle, see
Fig. 6). A changing environment or the robots’ actions themselves change the sensory input, thus, disturbing the current
equilibrium. As a reaction, the hormone system rearranges
the hormone concentrations and will reach (possibly another)
equilibrium in a self-organized process (see Fig. 6).
Furthermore, there exists a second process besides the dynamic environment which is a disturbing factor to the current
equilibrium of the hormone values. A reconfigurable robot
organism consists of autonomous robot modules. Further
modules dock to or connected modules release themselves
from the organism. This change of the shape of the robot
organism has, in turn, an effect on all the other modules of
the organism (see Fig. 6).
In an organism forming a simple line (e.g., snake-like
shape, see Fig. 6), environmental influence triggers, through
sensory input, the production of hormones that result in
a gradient of hormone concentrations within the organism.
In a process of symmetry breaking a differentiation into a
head module and a tail module is generated. Furthermore,
a threshold of a “head-” and a “tail-hormone” determines
the positions for legs in the middle of the snake-organism.
In this way different body shapes are established by selforganized reconfiguration processes. We prefer SO as the
main design paradigm instead of standard approaches (e.g.,
predefined hand-coded shapes) because the latter would be
constricted to situations for which they were designed. However, our applications will have dynamic environments with
unforeseen properties. The approach of self-organization in
connection with evolutionary methods will help to overcome
the challenge of designing adaptive behavior in dynamic
environments. The possibility of self-reconfiguration gives
the organism the needed plasticity and adaptability.
XII. C ONCLUSION : C OMMON V IEW
In this paper we represented an overview over diverse
adaptive processes in artificial organisms, most of which
use different self-organizing approaches as main control
mechanisms. Since these approaches and mechanisms are
work-in-progress, we are not intended to give any detailed
results, in contrast we are going to create a common picture
of all of them.
Generalizing, we observe SO-related processes on three
following levels:
• The whole artificial organism is a SO-system. It
consists of a group of autonomous robots, whose behavior is regulated by their individual controllers. These
entities interact with each other and the environment,
thus producing group behavior that is not a simple
linear aggregation of the individual behaviors. We face
the canonical challenge of engineering SO systems
here: the point of impact of the designer/experimenter
is at local level (specifying the controller for each
individual), while the target behavior is specified on
the level of the group.
• There exist particular types of controllers that are
SO systems themselves. This means that the controller
cannot be described as a usual input-output machine
(like a rule-set or neural net, or decision tree). Instead, the controller itself represent a set of interacting
agents/rules and the response of this controllers to a
particular input is determined by the interaction rules.
• Developmental phase. There are methods for making/finding/developing controllers that fall in the category of SO systems. The difference with the option
2 is that in option 2 it is the result of the making/finding/developing (i.e., the controller) that is an
SO system, here it is the method leading to this result.
There are several open problems of creating artificial SO:
benchmarks, performance measurement and optimizing SOcontrol - currently it is unclear how to do this; increasing
predictability of self-organizing control - it is intended to use
different derivation and evolving approaches in achieving
desired emergence by SO; scalability and reliability - since
not all SO-mechanisms provide scalability, we need to
develop approaches guaranteeing scalability in a defined
range.
Finalizing this work, we would like to point out one important issue: artificial organisms can be viewed as extremely
simplified analogues of living organisms. Both living and
artificial organisms face similar problems – getting energy,
surviving in environment, different forms of self-protection
and self-awareness, organization of long-term and short-term
developmental processes and others. On the basis of artificial
organisms we can gain deeper insights into such issues
as long-term evolution and its controllability, phenomena
of individual and collective intelligence, mechanisms of
multi-cellular regulation and others issues which are highly
relevant in our understanding of the complexity of life.
ACKNOWLEDGMENT
The “SYMBRION” project is funded by the European
Commission within the work programme “Future and Emergent Technologies Proactive” under the grant agreement no.
216342. The “REPLICATOR” project is funded within the
work programme “Cognitive Systems, Interaction, Robotics”
under the grant agreement no. 216240. Additionally, we
want to thank all members of projects for fruitful discussions.
R EFERENCES
[1] K. Astrom and B. Wittenmark, Adaptive Control.
Wesley, 1989.
Addison
[2] G. Weiss, Multiagent systems. A modern approach to distributed artificial intelligence. MIT Press, 1999.
[3] M. Mataric02, “Situated robotics,” Encyclopedia of Cognitive
Science, Nature Publishers Group, Macmillian Reference
Ltd., 2002.
[4] R. Pfeifer, F. Iida, and G. Gomez, “Morphological computation for adaptive behavior and cognition,” International
Congress Series, no. 1292, pp. 22–, 2006.
[5] L. Bull, M. Studley, A. Bagnall, and I. Whitley, “Learning
classifier system ensembles with rule-sharing,” IEEE Transactions on Evolutionary Computation, vol. 11, no. 4, pp. 496–
502, August 2007.
[6] S. Kernbach, R. Thenius, O. Kernbach, and T. Schmickl, “Reembodiment of honeybee aggregation behavior in artificial
micro-robotic system,” Adaptive Behavior, vol. 17, no. 3, pp.
237–259, 2009.
[7] D. Cliff, “Biologically-inspired computing approaches to
cognitive systems: a partial tour of the literature,” HewlettPackard Company, 2003.
[8] A. Eiben and J. Smith, Introduction to Evolutionary Computation, ser. Natural Computing Series. Springer, 2003.
[9] S. Kornienko, O. Kornienko, and P. Levi, “Ir-based communication and perception in microrobotic swarms,” in Proc. of
the IROS 2005, Edmonton, Canada, 2005.
[10] S. Murata and H. Kurokawa, “Self-reconfigurable robot:
Shape-changing cellular robots can exceed conventional robot
flexibility,” IEEE Robotics & Automation Magazine, 2007.
[11] H. Haken, Synergetics; Introduction and Advanced Topics.
Springer, 1983.
[12] S. Kornienko, O. Kornienko, A. Nagarathinam, and P. Levi,
“From real robot swarm to evolutionary multi-robot organism,” in Proc. of the CEC2007, Singapore, 2007, pp. 1483–
1490.
[13] S. Kornienko, O. Kornienko, and P. Levi, “Generation of desired emergent behavior in swarm of micro-robots,” in Proc.
of the 16th European Conf. on AI (ECAI 2004), Valencia,
Spain, 2004.
[14] R. A. Watson, S. G. Ficici, and J. B. Pollack, “Embodied
evolution: Distributing an evolutionary algorithm in a
population of robots,” Robotics and Autonomous Systems,
vol. 39, no. 1, pp. 1–18, April 2002. [Online]. Available:
http://eprints.ecs.soton.ac.uk/10620/
[15] G. B. Müller, “Evo-devo: extending the evolutionary synthesis,” Nature Reviews Genetics, vol. 8, pp. 943–949, 2007.
[16] R. Thenius, T. Schmickl, and K. Crailsheim, “Novel concept
of modelling embryology for structuring an artificial neural
network,” in MATHMOD 2009 - 6th Vienna International
Conference on Mathematical Modelling, 2009.
[17] S. Nolfi and D. Parisi, “Auto-teaching: networks that develop
their own teaching input,” in Proc. Second European Conference on Artificial Life, J. Deneubourg, H. Bersini, S. Goss,
G. Nicolis, and R. Dagonnier, Eds., 1993.
[18] A. Grushin and J. A. Reggia, “Automated design of distributed control rules for the self-assembly of prespecified
artificial structures,” Robot. Auton. Syst., vol. 56, no. 4, pp.
334–359, 2008.
[19] A. Christensen, R. O’grady, and M. Dorigo, “Swarmorphscript: a language for arbitrary morphology generation in selfassembling robots,” Swarm Intelligence, vol. 2, no. 2, pp.
143–165, December 2008.
[20] E. Bonabeau, “Agent-based modeling: Methods and techniques for simulating human systems,” Proceedings of the
National Academy of Sciences USA, vol. 99, pp. 7280–7287,
2002.
[21] T. S. Tanaka, T. Kunath, W. L. Kimber, S. A. Jaradat, C. A.
Stagg, M. Usuda, T. Yokota, H. Niwa, J. Rossant, and M. S.
Ko, “Gene expression profiling of embryo-derived stem cells
reveals candidate genes associated with pluripotency and
lineage specificity,” Genome Research, vol. 12, no. 12, pp.
1921–1928, 2002.
[22] R. N. Aslin and J. Fiser, “Methodological challenges for
understanding cognitive development in infants,” Trends in
Cognitive Sciences, vol. 9, no. 3, pp. 92–98, 2005.
[23] J. D. Farmer, N. H. Packard, and A. S. Perelson, “The
immune system, adaptation, and machine learning,” Physica
D, vol. 22, pp. 187–204, 1986.
[24] W.-M. Shen, M. Krivokon, H. Chiu, J. Everist, M. Rubenstein,
and J. Venkatesh, “Multimode locomotion for reconfigurable
robots,” Autonomous Robots, vol. 20, no. 2, pp. 165–177,
2006.
[25] Y. Zhang, M. Yim, C. Eldershaw, D. Duff, and K. Roufas, “Scalable and reconfigurable configurations and locomotion gaits for chain-type modular reconfigurable robots,”
in International Symposium on Computational Intelligence in
Robotics and Automation, 2004, pp. 893–899.
[26] Y. Sugita and J. Tani, Mirror Neurons and the Evolution of
Brain and Language. John Benjamins Publishing, 2002, ch.
A connectionist model which unifies the behavioral and the
linguistic processes: Results from robot learning experiments,
pp. 363–376.
[27] E. Tuci, V. Trianni, and M. Dorigo, “’feeling’ the flow of
time through sensorimotor co-ordination,” Connection Science, vol. 16:4, pp. 301–324, 2004.
[28] J. Tani and S. Nolfi, “Learning to perceive the world as
articulated: an approach for hierarchical learning in sensorymotor systems,” Neural Networks, vol. 12, no. 7-8, pp. 1131–
1141, 1999.
[29] R. Yokoya, T. Ogata, J. Tani, K. Komatani, and H. Okuno,
“Experience-based imitation using rnnpb,” Advanced
Robotics, vol. 21, no. 12, pp. 1351–1367, 2007.
[30] S. P. McKibbin, B. Amavasai, A. N. Selvan, F. Caparrelli, and
W. A. Othman, “Recurrent neural robot controllers: feedback
mechanisms for identifying environmental motion dynamics,”
Artif. Intell. Rev., vol. 27, no. 2-3, pp. 113–130, 2007.
[31] S. P. McKibbin, B. Amavasai, A. N. Selvan, F. Caparrelli, and
W. Othman, “The role of sensory-motor coordination: Identifying environmental motion dynamics with dynamic neural
networks,” in Proceedings of the International Conference
on Informatics in Control, Automation & Robotics (ICINCO
2008), 2008.
[32] D. Vernon, G. Metta, and G. Sandini, “A survey of artificial
cognitive systems: Implications for the autonomous development of mental capabilities in computational agents,” IEEE
Transactions on Evolutionary Computation, vol. 11, no. 2,
pp. 151–180, 2007.
[33] L. Itti and C. Koch, “Computational modelling of visual
attention,” Nature Reviews Neuroscience, vol. 2, no. 3, pp.
194–203, 2001.
[34] J. Bongard, “Evolving modular genetic regulatory networks,”
in Proceedings of The IEEE 2002 Congress on Evolutionary
Computation (CEC2002), 2002, pp. 1872–1877.
[35] M. Shanahan, “A cognitive architecture that combines internal
simulation with a global workspace,” Consciousness and
Cognition, vol. 15, no. 2, pp. 433–449, 2006.
[36] T. Schmickl and K. Crailsheim, “Modelling a hormonebased robot controller,” in MATHMOD 2009 - 6th Vienna
International Conference on Mathematical Modelling, 2009.
[37] J. Stradner, H. Hamann, T. Schmickl, and K. Crailsheim,
“Analysis and implementation of an artificial homeostatic
hormone system: A first case study in robotic hardware,” in
IEEE/RSJ International Conference on Intelligent RObots and
Systems (IROS’09). IEEE Press, 2009.