An Iterated Function Systems Approach to Emergence

An Iterated Function Systems Approach to Emergence Abstract An approach to action selection in autonomous agents is presented.. Emergent behavior may be defined as the global behaviors o

An Iterated Function Systems Approach

to Emergence

Abstract

An approach to action selection in autonomous agents is presented This approach is motivated by biological examples and the operation of the Ran-dom Iteration Algorithm on an Iterated Function System The approach is illustrated with a simple three-mode behavior that allows a swarm of 200 computational agents to function as a global optimizer on a 30-dimensional multimodal cost function The behavior of the swarm has functional simi-larities to the behavior of an evolutionary computation (EC)

One of the most important concepts in the study of artificial life is emergent behavior (Langton 1990; Steels 1994) Emergent behavior may be defined

as the global behaviors of a system of agents, situated in an environment, that require new descriptive categories beyond those that describe the local behavior of each agent Most biological systems exhibit some form of self-organizing activity that may be called emergent Intelligence itself appears

to be an emergent behavior (Steels 1994), arising from the interactions of in-dividual “agents,” whether these agents are social insects or neurons within the human brain

Many approaches have been suggested for generating emergent behav-ior in autonomous agents, both individually (Steels 1994; Maes 1994; Sims 1994; Brooks 1986) and as part of a larger group of robots (Resnick 1994; Col-orni et al 1992; Beckers et al 1994; Hodgins and Brogan 1994; Terzopoulos

et al 1994) These concepts have also been applied to emergent computa-tion, where the behavior of the agent has some computational interpretation (Forrest 1991) Exciting results have been achieved using these approaches, particularly through the use of global, non-linear optimization techniques such as evolutionary computation to design the controllers

Almost all control architectures for autonomous agents implicitly assume that there exists a single, best action for every agent state (including sensor

hoskinsd@bcsaic.boeing.com

to appear in ”Evolutionary Computation IV: The Edited Proceedings of the Fourth Annual Conference on Evolutionary Programming”, J R McDonnell, R G Reynolds and D B Fogel, Eds MIT Press

states), although most include some form of wandering behavior In this pa-per, an approach is presented that explicitly assumes the opposite — that there are many “correct” actions for a given agent state In doing so, we hope to gain access to analytical methods from Iterated Function Systems (Barnsley 1993), ergodic systems theory (Elton 1987) and the theory of im-pulsive differential equations (Lakshmikantham et al 1989; Bainov and Sime-onov 1989), which may allow qualitative analysis of emergent behaviors and the analytical synthesis of behavioral systems for autonomous agents Many important applications of emergent behavior, such as air traffic control, in-volve problems where system failure could lead to significant loss of life These systems, at least, would seem to require some ability to specify bounds

on the emergent behavior of the system as a function of a dynamically chang-ing environment

Section 2 by motivates the approach with two examples from biology Section 3 reviews the concepts of Iterated Function Systems (IFSs) A new action selection architecture, the Random Selection Rule, is presented in sec-tion 4 This approach is applied to an emergent computasec-tion in secsec-tion 5 These results are discussed in section 5 Concluding remarks are offered in section 6

Chemotaxis in single-celled organisms, such as E coli, is one of the simplest

behaviors that might be called intelligent Intelligence in this case is ability

of the organism to control its long term distribution in the environment

E coli exhibits two basic forms of motion — ‘run’ and ‘tumble’ (Berg 1993).

Each cell has roughly six flagella These can be rotated, either clockwise

or clockwise A ‘run’ is a forward motion generated by counter-clockwise rotation of the flagella This rotation causes the flagella to “form

a synchronous bundle that pushes the body steadily forward.” A ‘tumble’ results from rotating the flagella clockwise This causes the bundle to come apart, so that the flagella move independently, resulting in a random change

in the orientation of the cell A cell can control its long term distribution in its environment by modulating these two behaviors This is true even though

the specific movements of the cell are not directed with respect to the

envi-ronment The cell detects chemical gradients in its environment, and

modu-lates the probability of tumbling, not the orientation of the cell after the

tum-ble By tumbling more frequently when the gradient is adverse the cell ex-ecutes a biased random walk, moving the cell (on average) in the favored direction

Intelligent behavior is also seen in more complex animals, such as hu-mans In this case, the behavior of the organism is determined, in large part,

by motor commands generated by the brain in response to sensory input The brain is comprised of roughly 10 billion highly interconnected neurons (Kandel and Schwartz 1985) The overall patterns of these interconnections are consistent between individuals, although there is significant variation from individual to individual Connections between neurons are formed

by two structures: axons and dendrites Synapses are formed where these structures meet

Signals are propagated between neurons in a two-step process:

to-pulse” and “pulse-to-wave” (Freeman 1975) The first step is the genera-tion of acgenera-tion potentials at the central body of the neuron These essentially

all-or-none pulses of depolarization are triggered when the membrane

po-tential at the central body of the neuron exceeds a threshold value They propagate outward along the axon, triggering the release of neurotransmit-ter molecules at synapses This begins the second step, pulse-to-wave con-version The neurotransmitters trigger the opening of chemically gated ion channels, changing the ionic currents across the membrane in the dendrite

of the post-synaptic neuron These currents are a graded response that is

in-tegrated, over both space and time, by the dynamics of the membrane po-tential The pattern, sense (excititory or inhibitory), and magnitude of these connections determine the effect that one neuron has on the remainder of the network The wave-to-pulse conversion of the action potential’s gener-ation and propaggener-ation constitute a multiplicgener-ation and distribution (in space and time) of the output of the neuron, while the pulse-to-wave conversion performed by membrane dynamics and spatial distribution of the dendrites acts to integrate and filter the inputs to a neuron Action potential genera-tion in a neuron appears to act like a random process, with the average fre-quency determined by the inputs to the neuron

The “neurons” in artificial neural networks are not intended to be one-for-one models of biological neurons, especially in networks intended for computational applications They are viewed as models of the functional behavior of a mass or set of hundred or thousands of neurons with similar properties and connectivity In these models, the wave-to-pulse and pulse-to-wave character of communication between many biological neurons is abstracted out, and replaced by a single output variable This variable is taken as modeling the instantaneous information flow between the neural masses This output variable is a function of the state(s) of the artificial neu-ron

This most common approach to modeling artificial neurons makes the implicit assumption that the information state of a collection of biologi-cal neurons is representable by a fixed length vector of real numbers, e.g points in Rn Other information states may be accessible to a collection of neurons interacting with randomly generated all-or-none impulses These

“states” would be emergent patterns of behavior, exhibited in the metric spaces “where fractals live” (Barnsley 1993)

Iterated Function Systems (IFSs) have been widely studied in recent years, principally for computer graphic applications and for fractal image com-pression (Barnsley 1993; Hutchinson 1981; Barnsley and Hurd 1993) This section summarizes some key results for Iterated Function Systems, as dis-cussed Barnsley (1993) Four variations on an Iterated Function System are discussed: an IFS, the Random Iteration Algorithm on an IFS, an IFS with probabilities and a recurrent IFS In addition, the convergence of the

behav-ior of these systems to unique limits in derived metric spaces will be discussed,

and an approximate mapping lemma will be proved

An IFS is defined as a complete metric space, (X, d), together with a finite collection of contractive mappings, W = {wi(x)|wi: X→ X, i = 1, , N}

Figure 1: Clockwise from upper left: The sets A 0 through A 4 in the generation of a

A mapping, wi, is contractive in (X, d) if

d(wi(x), wi(y))≤ sd(x, y)

for some contractivity factor, si ∈ [0, 1) The contractivity factor for the com-plete IFS is defined as s = maxi=1, ,N(si)

The following example illustrates the operation of an IFS Consider three mappings of the form:

wi(x) = 0.5(x− ηi) + ηi

where ηi= (0, 0), (1, 0),or (0.5, 1.0) for i = 1, 2, 3, respectively and x, ηi∈ X

If this collection of mappings is applied to a set, Ak, a new set is generated,

Ak+1 = W (Ak)

= w1(Ak)∪ w2(Ak)∪ w3(Ak)

Repeating, or iterating, this procedure generates a sequence of sets, {Ak} This process is illustrated in figure 1, where the set A0is a circle and square The first four elements of the sequence are also shown Each application of

Wgenerates three modified copies of the previous set, each copy scaled and translated according to one of the wi The sequence converges to a unique limit set, A This limit does not depend on the initial set, and is invariant

un-der W , that is A = W (A) This set is referred to as the attractor of the IFS It

is a single element, or point, in the Hausdorff metric space, (H(X) , h) Points

in H(X) are compact sets in X, and the metric, h, on H(X) is based on the metric d on X The attractor for the IFS in this example is sometimes called the Sierpinski triangle, or gasket

The RIA generates a sequence, {xk}∞

k=0, by recursively applying

individ-ual mappings from an IFS This sequence, known as the orbit of the RIA, will

almost always approximate the attractor of the IFS At each step, a mapping

Figure 2: The first 10,000 elements in the sequence generated by an RIA on the IFS for the Sierpinski triangle.

is selected at random from W and then used to generate the next term in the sequence That is, we generate {xk} by

xk= wσ k(xk−1) where x0is an arbitrary point in X, and σkis chosen randomly from {1, , N} The infinite sequence of symbols, z = σ1σ2σ3σ4 ., is a point in the code

space, Σ, sometimes known as the space of Bernoulli trials Each (infinite)

run of the RIA defines one such a point, z

The first 10,000 points of such a run are shown in figure 2 The points generated by the RIA are converging to the Sierpinski triangle The distance, d(xk, A), from the point to the set is dropping at each step by at least s, the contractivity of the IFS Moreover, the sequence will “almost always” come within any finite neighborhood of all points on the attractor as k → ∞

The RIA suggests the concept of an IFS with probabilities This type of

IFS associates a real number, piwith each wi, subject to the constraint that

PN i=1pi= 1, pi > 0 This gives different “mass” to different parts of the at-tractor of the IFS, even though the atat-tractor in H(X) does not change This

is illustrated in figure 3

The attractors for the two RIA’s illustrated in figure 3 contain exactly the same set points in the plane They differ only in the distribution of “mass”

on the attractor Just as the limiting set of an IFS is a point in H(X) , these limiting mass distributions are points in another derived metric space, P(X) P(X) is the space of normalized of Borel measures on X The metric on this space is the Hutchinson metric, dH (Barnsley 1993; Hutchinson 1981) The mapping, W , on a set in H(X) was the union of its constituent mappings

Figure 3: RIA on an IFS with probabilities, for two different sets of probabilities The

Its analog in P(X) is the Markov operator, M(ν), given by

M (ν) = p1ν◦ w1−1+ + pNν◦ w−1N

Thus an IFS with probabilities generates a sequence of measures {νk} in P(X) , with the invariant measure, µ = M(µ), as its unique limit,

The recurrent IFS is a natural extension of the IFS with probabilities to Markov chains It applies conditional probabilities, pij, rather than indepen-dent probabilities, pi, to the mappings in W Here pijmay be interpreted as the probability of using map j if map i was the last mapping applied It is required thatPpij = 1, j = 1, , N for each i and that there exist a non-zero probability of transitioning from i to j for every i, j ∈ {1, , N} An example of an RIA on a recurrent IFS is shown in figure 4 The IFS has seven available mappings Three of these mappings, played independently, gen-erate a Sierpinski triangle, while the other four fill in a rectangle The matrix

of transition probabilities, {pij}, for the Markov chain is:

{pij} =













0.3 0.3 0.3 0.025 0.025 0.025 0.025 0.3 0.3 0.3 0.025 0.025 0.025 0.025 0.3 0.3 0.3 0.025 0.025 0.025 0.025 0.03 0.03 0.04 0.225 0.225 0.225 0.225 0.03 0.03 0.04 0.225 0.225 0.225 0.225 0.03 0.03 0.04 0.225 0.225 0.225 0.225 0.03 0.03 0.04 0.225 0.225 0.225 0.225













The two sets of mappings may be thought of as two “modes,” each of which is an IFS with probabilities, that have a low (10%) probability of tran-sitioning from one mode to the other The resulting distribution combines both modes In the limit, each mode serves as the initial set, A0, for a se-quence of sets converging to the other A more common aspect of recurrent IFSs, especially in computer graphics, is the inclusion of zero elements in the transition matrix and the use of transitions between different metric spaces The central result for computer graphics applications of IFSs is the Col-lage Theorem (Barnsley 1993) It has several versions, for the various types

of IFSs This theorem shows that an IFS can be selected whose attractor is a specific image Specifically, for H(X) ,

Collage TheoremLet (X, d) be a complete metric space Let L ∈ H(X) be

Figure 4: RIA on a recurrent IFS.

given, and let ² ≥ 0 be given Choose an IFS {X, W } with contractivity factor

0≤ s < 1, so that

h(L, W (L))≤ ²

Then

h(L, A)≤ 1 ²

− s,

where A is the attractor of the IFS.

This result means that if a finite set of mappings can be discovered that approximately covers an image with “warped” copies of itself (wi(L)) , then the attractor of the IFS formed from those mappings will be close to the orig-inal image

The goal of this effort is the generation and analysis of RIA-like behav-ior in autonomous agents A key result for the RIA (and our application) is

a corollary of Elton’s theorem (Barnsley 1993; Elton 1987) The sequence, {xn}, is generated by an RIA on an IFS with probabilities The invariant measure for the IFS is µ

Corollary to Elton’s Theorem (Elton 1987)Let B be a Borel subset of X and let µ(boundary of B = 0 Let N (B, n) = number of points in {x0, x1, , xn} ∩

B, for n = 1, 2, Then, with probability one,

µ(B) = lim

n→∞

½

N (B, n)

n + 1

¾

for all starting points x0 That is, the “mass” of B is the proportion of itertion steps which produce points in B when the Random Iteration Algorithm is run.

In other words, the limiting behavior of the RIA will “almost always” approximate µ

Autonomous agents will, in many cases, be real, physical systems such

as mobile robots An approach to the qualitative analysis of their behavior must be able to account for modeling errors, as no model of a physical sys-tem is perfect A first step toward this is the following lemma regarding the behavior of an RIA in H(X)

Approximate Mapping LemmaLet W and W0be sets of N mappings, where the IFS defined by (X, W ) has contractivity s, and the individual mappings in W0

are approximiations to those in W Specifically, let

d(w(x), w0(x))≤ ²

for all x ∈ X, i = 1, , N Let {xi}∞

i=0and {x0

i}∞ i=0 be the orbits induced by a sequence of selections, σ1σ2σ3 ., begining from the same point, x0 = x00 ∈ X.

Then,

d(xk, x0k)≤ 1 ²

− s

for all k and x0 ∈ X.

The proof follows that of the Collage Theorem in H(X) Let ek= d(xk, x0k) Then we have a strictly increasing sequence of error bounds, {ek}, where

e0 = 0

e1 = ²

≥ d(x1, x01)

The contractivity of W , and the assumed bound on mapping errors combine

in the triangle inequality to propagate the error bound:

d(xk+1, x0k+1) = d(w(xk), w0(x0k))

≤ d(w(xk), w(x0k)) + d(w(x0k), w0(x0k))

≤ sd(xk, x0k) + ² Replacing d(xk, x0

k)by ekgives a bound on d(xk+1, x0

k+1):

ek+1 = sek+ ²

= ²

k

X

i=0

si

and, as k → ∞, e = ²

1+s

This result provides a bound on the orbit whose actions at each step are close to that of a model system Note that the mappings in W0 are not re-quired to be strictly contractive Instead the assumptions of the lemma only require that

d(wi0(x), w0i(y))≤ sd(x, y) + 2² for all x, y ∈ X

The predictable long term behavior of the RIA, and Elton’s results in partic-ular, suggests a definition of emergent behavior — the characteristics of a

system’s response that are not sensitive to initial conditions.

Definition (tentative): Emergent BehaviorAn emergent behavior is de-fined as a repeatable or characteristic response by the elements of a system of in-teracting autonomous agents To state that a system exhibits an emergent behavior

is equivalent to asserting the existence of some attribute of the system that arises,

or evolves, predictably when agents interact with each other and their environment.

We will call the behavior of the system emergent with respect to Q for some testable property, Q(χ(t)), if that property holds for almost all trajectories, χ(t).

For example, Q(χ(t)) might be that the distribution of agent positions approaches a specific measure in the limit, as in the corollary to Elton’s the-orem

This definition treats emergence as a generalization of stability Asymp-totic stability in the usual sense is a trivial example of emergence under this definition Emergence becomes non-trivial if the attractor is a more complex property of the trajectory, such as it’s limiting distribution in the state space The RIA also suggests an action selection strategy for autonomous agents This strategy is called the Random Selection Rule (RSR) Where an RIA generates an abstract sequence of points in a metric space, state tra-jectories for a mobile robot must be at least piecewise continuous (includ-ing control states) This is accomplished in the RSR by augment(includ-ing the basic RIA structure with an additional set of functions to define the dura-tion of each behavior Where the RIA generates a point in a code space:

a = σ1σ2σ3 ., the RSR generates points in an augmented code space: α = (σ1, δ1)(σ2, δ2)(σ3, δ3) It is conjectured that the results obtained for Iter-ated Function Systems can be extended to this augmented space, and used with the above definition to characterize the emergent behavior of interact-ing, situated autonomous agents

Specifically, an RSR has three components:

1 Finite Behavior Suite: Each agent has a finite set of possible agent

be-haviors (e g closed loop dynamics), B = {bi}, i = 1, , N During the interval, (tk, tk+1], the state of the agent is given by

x(t) = bσ k(t, tk, x(tk))

2 Random Behavior Selection: A selection probability function,

p(x, σk −1, m), must be defined for each behavior Dependence on the agent state, x, implicitly incorporates reactive, sensor driven behav-ior, since sensor inputs necessarily depend on the state of the agent

— its position and orientation Dependence of the behavior selection probability on the identity of the current behavior, σk−1, echoes the notion of a recurrent IFS, and provides a method of encorporating multiple modes of behavior Finally, the action selection probability may depend on messages, m, transmitted between agents This type

of dependence is not explicitly covered under standard IFS theory It’s significance and a proposed mechanism for applying IFS theory

to the behavior of the collection are discussed below

3 Distinct Action Times: Agents exhibit only one behavior at any given

time The length of time that a behavior is applied before the next ran-dom selection is made is defined by a duration function, δ(tk−1, x) >

0 This function may be as simple as a fixed interval, or as complex

as an explicit dependence on tk−1and x, so that the new action selec-tion occurs when the state trajectory encounters a switching surface

We require only that the duration be non-zero In the present work, the duration is an interval drawn from an exponential distribution, so that the sequence of action selection times, {tk}, is a sequence of Pois-son points

Dependence of the behavior selection probabilities on messages from other agents presents both a problem and an opportunity On the one hand, the remainder of the population may be viewed as simply part of a dynam-ically changing environment, so that the messages are simply another sen-sor input On the other hand, we may treat a collection of agents as a single

“swarm agent” or a “meta-agent.”

First, a collection of M agents has a well defined state It is simply the set

of M states, χ(t) = {x(1), , x(M )

} Second, it has a well defined, finite set

ofQM i=1n(i)distinct behaviors, where n(i) is the number of behaviors avail-able to the ith agent Third, there is a behavior selection probability func-tion associated with each possible behavior This probability may depend

on the previous behavior state and on the current state, χ(t), of the

collec-tion of agents Messages between agents are internal to these seleccollec-tion funccollec-tions.

Fourth, there is a well defined sequence of action times {tk} for the swarm agent It is simply the union of the sequences for the component agents The behavior duration function is simply δk= tk+1− tk

In sum, a swarm of communicating agents implementing an RSR is it-self a “agent”, with a well defined state, operating under an RSR This raises the possibility of the recursive construction of complex swarm behaviors from simple elemental behaviors Moreover, such complex behaviors may

be subject to analysis, or even analytic design, if the mathematics of IFSs and related results can be extended to cover the behavior of the Random Selec-tion Rule for the elemental behaviors

The simulation results presented below are for a swarm of computa-tional RSR agents, whose behaviors that are simple contractive mappings followed by an exponentially distributed wait at the resulting position This generates a piecewise continuous trajectory that is a series of “sample and holds.” Action selection probabilites depend on sensing (of a cost function), previous behavior (to allow multiple modes) and communication, to facili-tate an indirect competition for a favored mode

The RSR approach was used to define a set of behaviors for a swarm of 200 simulated agents The “sensor inputs” to these agents consist of evaluation

of a cost function in a 30-dimensional search space, and a limited amount