Psychological review psychological entropy a framework for understanding uncertainty related anxiety

Four major tenets of EMU are proposed: a Uncertainty poses a critical adaptive challenge for any organism, so individuals are motivated to keep it at a manageable level; b uncertainty em

Psychological Review

Psychological Entropy: A Framework for Understanding Uncertainty-Related Anxiety

Jacob B Hirsh, Raymond A Mar, and Jordan B Peterson

Online First Publication, January 16, 2012 doi: 10.1037/a0026767

CITATION

Hirsh, J B., Mar, R A., & Peterson, J B (2012, January 16) Psychological Entropy: A

Framework for Understanding Uncertainty-Related Anxiety Psychological Review Advance

online publication doi: 10.1037/a0026767

Psychological Entropy: A Framework for Understanding

Uncertainty-Related Anxiety

Jacob B Hirsh

University of Toronto

Raymond A Mar

York University

Jordan B Peterson

University of Toronto

Entropy, a concept derived from thermodynamics and information theory, describes the amount of uncertainty and disorder within a system Self-organizing systems engage in a continual dialogue with the environment and must adapt themselves to changing circumstances to keep internal entropy at a manageable level We propose the entropy model of uncertainty (EMU), an integrative theoretical framework that applies the idea of entropy to the human information system to understand uncertainty-related anxiety Four major tenets of EMU are proposed: (a) Uncertainty poses a critical adaptive challenge for any organism, so individuals are motivated to keep it at a manageable level; (b) uncertainty emerges as a function of the conflict between competing perceptual and behavioral affordances; (c) adopting clear goals and belief structures helps to constrain the experience of uncertainty by reducing the spread of competing affordances; and (d) uncertainty is experienced subjectively as anxiety and is associated with activity in the anterior cingulate cortex and with heightened noradrenaline release By placing the discussion of uncertainty management, a fundamental biological necessity, within the framework of information theory and self-organizing systems, our model helps to situate key psycho-logical processes within a broader physical, conceptual, and evolutionary context

Keywords:entropy, uncertainty, anxiety, behavioral inhibition, self-organization

Recent years have witnessed a growing interest in the topic of

uncertainty (Heine, Proulx, & Vohs, 2006; Hogg, 2000;

McGregor, Zanna, Holmes, & Spencer, 2001; Peterson, 1999; van

den Bos, 2001) As the body of research on uncertainty continues

to grow, the need for an integrative theoretical framework to

establish its psychological significance and provide a context for

its neural underpinnings and behavioral consequences has become

increasingly apparent In the current article, we propose that the

concept of entropy as derived from information theory provides a

useful framework for understanding the nature and psychological

impact of uncertainty By drawing upon dynamical models of

self-organizing systems, we argue that uncertainty presents a

fun-damental (and unavoidable) challenge to the integrity of any

complex organism The entropy-based model developed

through-out this article provides an organizing framework for

understand-ing the critical importance of uncertainty management for an

individual’s survival, well-being, and productivity, situated within

a broader evolutionary and physical context In doing so, it helps

to draw together numerous research literatures in which

uncer-tainty plays an important role, integrating them into a coherent theoretical framework for conceptualizing the neural and behav-ioral responses to uncertain situations

The article proposes the entropy model of uncertainty (EMU), a framework based on four major tenets: (a) Uncertainty poses a critical adaptive challenge for any organism, so individuals are motivated to keep it at a manageable level; (b) uncertainty emerges

as a function of the conflict between competing perceptual and behavioral affordances; (c) adopting clear goals and belief struc-tures helps to constrain the experience of uncertainty by reducing the spread of competing affordances; and (d) uncertainty is expe-rienced subjectively as anxiety1and is associated with activity in the anterior cingulate cortex and heightened noradrenaline release

We begin by describing the origins and definitions of the en-tropy construct, outlining its relevance for biological organisms in general and human behavior in particular We then apply this idea

to cognitive processes by introducing the construct of

psycholog-ical entropy, defined as the experience of conflicting perceptual and behavioral affordances We next examine how EMU accounts for our current understanding of the neurophysiology of uncer-tainty Finally, we discuss how the cognitive and behavioral con-sequences of heightened uncertainty can be understood within this entropy-based framework

1We are using the term anxiety in the same manner as Gray and

McNaughton (2000), who distinguished it from the emotion of fear This distinction is further elaborated upon later

Jacob B Hirsh, Rotman School of Management, University of Toronto,

Toronto, Ontario, Canada; Raymond A Mar, Department of Psychology,

York University, Toronto, Ontario, Canada; Jordan B Peterson,

Depart-ment of Psychology, University of Toronto

Correspondence concerning this article should be addressed to Jacob B

Hirsh, 105 St George Street, Toronto, Ontario M5S3E6, Canada E-mail:

jacob.hirsh@rotman.utoronto.ca

1

Rudolf Clausius (1865), working in the field of

thermodynam-ics, originally defined entropy as the amount of energy within a

system that cannot be used to perform work (i.e., cannot be used

to transform the system from one state to another) Maximum

entropy occurs during complete thermodynamic equilibrium, when

energy is equally dispersed across all parts of a system At this

point, no useful work can be performed as work always depends on

the movement of energy from one area to another

Ludwig Boltzmann (1877), a defining figure in statistical

me-chanics, extended this work by defining entropy as a function of

the number of microstates that could potentially comprise a

par-ticular macrostate, mathematically linking this definition to

Clau-sius’s thermodynamic concept The more microstates that are

possible, given any particular macrostate, the higher the entropy of

the observed system In this respect, entropy reflects the amount of

uncertainty about a system: The greater the number of plausible

microstates, the more uncertainty about which microstate currently

defines the system

Since World War II, the concept of entropy has been generalized

to all information systems, not just thermodynamic ones Claude

Shannon (1948), a seminal figure in the field of information

theory, defined entropy as the amount of uncertainty associated

with a random variable Shannon demonstrated that the information

content of a given signal could be measured as a function of the

number of signals that could potentially have been received The

information content of a signal is thus quantified in relation to

the amount of uncertainty reduced by receiving the message; this is

directly linked to the prior distribution of possible outcomes

Although these various definitions of entropy are all

mathemat-ically related, only this latter conceptualization has the advantage

of generalizing to a broad range of information systems (Jaynes,

1957; Pierce, 1980) Building on this work, Norbert Wiener

(1961), the founder of cybernetics, defined entropy as the

disor-ganization within cybernetic information systems (goal-directed

self-regulating systems) As a system’s disorder and uncertainty

increase, its ability to perform useful work is hampered by reduced

accuracy in specifying the current state, the desired state, and the

appropriate response for transforming the former into the latter

It is the cybernetic view of entropy that plays a prominent role

in modern nonlinear dynamical systems approaches These

ap-proaches deal with the emergent properties of complex systems,

with an emphasis on tendencies toward self-organization (Nicolis

& Prigogine, 1977) Self-organization describes the emergence of

a patterned structure of relationships between the constituent

ele-ments of a complex system (Ashby, 1947, 1956) High-entropy

states, in this context, reflect a lack of internal constraints among

the system’s interacting parts, such that knowing the state of one

component provides minimal information about the others

Be-cause dynamical systems continually change over time, entropy

can be related to the predictability of successive states given

knowledge of the current state (Shannon & Weaver, 1949)

According to the second law of thermodynamics, the amount of

entropy within a closed system can only increase over time Any

mechanical process involves some irreversible energy loss (e.g.,

inefficiencies resulting in heat loss), and heat will not move from

a colder body to a warmer body without additional energy input

Accordingly, a system moves closer to a state of thermodynamic

equilibrium as it performs work Unless more energy is added, the amount of potential work it can produce will inevitably decrease with time Information systems also lose energy over time as a result of inefficiencies, so they too will eventually dissipate and dissolve unless additional energy is incorporated to sustain struc-tural coherence and minimize internal disorder We propose that understanding the relationship between entropy and the potential

of systems to perform work (i.e., to pursue and achieve goals) can illuminate the significance of uncertainty to biological systems in general and psychological systems more specifically

Entropy Management as a Fundamental Principle of

Organized Systems

Application of the second law of thermodynamics to psychology produces the first major tenet of EMU, that uncertainty poses a critical adaptive challenge, resulting in the motive to reduce uncertainty This tenet is partly predicated on research examining the emergence and maintenance of order within complex systems In his groundbreaking

book, What Is Life?, the physicist Erwin Schrödinger (1944) argued

that living systems survive by reducing their internal entropy, while simultaneously (and necessarily) increasing the entropy that exists in their external environment Although the total amount of entropy in the universe as a whole can only increase (as expressed in the second law of thermodynamics), living organisms can stem the rise of en-tropy found within their biological systems by consuming energy from the environment, using it to maintain the integrity and order of their own biological systems, and displacing their entropy into the outside world

In the dynamical systems literature, this entropy-reduction framework has been extended to the view of biological organisms

as dissipative systems (Prigogine & Stengers, 1997) For an

organ-ism to survive, it must effectively dissipate its entropy into the environment Dissipative systems are open systems operating far from thermodynamic equilibrium, requiring energy intake to sus-tain a stable structural organization If the environment changes to produce more entropy for an organism (thereby challenging its structural coherence), that organism must adopt new patterns of self-organization that are capable of accommodating the environ-mental changes Self-organization describes the process by which novel dissipative structures emerge in response to higher entropy levels Dynamical systems theorists therefore propose that stable information systems survive only insofar as they are able to effectively manage their internal entropy Those that cannot effec-tively dissipate this entropy are destroyed, in a Darwinian fashion (Kauffman, 1993) One consequence of this process is that com-plex systems tend to return to a relatively small number of stable,

low-entropy states (known as attractors; Grassberger & Procaccia,

1983) This is because the vast majority of states that these systems could theoretically inhabit do not provide effective entropy man-agement and are therefore characterized by instability

Given that the principles of entropy and self-organization can be employed to examine any complex information system, it may not

be surprising that these frameworks have also been used to study psychological phenomena (Barton, 1994; Carver & Scheier, 2002; Hollis, Kloos, & Van Orden, 2009; Vallacher, Read, & Nowak, 2002) For instance, researchers have observed self-organizing dynamics during the problem-solving process (Stephen, Bon-coddo, Magnuson, & Dixon, 2009; Stephen, Dixon, & Isenhower,

2009) In particular, as an initially adopted strategy becomes

ineffective, a quantifiable increase in the entropy of the

problem-solving behavior is observed (measured as the irregularity and

unpredictability of participants’ responses) This increase in

be-havioral entropy precedes subsequent changes in solution strategy,

followed by a return to predictable, stable, low-entropy behavioral

patterns What this suggests is that cognitive-behavioral systems

follow the same basic principles as other dissipative systems If the

system finds itself unable to effectively handle environmental

challenges, its internal entropy levels will increase and force the

adoption or development of alternative cognitive structures

Alter-natively, if such structures cannot be found, the system may fail to

adapt, become overwhelmed, and start to deteriorate

Similar interpretive frameworks have been applied to

under-standing the neural substrates of cognitive operations In

particu-lar, a number of techniques for quantifying entropy levels within

neural systems have been developed (Borst & Theunissen, 1999;

Nemenman, Bialek, & de Ruyter van Steveninck, 2004; Paninski,

2003; Pereda, Quiroga, & Bhattacharya, 2005; Strong, Koberle, de

Ruyter van Steveninck, & Bialek, 1998; Tononi, Sporns, &

Edel-man, 1994) Several models of neural functioning suggest that

patterns of neural activity are also characterized by transitions

between familiar low-entropy attractor states, albeit within the

context of a great deal of complexity and chaotic activity (Amit,

1992; Tsuda, 2001) Karl Friston and colleagues, for example,

have explicitly emphasized the importance of entropy

minimiza-tion as an organizing principle of neural funcminimiza-tion (Friston, 2009,

2010; Friston, Kilner, & Harrison, 2006) According to these

authors, an important goal of any nervous system is to minimize

the experience of entropy and unpredictability by continually

modifying neural structures in response to environmental

informa-tion that arises during goal pursuit They proposed that the

mini-mization of entropy at the neural level supports cognitive and

behavioral adaptation at the level of the individual by providing

more pragmatically adaptive representations of the environment

Within a dissipative systems context, the brain is able to adapt to

changing environmental events and contingencies by continually

reforming its patterns of structural organization, minimizing the

entropy that is encountered while trying to satisfy the organism’s

basic needs (Friston, 2010; Kelso, 1995) The effort to reduce the

spread of entropy is an ongoing process, as entropy levels will

continually fluctuate as the brain shifts through dynamic patterns

of activation and connectivity It is the psychological experience of

entropy that we focus on in our model and to which we now turn

Psychological Entropy: Uncertainty in

Perception and Action

From an evolutionary perspective, the fundamental goal of a

nervous system is to integrate appropriate perceptual frames and

behavioral responses with the steady flow of sensory information,

so that biological needs can be adequately satisfied (Swanson,

2003) Consequently, there are two primary domains of

uncer-tainty that must be contended with from a psychological

perspec-tive: uncertainty about perception and uncertainty about action

The second major tenet of EMU, elaborated below, is that

uncer-tainty can be understood psychologically in terms of the

conflict-ing actions and perceptions that can potentially be brought to bear

on a given situation

In any situation, the organism is presented with an array of perceptual and behavioral affordances that specify the possible actions that can be implemented (Gibson, 1979) These affor-dances reflect the combination of incoming sensory information with the cognitive and behavioral potentialities of the organism (Cisek, 2007; Cisek & Kalaska, 2010; Warren, 2006; Zhang & Patel, 2006) The EMU conceptualizes both the perceptual and behavioral domains as probability distributions Perception can be understood as the interpretation of sensory input in accordance with expectations, motives, and past experience Accordingly, there is a probability distribution of potential meanings and per-ceptual experiences that can be derived from any given array of sensory input This distribution is influenced by both the structure inherent within the input itself and the structure of the perceptual system doing the interpreting Similarly, in any moment, there is a probability distribution of possible actions that can be brought to bear on the environment Importantly, these probability distribu-tions are in part subjectively defined

We contend that the amount of uncertainty associated with a given perceptual or behavioral experience can be quantified in terms of Claude Shannon’s entropy formula, which reflects the negative sum

of the log probabilities of each possible outcome (see Figure 1A) This formulation indicates that low-entropy levels are reflected in proba-bility distributions in which some outcomes are much more probable than others (see Figure 1B) High-entropy levels, in turn, are associ-ated with flatter probability distributions, in which no outcome is clearly more likely than the others (see Figure 1C)

According to this framework, uncertainty, or psychological en-tropy, therefore varies as a result of any experience that alters the shape of these probability distributions The probability of any given action or perception taking place, represented

mathemati-cally as p(x i), is a function of the weighted neural input for that possibility (relative to other possibilities) during the moment of experience Computationally, the selection of competing affor-dances appears to occur through a process of parallel constraint satisfaction and pattern recognition (Bishop, 2006; Rogers & Mc-Clelland, 2004; Rumelhart & McMc-Clelland, 1986), during which the brain’s neural networks attempt to find the most appropriate in-terpretive frame for a situation, given the current pattern of acti-vations (reflecting perceptual input, motivational frames, embod-ied motor state, etc.) In a simple connectionist network, the strength of activation for any possible action or perception will depend on its combined inputs from sensory experience and mem-ory representations The strengths of these inputs are in turn influenced by selective attention processes that prioritize informa-tion relevant to the current goal (see Figure 1D).2

2EMU differs from previous applications of Shannon’s formula to cognitive psychology (e.g., Hick, 1952) in that it does not focus exclusively

on the distribution of objective stimulus characteristics (e.g., the number of response buttons) Rather, our model of uncertainty focuses on the weighted distribution of potential actions and perceptions as subjectively experienced by the individual This distribution is a function of both the objective stimulus characteristics and the individual’s current repertoire of perceptual and behavioral habits Consequently, EMU is not affected by the same limitations affecting some of these classic information-theoretic approaches that focus only on the distribution of external stimulus char-acteristics (Luce, 2003)

If the environment is well specified (i.e., personally familiar),

the brain is able to settle relatively quickly into a particular

perceptual-behavioral frame, based on patterns of habitual

re-sponding and reliable estimates of likely outcomes The brain’s

operation during these familiar situations is relatively efficient, as

there is a rapid matching of environmental input with habitual

perceptual and behavioral patterns (reflecting a deep attractor

basin in the neural network) The EMU framework describes these

situations as states of low entropy because the distributions of possible meanings and actions are heavily weighted toward a single dominant affordance Progress toward long-term goals can

be reached in such circumstances via a simple process of means– ends analysis, using already instantiated procedures, perceptions, and suppositions

The EMU framework proposes that low-entropy distributions such as those that obtain for familiar situations are characterized

by strong neural inputs for a single affordance This results in a greater degree of computational constraint when interpreting the situation and selecting the appropriate response (i.e., there is a lack

of neural competition for alternative outputs) High-entropy dis-tributions, in contrast, have reduced constraint due to a lack of clearly dominant inputs to the perceptual and behavioral systems These distributions are, accordingly, characterized by higher levels

of neural competition and ambiguity (as obtains more often in unfamiliar or unexpected situations) EMU proposes that the amount of uncertainty that an individual will experience in any given situation emerges as a function of the degree of constraint that is placed upon the interpretation of sensory information and the selection of behavioral responses As indicated by Shannon’s formula, the amount of uncertainty (expressed as entropy) will increase in proportion to the number of competing possibilities that must be selected from Unconstrained situations with a large range

of perceived possibilities will result in states of relatively greater uncertainty, while constrained situations with a narrow range of possibilities will result in states of relatively less uncertainty The first major tenet of the EMU framework indicates that because individuals will be motivated to reduce the experience of uncer-tainty to a manageable level, psychological discomfort will in-crease along with the degree of perceptual and behavioral ambi-guity within a situation

While some of the constraint on action and perception emerges from the structural and functional limitations of the human body and brain in combination with past experience, the third major tenet of EMU is that additional constraints are provided by the goals that the individual is pursuing There is growing evidence that the goals adopted by an individual serve to bias both percep-tion and acpercep-tion in line with goal-relevant informapercep-tion and behav-ioral options (Aarts, 2007; Bargh & Chartrand, 1999; Bargh, Gollwitzer, Lee-Chai, Barndollar, & Trötschel, 2001) A similar notion is emphasized by clinical psychologists who attempt to help their clients move beyond the narrow horizons provided by mal-adaptive goals that are difficult or impossible to achieve (e.g., Hayes, 2004) From a neural perspective, goal-related biasing of information flow appears to be instantiated by top-down atten-tional control mechanisms in the dorsolateral prefrontal cortex, which constrain the activation of perceptual and motor programs in the rest of the brain (E K Miller, 2000) As an individual’s goals change, so does the distribution of possible meanings and actions that can be derived from the same experience From a self-organizing systems perspective, goals thus operate as the attractors around which human behavior is organized (Carver & Scheier, 2002)

We propose that whenever a goal is selected, the distribution of possible actions and perceptions that are afforded to an individual

is weighted toward those behaviors and interpretive frames that can most efficiently result in movement toward the desired state Computationally, it appears that this process can be described

Figure 1. A: Shannon’s formula for information entropy Entropy

in-creases as the number of possible outcomes inin-creases and the probability of

any particular outcome, p(x i), decreases B: Low psychological entropy

occurs during situations in which there is a high probability of employing

a particular action or perceptual frame, x i C: High psychological entropy

occurs during situations in which there are multiple competing frames and

behavioral options (e.g., x1, x2, x3), none of which is clearly more strongly

activated than the others D: The probability of any given action or

perceptual frame being employed, p(x i), is a function of the weighted

neural input for its deployment, as influenced by the combination of

sensory input, strength of memory representations, and goal-related

atten-tional processes

within the framework of optimal control theory, the application of

which allows for the calculation of the optimal path to a goal,

while minimizing the cost function associated with goal pursuit

(Todorov, 2004; Todorov & Jordan, 2002) For instance, an

indi-vidual who wishes to find a drink of water is more likely to walk

to the water cooler in the next room than the water cooler in the

next building Implementation of either plan would satisfy the

goal, but the first is much more efficient, in that it conserves

valuable metabolic and material resources In this sense, optimal

control processes help to maintain an economy of action and

perception, ensuring the efficient pursuit of a goal Behaviors that

appear to provide the optimal (i.e., most efficient) path to a goal in

any given moment thus come to be weighted more heavily in the

distribution of possible actions More complicated higher order

goals can also be optimized in such a manner by using dynamic

programming techniques that operationalize complex goals as a

series of subgoals that can in turn be optimized (Bellman, 1952,

1957; Sutton & Barto, 1981)

It should be noted, however, that the activation of potential

actions is a function of their perceived values, rather than their

objective utilities Consequently, the weighted distribution of

po-tential actions will not necessarily conform to classic economic

models of rational decision making based on expected utility

calculations (Schoemaker, 1982) Rather, the distribution will be

influenced by the numerous psychological biases that characterize

human decision making, such as loss aversion and the

overweight-ing of extreme but unlikely outcomes (Kahneman & Tversky,

1979; Tversky & Kahneman, 1992), as well as other

temperamen-tal biases in perception and action While there are many specific

biases and preferences that influence the relative activation of each

possible action (i.e., determining the specific choice that will be

made), these are not the primary focus of the EMU framework

Rather, EMU pertains mainly to the weighted distribution of these

perceptual frames and possible actions and the amount of entropy

that characterizes such distributions

While goals provide an important source of constraint for the

cognitive system, EMU does not suggest that all goals provide the

same degree of constraint on the moment-to-moment distribution

of perceptual and behavioral affordances Even the same goal can

provide varying levels of constraint in response to different events

that facilitate or threaten the goal-pursuit process In particular, a

goal can reduce uncertainty only as long as it allows for rapid

functional categorization of sensory information as well as

calcu-lation of the (subjectively perceived) optimal response to any

given situation Poorly defined or vague goals are therefore less

likely to provide effective uncertainty-reducing effects, as they are

incapable of sufficiently narrowing the range of potentially

rele-vant affordances

Additionally, EMU posits that psychological entropy levels rise

whenever the number of obstacles to obtaining a currently selected

goal increases Each of these obstacles will contribute additional

uncertainty and inefficiency to the situation, making it harder to

compute the optimal action and interpretive frame and,

conse-quently, flattening the probability distribution of affordances The

result of these emerging obstacles is that more work will be needed

to transform the system’s current state (e.g., hungry and wanting to

find food) to the desired state (e.g., full and in possession of food)

In terms of optimal control theory, these obstacles increase the

cost function associated with a particular behavioral strategy and,

therefore, reduce the weighted activation of related affordances If the obstacles to obtaining a goal become too severe, the integrity

of the goal-pursuit process may be threatened, which EMU pre-dicts would reduce the system’s ability to maintain effective con-straints on perception and behavior Removal of these concon-straints results in heightened uncertainty and less efficient goal pursuit These principles apply equally to simple biological goals (e.g., satiating hunger) and more abstract higher order and long-term goals (e.g., pursuing a career) If an individual wants to become a famous musician, for example, his or her perceptual and behav-ioral affordances will be weighted toward goal-relevant opportu-nities and actions The integrity of that goal (reflecting its coher-ence and attainability) can be weakened by the emergcoher-ence of obstacles that increase the work needed to obtain the desired state (e.g., a broken hand), or it can be strengthened by events that reduce the distance of the goal (e.g., befriending a record pro-ducer), with concomitant changes in the experience of uncertainty Events in the world can increase uncertainty by adding obstacles (jeopardizing the current plan) or reduce uncertainty by providing

a clear path to the goal (increasing the efficiency of the current plan)

More generally, a plan involves the estimation of the optimal action for achieving a goal, taking into consideration (a) a partic-ular starting point, (b) a desired end point, and (c) the steps required to transform the original state into the end goal state (Austin & Vancouver, 1996; G Miller, Galanter, & Pribram, 1971; Schank & Abelson, 1977) Each time that goal-relevant informa-tion is received, the costs and probabilities of attaining the out-come using the current strategy have to be recalculated (with greater or lesser uncertainty being introduced in the process) While the natural tendency of all information systems is to return

to a state of dissolution and energy dispersal, behavioral plans help organisms to minimize their overall entropy levels (i.e., strength-ening their coherence as a functional entity) by providing clear and specific strategies for acquiring needed resources in the face of uncertainty and determining the appropriate way to interpret and respond to environmental input Effective plans are thus essential tools for combating the inevitable thermodynamic dissolution that comes with time, as they help to maintain the structural integrity of complex biobehavioral systems

Entropy and Combinatorial Explosion

During Uncertainty

As described above, EMU proposes that the entropy experi-enced by a goal-directed system is inversely related to the amount

of perceptual and behavioral constraint provided by a goal The extent to which a goal is able to effectively provide such con-straints is also related to the work needed to attain the goal or, alternatively, the probability of obtaining the goal based on avail-able actions The work that is required during goal pursuit can be considered in terms of the path length to goal attainment In some cases, the path length is relatively short, requiring minimal effort, few steps, or transformations of state to achieve the goal and typifying an efficient low-entropy situation of high stability In other cases, the path length to a goal is relatively long This is more likely with complex goals that subsume numerous subgoals While such goals certainly take a longer time to achieve, the system that holds them will remain in a state of relatively low entropy as long

as the behavioral path is well specified and the necessary resources

are available Note that it is not the number of subgoals per se that

influences entropy levels but rather their specificity and perceived

attainability given current knowledge and resources (cf Maddux,

Norton, & Stoltenberg, 1986)

A very different process occurs during situations of uncertainty,

however Uncertainty arises when plans are unexpectedly

dis-rupted (e.g., by the emergence of unforeseen obstacles) and the

appropriate perceptual frame and behavioral response are not made

immediately clear When no alternative paths to achieving a

de-sired goal are apparent, there is a massive increase in entropy as

the individual’s well-delineated plan of action gives way to

uncer-tainty about the best way to construe the situation and move

toward the goal (and indeed, whether it is even possible to do so;

Bandura, 1982, 1988) In some cases, the disruption of a plan will

be caused by a well-understood event (e.g., a flat tire), in which

case only behavioral uncertainty will result (i.e., “What should I

do?”) In other cases, the nature of the disrupting event itself will

not be immediately clear (e.g., an unexpected earthquake),

result-ing in both behavioral and perceptual uncertainty (i.e., “What is

happening, and what should I do?”) While a plan can reduce

uncertainty by specifying a dominant behavioral response and

interpretive frame, its disruption results in the emergence of a

high-entropy distribution of environmental affordances

To understand how an individual’s experience of the world can

change so dramatically during states of uncertainty, it is important

to remember that the environment is not experienced directly

Subjective experience is based on partial, incomplete, and

prag-matically driven representations of the environment As a result,

the experience of the environment (including its meaning and

perceptual contours) can change suddenly and quite considerably

when perceptual assumptions or behavioral habits are challenged

by completely unexpected events that undermine goal pursuit

Under such situations, the appropriate response is no longer clear,

and the value and nature of encountered objects become uncertain,

concomitant with the sudden activation of competing perceptual

and behavioral affordances that were previously constrained by the

no-longer dominant goal

Not all experiences of uncertainty are equally severe

Uncertainty-inducing events that pose a threat to central life goals

produce a much larger psychological response Personal goal

hi-erarchies provide a useful framework for interpreting the

impor-tance of a particular experience from an uncertainty-management

perspective (Austin & Vancouver, 1996; Carver & Scheier, 1998;

Peterson, 1999) The highest level of a goal hierarchy consists of

an end state, with the subordinate levels including the perceptions,

actions, and subgoals required to achieve the desired end (Powers,

1973) Lower order, behavioral “doing” goals (e.g., getting a good

job) are often enacted in support of higher order, more abstract and

conceptual “being” goals (e.g., the sense of being a productive

member of society; Carver & Scheier, 1998; Powers, 1973;

Val-lacher & Wegner, 1985) These higher order self-goals organize an

individual’s actions and perceptions across a large number of

situations and over an extended period of time

The dissolution of these more abstract self-goals has broader

implications than the loss of simple behavioral goals, so the

concomitant increase of psychological entropy is greater and more

widespread Disrupting a higher order goal means that many

behavioral and perceptual affordances previously constrained by

this goal are suddenly allowed to vary freely (see Figure 2A) Accordingly, while challenges to lower order goals may lead to relatively minor experiences of anxiety (instantiated as a slight and temporary flattening of the distribution of possible actions and interpretive frames), challenges to an individual’s higher order goals can lead to states of profound behavioral and affective destabilization (instantiated as a rapid flattening of the perceptual and behavioral probability distributions across multiple situations; Figure 2B)

Adopting a goal hierarchy perspective helps to explain why people will sometimes voluntarily enter into uncertain situations

In particular, exposing oneself to a measured degree of uncertainty

at one level of the goal hierarchy may actually help to reduce uncertainty at a higher order level An individual who is facing an identity crisis, for example, experiencing dissatisfaction at work, may leave the familiarity of his or her current job to explore alternative career possibilities In the short term, this will increase the experience of uncertainty as different career options are ex-plored To the extent that the exploratory behavior is successful, however, the individual will identify a career path that provides a clearer sense of self, thus reducing uncertainty at a higher level of the goal hierarchy and constraining perceptual and behavioral affordances across a broader range of situations

In terms of the entropy formula, exploration will initially in-crease entropy levels as the range of perceived options inin-creases

If any of the newly perceived possibilities is deemed to be more desirable than the previously recognized ones, however, it will emerge as the dominant option, and entropy levels will drop below their previous values Voluntarily confronting and exploring un-certainty in the short term can thus help to reduce unun-certainty in the long term by helping an individual to identify the optimal path

to a goal Such exploration is inherently risky, however, as

desir-Figure 2. Goals are hierarchically structured such that major goals are achieved through the pursuit of multiple minor goals (or subgoals) Major goals inform a greater number of possible behaviors and situations, with minor goals affecting a smaller subset of situations In this way, major goals influence a larger part of the experienced world compared to minor goals, serving to constrain various actions and perceptions (each possible

representation depicted as x i) across a broader array of situations (Panel A) Disruption of major goals therefore produces a more widespread increase

in psychological entropy (flattening of probability distributions pertaining

to more situations and experiences) compared to minor goals (Panel B) Conversely, the disruption of minor goals results in relatively smaller increases in psychological entropy

able outcomes are seldom guaranteed Consequently, the risks of

voluntary exposure to uncertainty have to be weighed against the

potential benefits that might emerge

Under circumstances of sufficiently severe and potentially

trau-matic uncertainty, as is likely to emerge when the highest levels of

a goal hierarchy are destabilized, the individual can no longer

clearly determine the significance of any given object, action, or

experience; all of these must be understood and constrained in

relation to a particular goal or reference point Calculating the

appropriate response without such a goal becomes extremely

dif-ficult, as the number of potential options grows exponentially and

the distribution of possible actions and perceptions extends beyond

the individual’s computational capacities It should thus be clear

that uncertainty is not merely a cognitive phenomenon reflecting a

lack of knowledge about a particular domain Rather, the EMU

framework proposes that uncertainty is an intensely affective

ex-perience, as it is directly relevant to the ability to fulfill basic

motivational needs Understanding the affective significance of

such experiences can be further assisted by examining research on

the neurophysiology of uncertainty

Neurophysiology of Uncertainty

Over the last few decades, substantial progress has been made in

describing the neurophysiological processes by which uncertainty

is detected and resolved Such descriptions aid in the elaboration of

the fourth major tenet of EMU: that uncertainty is associated with

the experience of anxiety and is linked to activation of

anxiety-related brain circuits For an organism to adapt to a complex,

ever-changing environment, it is necessary for it to possess flexible

cognitive and behavioral frameworks To maintain such flexibility,

the organism must be capable of recognizing discrepancies

be-tween its desired or expected outcomes and the outcomes that it

actually experiences Organisms that fail to recognize such

dis-crepancies will continue to use outdated models of the

environ-ment and will remain unaware of the dangers and opportunities

that lie beyond their current conceptualizations

Early researchers investigating the neurophysiology of

uncer-tainty identified a process known as the orienting reflex or

orient-ing response.This reflex serves as an anomaly detector, helping to

draw an organism’s attention to unexpected sensory events

(Pav-lov, 1927; Sechenov, 1863/1965; Soko(Pav-lov, 2002) The behavioral

expression of the orienting response involves a rapid shift of

attention (usually accompanied by head and eye turn) toward an

unexpected or novel stimulus After repeated presentations of the

same stimulus, the orienting response tends to decrease, reflecting

the process of habituation

Neurophysiologically, the orienting response appears to be

pri-marily instantiated in the septo-hippocampal comparator system,

which compares neural signals from cortical representations of the

environment (models and expectations) with incoming sensory

information (Vinogradova, 2001) Whenever there is a mismatch

between these two inputs (i.e., whenever the organism’s actions or

perceptions are not producing the expected or desired outcome),

tonic inhibition of the reticular formation by hippocampal CA3

neurons is removed As a result of this disinhibitory process,

emotional arousal is heightened via the release of noradrenaline,

and attention is rapidly focused on the anomalous occurrence As

attention is focused on the unexpected event, an updated cortical

representation develops, such that future presentations of the same stimulus or event will not produce the same orienting response If representations of the unexpected event are not updated, then that event will continue to be a source of uncertainty for the organism (along with the associated stress and attentional distraction) Building on this line of work, Jeffrey Gray published an influ-ential model of anxiety, proposing that mismatch between pre-dicted and actual sensory events is one of the inputs that can produce increased activity within the behavioral inhibition system (BIS; Gray, 1982; Gray & McNaughton, 2000) The BIS is a neural system responsible for suppressing behavior, increasing attention to novel features of the environment, and increasing levels of arousal Gray identified this system as the neural substrate

of anxiety, based on pharmacological studies of antianxiety drugs and their behavioral and neural effects In particular, Gray asso-ciated the BIS with a 7.7-Hz hippocampal theta response, driven

by activity in the septal area This theta response typically accom-panies behavioral indicators of anxiety, such as the slowing or cessation of goal-directed behavior Septal lesions, pharmacolog-ical interventions, and other techniques for blocking this theta activity all have the effect of reducing the associated behavioral inhibition

The septal activity that drives the hippocampal theta response also appears to be dependent upon signals from the dorsal ascend-ing noradrenergic bundle, which originates in the locus coeruleus

of the brainstem and which innervates the hippocampus, septum, and some cortical regions (McNaughton & Mason, 1980) Selec-tive lesions of this pathway eliminate the hippocampal theta rhythm, as well as the behavioral expressions of anxiety The release of noradrenaline in response to faulty expectations thus appears to be one of the key processes in the cascade of neural activity underlying anxiety (Tanaka, Yoshida, Emoto, & Ishii, 2000) It is of interest to note, from the perspective of the EMU framework, that activity in the hippocampal system has also been linked directly to the entropy of a visual stimulus stream; less predictable sequences result in greater hippocampal activity, as anticipated by Gray’s model (Strange, Duggins, Penny, Dolan, & Friston, 2005) It is of further interest, from the EMU perspective, that the same neural system and behavioral consequences were observed in response to uncertainty, unexpected nonreward, and cues of impending punishment (Gray, 1982)

In the second edition of Gray’s influential book, co-authored by Neil McNaughton, these multiple pathways to BIS activation were integrated within the framework of goal conflict (Gray & Mc-Naughton, 2000), such that BIS activation is most likely when an individual is faced with multiple competing perceptual and behav-ioral affordances Accordingly, even approach–approach conflicts, during which an individual is faced with the opportunity to pursue two competing rewards, can trigger BIS-related anxiety It may seem counterintuitive to think of anxiety as resulting from multiple positive opportunities However, it is important to keep in mind that EMU predicts greater uncertainty (and hence BIS-related anxiety) whenever the optimal behavioral path is obscured by multiple competing possibilities, regardless of their valence In these situations, the BIS and its concomitant anxious arousal aid in the search for an appropriate response (cf Schwartz, 2005) Gray and McNaughton (2000) also made an important distinc-tion between anxiety and fear, the latter of which is insensitive to anxiolytic drugs (cf Perkins, Kemp, & Corr, 2007) In particular,

they posited that anxiety reflects the experience of BIS-related

uncertainty about the appropriate response, while fear reflects the

expression of avoidance motivation Thus, situations characterized

by clear threat and a clear strategy for avoiding it are more likely

to elicit fear If, conversely, the situation has a clear threat but does

not elicit a clear strategy for threat avoidance (or if there is some

incentive to approach the threat), this uncertainty regarding the

appropriate behavioral response would produce BIS-related

anxi-ety This is in addition to the fear and avoidance produced by the

threat itself

The avoidance responses that characterize fear are thought to be

instantiated by a neural system distinct from the BIS, the fight–

flight–freeze system (FFFS; McNaughton & Corr, 2004) Because

fear responses are often situated within ongoing goal pursuit,

however, they can increase behavioral uncertainty, which triggers

anxiety responses in the BIS More generally, the appropriate

response in a fearful situation is often unclear In fact, the close

relation between these two systems led Gray and McNaughton

(2000) to the conclusion that the dispositional sensitivity to threat,

as reflected in the personality trait of Neuroticism, was jointly

determined by the BIS and the FFFS (cf Cunningham, Arbuckle,

Jahn, Mowrer, & Abduljalil, 2010; DeYoung, Quilty, & Peterson,

2007) Although many emotion researchers regard anxiety as a

mild form of fear (e.g., Scherer, 2001), the EMU framework

follows Gray and McNaughton’s research in conceptualizing

anx-iety as a distinct affective system associated with goal conflict and

uncertainty

More recently, the anterior cingulate cortex (ACC) has also

received attention for its involvement in error processing, conflict

monitoring, and uncertainty (Botvinick, Braver, Barch, Carter, &

Cohen, 2001; Carter et al., 1998; Critchley, Mathias, & Dolan,

2001; Holroyd & Coles, 2002; Yeung, Botvinick, & Cohen, 2004)

Activity in this brain region is also associated with anxiety and

sympathetic arousal (Critchley, Tang, Glaser, Butterworth, &

Dolan, 2005; Hajcak, McDonald, & Simons, 2003a, 2003b) and

has been conceptualized as a cortical alarm bell that indicates the

need for attentional resources to be deployed to address a cognitive

or behavioral anomaly Importantly, this holds true whether the

anomaly involves perceptual or motor conflict, performance

er-rors, or uncertainty about a goal-relevant domain

The functions of the ACC make it appear as a cortical extension

of the BIS, as it shares many features with Gray’s subcortical

network, including electrical activity centered around 7.7 Hz (Luu,

Tucker, Derryberry, Reed, & Poulsen, 2003; Pizzagalli, Oakes, &

Davidson, 2003), activation following uncertainty and reward

pre-diction errors, associations with increased levels of anxiety, and

potentiation by noradrenergic agonists (Riba, Rodriguez-Fornells,

Morte, Munte, & Barbanoj, 2005) Additionally, neural activity in

the ACC has been directly related to the orienting response in

humans, assisting with the detection of novelty and anomaly

(Dietl, Dirlich, Vogl, Lechner, & Strian, 1999; Williams et al.,

2000) Furthermore, the personality trait of Neuroticism has been

linked to ACC activity during the commission of errors, when

participants experience response conflict and uncertainty (Haas,

Omura, Constable, & Canli, 2007; Hirsh & Inzlicht, 2008; Luu,

Collins, & Tucker, 2000; Pailing & Segalowitz, 2004; Paulus,

Feinstein, Simmons, & Stein, 2004), as have dispositional

mea-sures of BIS sensitivity (Amodio, Master, Yee, & Taylor, 2008;

Boksem, Tops, Wester, Meijman, & Lorist, 2006) Accordingly,

the fourth major tenet of the EMU framework is that the BIS–ACC system serves as the neural substrate for the experience of uncer-tainty, which is associated, in part, with the subjective state of anxiety

Interestingly, uncertainty-related activity in the ACC also ap-pears to occur in response to positive feedback, but only when it is not expected (Jessup, Busemeyer, & Brown, 2010; Oliveira, Mc-Donald, & Goodman, 2007; Pfabigan, Alexopoulos, Bauer, & Sailer, 2011) This finding is consistent with Gray and Mc-Naughton’s (2000) proposal that novel and unpredicted events initially result in BIS activation due to the inherent ambiguity of such experiences Importantly, the EMU framework proposes that

it is not the valence of the unexpected event that determines the magnitude of the BIS response to novelty but rather the extent to which it results in the simultaneous activation of competing inter-pretive frameworks and response tendencies During unexpected positive events, the interpretation of the event as positive neces-sarily conflicts with the preexisting belief that no positive event will occur Although the initial uncertainty about such an event may be resolved relatively quickly, the BIS will be engaged as long as competing representations of the unexpected event are simultaneously active, preventing the adoption of a single domi-nant interpretive frame or behavioral response

While the septo-hippocampal system and ACC serve to alert an individual to any anomalous events or conflicting representations that are encountered, a cognitive-behavioral system must also have

a means of revising its perceptual and motor programs to develop

an appropriate behavioral response to the situation In this way, future prediction errors and uncertainty can be minimized, and the system’s goals can therefore be achieved with fewer unwelcome interruptions The ACC appears to facilitate this function by sub-sequently engaging the processing resources of the dorsolateral prefrontal cortex (DLPFC), which can support the selection of the appropriate perceptual state or behavioral response (Cohen, Bot-vinick, & Carter, 2000; Kerns et al., 2004; MacDonald, Cohen, Stenger, & Carter, 2000)

While the ACC has been conceptualized as an evaluative sys-tem, indicating the need for greater attentional resources, the DLPFC has been conceptualized as an executive system, able to selectively excite or inhibit activity in the rest of the brain to optimize effective goal pursuit (Carter et al., 2000) The DLPFC is involved in planning, conceptual integration, working memory, and cognitive control processes (Kane & Engle, 2002; E K Miller, 2000; E K Miller & Cohen, 2001), which are of critical value when an organism confronts complex problems that have defied all previously functional conceptual frameworks The cog-nitive resources of the DLPFC thus appear to allow for the detailed exploration of an unexpected outcome, so that it can be analyzed for its causes, motivational significance, relevant perceptual prop-erties, and implications for future behavior Similarly, engagement

of the DLPFC should help to reduce uncertainty by facilitating choices between competing interpretive frames (Yoshida & Ishii, 2006) We propose that the extent to which revised cognitive models and behavioral strategies generated during such active engagement are pragmatically adaptive, uncertainty-related activ-ity in the BIS and ACC should decrease When incoming sensory information is no longer unexpected or in conflict with goal-directed activity, there should no longer be heightened noradren-ergic innervation of the septo-hippocampal system

As described previously, the experience of uncertainty will thus

depend upon the clarity and pragmatic effectiveness of an

individ-ual’s goal (acting as a framework for organizing action and

per-ception) within a given environment In the next section, we

further examine the relationship between uncertainty and goal

conflict, at a behavioral level

Uncertainty and Goal Conflict

The EMU framework proposes that situations of uncertainty

will be minimized if an individual has a functionally adequate

mental map of the environment and knowledge of the appropriate

responses that should be made to further important goals (Peterson,

1999; Peterson & Flanders, 2002) Such an individual has

effec-tively reduced the entropy within the cognitive system, as the

distribution of possible meanings and behaviors associated with a

given event or situation will be narrowed to a single optimal

response Consequently, less metabolic energy will be wasted

during perception and goal pursuit The well-ordered and adapted

knowledge structure of such an individual allows for the efficient

execution of the behavioral acts needed to obtain a desired state in

the world (i.e., perform work)

Individuals in situations of chronic uncertainty, by contrast,

must exert much more energy to accomplish a goal, as they will

waste precious metabolic (and cognitive) resources on activities

that do not further their interests The simultaneous activation of

conflicting goals can also result in energy loss, as actions that

support movement toward one goal may actually hinder progress

toward another For example, a person may wish to attain career

success but also spend time with his or her family In such a

situation, the choice to work late to finish a project directly

conflicts with arriving home in time for the family dinner

The highest levels of entropy and metabolic waste will exist

when the goal itself is not well specified or in the case when a

previously held goal is abandoned and has not yet been replaced by

an alternative goal (Carver & Scheier, 2003) In such cases, it

becomes impossible to specify the motivational significance of any

given event, as there will be no clear reference value by which to

judge the experience No event has a predefined meaning, from an

objective perspective As Hume (1739) so famously implied

cen-turies ago, it is only the subjective relevance of an event to an

individual’s particular goals (including desires and motivations)

that defines and constrains the value of an event, for better or for

worse (Baumeister, 1991; Carver & Scheier, 1998; Ferguson &

Bargh, 2004; Frankl, 1971; Hirsh, 2010; Little, 1998; Markman &

Brendl, 2000; Peterson, 1999) According to the EMU framework,

BIS activity should be maximal during situations of complete

uncertainty, when there are no clear goal structures constraining

the interpretation of an event’s significance (or the appropriate

behavioral response that should be generated) In such a case, the

objects and situations presenting themselves to the observer will

suggest an overwhelming jumble of affordances, none of which

will be clearly superior to the others in terms of its subjective value

or likelihood of producing desired results

It is important to note the similarity between situations of

conflict and uncertainty At a computational level, conflict reflects

the simultaneous activation of competing interpretive frameworks

for a given event, with no clear dominance of any one (Berlyne,

1957) If any of the networks supporting these interpretations were

clearly more active, based on the constraints of the situation and the individual’s knowledge structure, conflict would be minimal and the optimal framework would be rapidly decided (Bishop, 2006; Rogers & McClelland, 2004; Rumelhart & McClelland, 1986) However, to the extent that substantial conflict exists, there are two or more possible frameworks for construing the same situation, with none of them clearly superior to the others As a result, conflict necessarily implies uncertainty, where the optimal response to a given event remains unspecified Gray and Mc-Naughton (2000) argued that goal conflict is one of the precipita-tors of BIS activation, reflecting indecision about how best to construe and respond to a stimulus (e.g., whether to approach or to avoid) Response conflict has also been found to reliably elicit ACC activity and subsequent engagement of the DLPFC (Botvin-ick et al., 2001; Kerns et al., 2004; Yeung et al., 2004) The resolution of such conflict involves careful examination of the situation to determine the optimal response

What this suggests is that situations with the fewest constraints can be the most anxiety producing as a consequence of their inherent uncertainty (reflecting the large number of possible inter-pretive frames and response options) When the affordances of a given situation are equipotential, meaning that no interpretive framework or behavioral response is clearly the most appropriate, there will be a parallel activation of many different perceptual and motor response options This high-entropy state should engage the BIS and produce the associated experience of anxiety Note, once again, that this is distinguishable from a situation in which a fear response is produced and a clear escape route is provided; such situations involve a clear dominance of perceptual and behavioral affordances related specifically to escaping the current situation It

is in the situations where the possible responses are truly equipo-tential (meaning that it is not clear whether or how one should remain, approach, or escape) that a BIS response should be most likely (Gray & McNaughton, 2000) The EMU framework pro-poses that the levels of anxiety experienced in such situations should be proportional to the degree of uncertainty about the appropriate action Just as the state of maximal physical entropy occurs when there is a perfect thermodynamic equilibrium of particles within a system, so too does the state of maximal psy-chological entropy occur when there is a perfect equilibrium of perceptual and behavioral affordances This occurs when one has absolutely no idea what is happening or what one should do: No candidate options reveal themselves as more appropriate than any other

High-entropy situations are also distinct from states of behav-ioral quiescence, where no overt behavior receives strong activa-tion and the individual is in a restful state Behavioral quiescence

is most likely to occur when the potential costs of action are perceived to be higher than the potential rewards, so that refraining from action is perceived as the optimal response (Anderson, 2003)

A greater tendency toward inaction is observed, for instance, amongst those who anticipate negative consequences from their actions (Tykocinski & Pittman, 1998) and those with reduced incentive motivation (Depue & Collins, 1999; Hirsh, DeYoung, & Peterson, 2009; Shankman, Klein, Tenke, & Bruder, 2007) States

of satiation can likewise foster inactivity by reducing an action’s perceived value (Schultz, 2006) During quiescent states, there is a clear perception that no active behavior is required or encouraged

by the situation, such that resting or behavioral calmness is the