Rewards cause eddies in the stream of time

Cregor Fetterman b a Department of Psychology, Arizona State University, Tempe, Arizona, USA, and b Indiana University, Purdue University at Indianapolis, Indiana, USA Ac c e p te d

0 1993 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBAElse vie r Sc ie nc e Pub lishe rs B.V All rig hts re se rve d 0376-6357/ 93/ $06.00

BEPROC 00498

C ha ng ing ra te s o f re info rc e me nt p e rturb s the flo w

o f time

Laura Morgan a, Peter R Killeen a and J Cregor Fetterman b

a Department of Psychology, Arizona State University, Tempe, Arizona, USA, and

b Indiana University, Purdue University at Indianapolis, Indiana, USA

(Ac c e p te d 13 Ma y 1993)

Abstract

This experiment tested the hypothesis that changes in rate of reinforcement affect the rate of an internal pacemaker as suggested by the Behavioral Theory of Timing (BeT: Killeen and Fetterman, 1988) Pigeons were trained to discriminate durations of 10 s and

20 s, and then exposed to higher or lower rates of fre e ly d e live re d re info rc e rs Whe n

re turne d to the ’ d isc rimina tio n ta sk, jud g me nts we re re lia b ly b ia se d in the p re d ic te d

d ire c tio ns: tho se re turning fro m a ric he r c o nte xt jud g e d sta nd a rd d ura tio ns to b e lo ng e r tha n d id tho se re turning fro m a p o o re r c o nte xt The se re sults va lid a te a ke y a ssump tio n o f

Be T, a nd p ro vid e a n e xp la na tio n o f ho w c ha ng ing te mp o s o f life b ia s the p e rc e p tio n o f time

Key words: Time p e rc e p tio n; Pa c e ma ke r sp e e d ; Be ha vio ra l The o ry o f Timing (Be T);

Temporal discrimination; Reinforcement rate; Response bias; Choose-short effect; Pigeon

Introduction

Who ha s no t e xp e rie nc e d the slo w p a ssa g e o f time d uring a rura l va c a tio n, o r the fre ne tic p a c e o f a ma jo r me tro p o lis o ne is ne wly a rrive d a t? This se nse o f b e ing “ o ut o f sync ” with the c o nte xt fa d e s q uic kly, as our sense of time acclimates to the tempo of a new environment What is the origin of these perceptions and adjustments?

Correspondence to: P Killeen, Department of Psychology, ASU, Tempe, AZ 85287-l 104, USA

260 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

The o rie s o f time p e rc e p tio n usua lly invo ke a p a c e ma ke r-c o unte r mo d e l no t unlike tho se

fo und in mo st c lo c ks: a c e ntra l o sc illa to r, o r p a c e ma ke r, e mits p ulse s whic h a re a c c umu-

la te d b y so me typ e o f c o unte r (e g G ib b o n a nd C hurc h, 1990; Ro b e rts, 1981; Tre isma n, 1963) De p e nd ing o n the kind s o f e rro r a sso c ia te d with e a c h o f the c o mp o ne nts, suc h simp le syste ms c a n p ro vid e a c c ura te me a sure me nts o f time d e sp ite sub sta ntia l fluc tua tio ns

in the inte rva ls b e twe e n suc c e ssive p ulse s (Kille e n, 1992; Kille e n a nd We iss, 1987) But suc h syste ms a re una b le to d e a l with shifts in the a ve ra g e sp e e d o f the p a c e ma ke r, a nd it is

to suc h shifts tha t we a ttrib ute the tra nsie nt b ia se s o b se rve d whe n the te mp o o f the

e nviro nme nt c ha ng e s

In a typ ic a l time d isc rimina tio n e xp e rime nt the o nse t o f o ne stimulus sig na ls the sta rt o f

a n inte rva l to b e time d , a nd the o nse t o f a no the r sig na ls the e nd o f the inte rva l a nd the

o c c a sio n fo r the sub je c t to ind ic a te the d ura tio n o f the sig na l If the inte rva l w a s sho rt,

re sp o nse s to o ne switc h mig ht b e re info rc e d , whe re a s if it w a s lo ng , re sp o nse s to the o the r switc h mig ht b e re info rc e d (e g Stub b s, 1968) Wha t c o rre sp o nd s to the c o unte r a nd the

p a c e ma ke r in suc h situa tio ns ? Sinc e va rio us a d junc tive b e ha vio rs (e g p a c ing , turning ,

p e c king , e tc ) g e ne ra lly o c c ur d uring d istinc t p o rtio ns o f the inte rva l, Kille e n a nd Fe tte rma n (1988) hyp o the size d tha t the y mig ht c o nstitute the c o unte r Fig ure 1 sho w s d istrib utio ns o f

a d junc tive re sp o nse s in o ne p ig e o n w ho w a s re q uire d to d isc rimina te whe the r a n inte rva l

w a s ‘ lo ng ’ (12 s) o r ‘ sho rt’ (4 s) The b ird w a s usua lly c ro uc hing a t the e nd o f the sho rt inte rva l; sinc e re sp o nse s to the ‘ sho rt’ ke y a re the n re info rc e d , c ro uc hing p re d ic ts the

c o rre c t c ho ic e re sp o nse , a nd sho uld the re fo re b e c o me e sta b lishe d a s a c ue , o r ‘ o c c a sio n

se tte r’ (Ho lla nd , 1980) fo r the c o rre c t c ho ic e C o nve rse ly, if the p ig e o n w a s e xte nd ing its

ne c k o r p a c ing when the interval ended, those behaviors, being predictors of the correct

‘long’ response, may b e c o me c o nd itio ne d a s d isc rimina tive stimuli fo r it

The nature of the pacemaker that drives transition from one adjunctive behavior to the next is less certain Indeed, it is not necessary to have a periodic pacemaker at all: a Poisson emitter that moves the organism from one state to the next with constant probability will suffice (Killeen and Fetterman, 1993) If the period of the pacemaker (or the time constant of the Poisson emitter) is proportional to the interval being timed, then

A Crouching + Neck Extension

Time (s) Fig 1 The time course of three adjunctive responses emitted by one subject in a timing experiment

The data are from Fetterman and Killeen (1991)

261 adjunctive behaviors should accordion to fill the interval They do (e.g Killeen, 1975) If

temporal discriminations are based on them, then discriminative accuracy should be

proportional to the length of the interval being timed It is - or at least it is a linear function

of - the interval being timed (e.g Gibbon, 1977; Killeen and Weiss, 1987) This is Weber’s

Law, which, in this context, is also called scalar timing

In their behavioral theory of timing (BeT) Killeen and Fetterman (1988) assumed that

the speed of the pacemaker was proportional to the rate of reinforcement in the

experimental context Under normal circumstances this is highly - but not perfectly -

correlated with the duration of the intervals to be timed Their theory tied pacemaker

speed to reinforcement rate because reinforcers are more biologically potent events than

arbitrary stimuli By constructing BeT in this manner, many other phenomena such as

contrast effects and foraging strategies might be accommodated (Killeen, 1991) Further-

more, distributions of adjunctive behaviors such as those shown in Fig zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA1 ha ve b e e n fo und zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

to e xp a nd a nd c o ntra c t in w a ys tha t a re c o nsiste nt with shifts in the p a c e ma ke r sp e e d a s a

func tio n o f c ha ng e s in re info rc e me nt ra te , e ve n whe n the d ura tio n o f the inte rva l in whic h

the y o c c ur is he ld c o nsta nt (Ha ig ht a nd Kille e n, 1991)

If ra te o f re info rc e me nt d rive s the sp e e d o f the p a c e ma ke r, the n c ha ng ing the ra te o f

re info rc e me nt while le a ving the inte rva l to b e d isc rimina te d inta c t sho uld b ia s te mp o ra l

e stima te s In p a rtic ula r, if we ta ke ‘ c ity p ig e o ns’ o ut to the c o untry fo r a we e k - re d uc e

the ir ra te s o f re info rc e me nt in the e xp e rime nta l c ha mb e r - the ir p a c e ma ke rs sho uld slo w

d o w n: the d istrib utio ns in Fig 1 sho uld shift to the rig ht Whe n we e ve ntua lly re turn the m

to the ‘ c ity’ a nd te st the ir d isc rimina tio ns, the y sho uld b e b ia se d to se e time p a ssing mo re

q uic kly tha n b e fo re the va c a tio n: b ia se d to c a ll lo ng inte rva ls ‘ sho rt’ (b e c a use the b e ha vio rs

suc h a s c ro uc hing tha t me d ia te the ‘ sho rt’ re sp o nse ha ve no w e xp a nd e d into the time tha t

ha d p re vio usly b e e n o c c up ie d b y ne c k e xte nsio n) C o nve rse ly, if we ta ke c ity b ird s a nd

e xp o se the m to a ma jo r me tro p o lis, the y sho uld re turn ho me to find a c ity tha t ha s se e me d

to slo w d o w n Ho w e ve r we b ia s the a nima ls, we e xp e c t the m to e ve ntua lly a d a p t: up o n

re turn to the b a se line re info rc e me nt c o nte xt the a d junc tive b e ha vio rs sho uld mig ra te b a c k

to the ir ‘ p ro p e r p la c e s’ a nd te mp o ra l p e rc e p tio n will re turn to the sta tus q uo a nte

If re info rc e me nt ra te within the te sting c o nte xt itse lf is shifte d , the a d junc tive b e ha vio rs

will re ma in ske we d fro m b a se line The b e ha vio ra l c ue s tha t re ma in misle a d ing must

b e c o me re c o nd itio ne d to the a lte rna tive re sp o nse b e fo re a c c ura c y c a n re turn to b a se line

le ve ls Thus, if in Fig 1 a hig he r ra te o f re info rc e me nt in the te sting c o nte xt shifte d the

d istrib utio ns to the le ft fro m the ir b a se line p o sitio n, p a c ing will b e c o me a mo re re lia b le c ue

fo r the ‘ lo ng ’ re sp o nse tha n ne c k e xte nsio n, whic h, b e ing e q uivo c a l, ma y e xting uish a s a

d isc rimina tive stimulus We c a ll suc h re c o nd itio ning o f the c ue s tha t c o nstitute the c o unte r

‘ re c a lib ra tio n’ Both adjustment of the pacemaker speed and compensatory recalibration of

the counter to maintain accuracy are continual processes; the rate at which they will occur

is not predicted by the present theory, but ha s b e e n a d d re sse d e lse whe re (Fe tte rma n a nd

Kille e n, 1991; Kille e n, 1984) In p ra c tic e , the a d justme nt o f the p a c e ma ke r b e g ins

imme d ia te ly a nd is virtua lly c o mp le te within 100 re info rc e me nts Re c a lib ra tio n o f the

c o unte r d e p e nd s o n the c o nting e nc ie s tha t sha p e a c c ura te re sp o nd ing a nd thus the

se le c tio n o f d iffe re nt c o unts a s c rite ria 1 c ue s, a nd o n the a va ila b ility a nd p re vio us histo ry o f

re info rc e me nt o f tho se p o te ntia l c ue s In p ra c tic e the re c a lib ra tio n ma y la g o nly a c o up le o f

re info rc e me nts b e hind the a d justing p a c e ma ke r - o r re c a lib ra tio n ma y ne ve r o c c ur,

le a ving d isc rimina tio ns sho w ing a p e rma ne nt hyste re sis e ffe c t

The p re se nt e xp e rime nt p ro vid e d a te st o f the se id e a s To e xa mine the p re d ic te d shifts

in time p e rc e p tio n we c o nd itio ne d a te mp o ra l d isc rimina tio n, the n shifte d the a nima ls to a

2 6 2 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

hig he r (o r lo we r) ra te o f fre e re info rc e me nt, a nd the n re turne d the a nima ls to the

d isc rimina tio n ta sk We me a sure d b ia s e a rly in the shift p e rio d a nd withhe ld re info rc e me nt

d uring te sting to a sse ss p e rfo rma nc e d uring a nd a fte r re a d justme nt to the te sting c o nte xt

Fina lly, we inc re a se d ra te o f re info rc e me nt within the e xp e rime nta l c o nte xt itse lf zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

Ma te ria ls a nd Me tho d s

Subjects

Eig ht ma le p ig e o ns (C o lumb a livia ) with p re vio us histo rie s o f e xp e rime nta tio n we re

ma inta ine d a t 80-85% o f the ir fre e -fe e d ing we ig hts

Apparatus

The e xp e rime nt w a s c o nd uc te d in thre e id e ntic a l Le hig h Va lle y p ig e o n b o xe s ha ving a

c ha mb e r o f 31 x 36 x 37 c m fo r the sub je c ts The re we re thre e G e rb ra nd s re sp o nse ke ys

o n the fro nt wa ll 24 c m a b o ve the flo o r The c e nte r ke y c o uld b e illumina te d with a re d

lig ht to sig na l the o nse t o f the inte rva l to b e time d The sid e ke ys c o uld b e illumina te d with

white lig ht to sig na l a c ho ic e o p p o rtunity, with re sp o nse s to the le ft ke y sig nifying ‘ sho rt’ ,

a nd re sp o nse s to the rig ht sig nifying ‘ lo ng ’ The re info rc e r (2.3 s o f a c c e ss to mixe d g ra in)

c o uld b e o b ta ine d thro ug h a c e ntra lly lo c a te d a p e rture , situa te d 11 cm above the flo o r

White no ise a t 40 d B w a s c o ntinuo usly p re se nt At the ve ry to p o f the fro nt wa ll a sma ll

b ulb p ro vid e d g e ne ra l c ha mb e r illumina tio n

Procedure

Pretraining

Tria ls c o nsiste d o f illumina tio n o f the c e nte r ke y fo r e ithe r 10 s (the sho rt stimulus) o r 20

s (the lo ng stimulus) fo llo we d b y illumina tio n o f the c ho ic e ke ys A re sp o nse o n the le ft ke y

w a s c o rre c t a fte r the 10 s stimulus a nd a re sp o nse o n the rig ht ke y w a s c o rre c t a fte r the 20

s stimulus Fo r the first IO se ssio ns c o rre c t re sp o nse s we re a lwa ys fo llo we d b y fo o d ;

inc o rre c t re sp o nse s we re fo llo we d b y a 3 s b la c ko ut o f a ll lig hts A c o rre c tio n p ro c e d ure

w a s in e ffe c t d uring this p a rt o f p re tra ining , with inc o rre c t re sp o nse s fo llo we d b y a

re p e titio n o f the tria l The re a fte r no c o rre c tio n p ro c e d ure w a s use d Afte r 10 a d d itio na l

se ssio ns the p ro b a b ility o f re info rc e me nt fo r a c o rre c t re sp o nse w a s re d uc e d to 0.5 Sho rt

a nd lo ng tria ls o c c urre d q ua si-ra nd o mly with e q ua l fre q ue nc y d uring e a c h g -tria l b lo c k

Re sp o nse s o n unre info rc e d tria ls we re fo llo we d b y a 3 s b la c ko ut Se ssio ns e nd e d a fte r 60

re info rc e me nts, a nd p re tra ining a fte r a to ta l o f 40 se ssio ns

Baseline 7

The p ro b a b ility o f re info rc e me nt for correct responses was decreased to 0.25 The short

(IO s) and long (20 s) durations were presented on 75% of the trials, with the trials equally

divided between the two training durations A 14-s probe duration was presented on the

remaining (25%) trials The probe was used to evaluate temporal discrimination near the

263 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

p o int o f sub je c tive e q ua lity whe re we e xp e c te d g re a te st se nsitivity to shifts in b ia s C ho ic e

re sp o nse s o n the p ro b e tria ls we re ne ve r re info rc e d , b ut we re a lwa ys fo llo we d b y a 3 s

b la c ko ut, a s we re inc o rre c t c ho ic e re sp o nse s a nd c o rre c t c ho ic e re sp o nse s o n tria ls no t

sc he d ule d fo r re info rc e me nt The me a n inte rfo o d inte rva l d uring b a se line te sting w a s

b e twe e n 60 s a nd 70 s, d e p e nd ing o n a c c ura c y Se ssio ns e nd e d a fte r 60 re info rc e me nts,

a nd the b a se line c o nd itio n a fte r a to ta l o f 20 se ssio ns

Initial adjustment of the pacemaker

The ma jo r ma nip ula tio n c o nsiste d o f fe e d ing the a nima ls a t sho rte r o r lo ng e r inte rfo o d

inte rva ls tha n the y e xp e rie nc e d d uring the b a se line c o nd itio n Ha lf o f the b ird s re c e ive d

fre e re info rc e rs a t a hig he r ra te tha n tha t e xp e rie nc e d d uring b a se line (fa st b ia s); the

re ma ining b ird s re c e ive d the fre e re info rc e rs a t a lo we r ra te (slo w b ia s) Tria ls b e g a n with

the illumina tio n o f the re d c e nte r ke y lig ht Und e r the fa st b ia s c o nd itio n, the ke y lig ht

re ma ine d o n fo r 10 s a nd with a p ro b a b ility o f 0.5 w a s fo llo we d b y 2.3 s a c c e ss to g ra in,

a nd o the rwise b y a 3 s b la c ko ut The p ro c e d ure w a s simila r und e r the slo w b ia s c o nd itio n

e xc e p t tha t the ke y lig ht sig na l re ma ine d o n fo r 60 s a nd with a p ro b a b ility o f 0.5 w a s

fo llo we d b y 2.3 s a c c e ss to g ra in, a nd o the rwise b y a 3 s b la c ko ut The c ho ic e ke y lig hts

we re no t illumina te d , a nd re sp o nse s we re no t re q uire d fo r re info rc e me nt (te c hnic a lly,

the re fo re , the se a re Fixe d -Time sc he d ule s) zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBASe ssio ns e nd e d a fte r 60 re info rc e me nts, a nd

this c o nd itio n a fte r 8 se ssio ns

First test

Afte r the e ig ht se ssio ns o f fre e ly-d e live re d re info rc e rs, sub je c ts we re re turne d to the

te mp o ra l d isc rimina tio n ta sk, with ha lf the tria ls la sting 14 s, a nd the o the r ha lf la sting

e ithe r 10 s o r 20 s No re info rc e me nt w a s g ive n o n a ny tria l, i.e the te sting w a s d o ne in

e xtinc tio n The te sting p ro c e d ure w a s use d to me a sure the e ffe c ts o f p rio r e xp o sure to the

hig h a nd lo w re info rc e me nt ra te c o nte xts Se ssio ns e nd e d a fte r 112 tria ls, a nd this

c o nd itio n a fte r 2 se ssio ns

Fast testing context (1)

Whe n the first te st w a s c o mp le te d , c ho ic e re sp o nse s we re a g a in e lig ib le fo r re info rc e -

me nt O ne se ssio n w a s g ive n imme d ia te ly a fte r the te st p ha se d uring whic h a ll c o rre c t

re sp o nse s to the sho rt (IO s> a nd lo ng (20 s) stimuli we re re info rc e d As b e fo re , re sp o nse s

o n p ro b e (14 s) tria ls we re no t re info rc e d This p ro c e d ure e le va te d the ra te o f re info rc e -

me nt in the c o nte xt fro m ze ro (e xtinc tio n) to a p p ro xima te ly 3 re info rc e rs p e r minute ,

d e p e nd ing o n a c c ura c y The ma nip ula tio n p ro vid e d a no the r a sse ssme nt o f the e ffe c ts o f

c ha ng e s in re info rc e me nt o n p ig e o ns’ timing b e ha vio r, b ut o ne in whic h the c ha ng e in the

d e nsity o f fo o d o c c urre d within the te sting c o nte xt Se ssio ns e nd e d a fte r 60 re info rc e rs,

a nd this c o nd itio n a fte r 1 se ssio n

Baseline 2

Afte r the te st with e le va te d ra te s o f re info rc e me nt, a ll sub je c ts we re re turne d to the

b a se line c o nd itio n fo r 25 se ssio ns, 60 re info rc e rs p e r se ssio n

Readjustment of the pacemaker

The n the b ia s ma nip ula tio ns we re re ve rse d Sub je c ts tha t initia lly e xp e rie nc e d the hig h

ra te o f fre e -re info rc e me nt (fa st b ia s) we re e xp o se d to the lo w ra te (slo w b ia s), a nd vic e

ve rsa As b e fo re , this p ha se la ste d fo r e ig ht se ssio ns, e a c h c o nsisting o f 60 re info rc e me nts

Second test zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

Afte r the e ig ht se ssio ns o f fre e ly-d e live re d re info rc e rs, sub je c ts we re re turne d to the

te mp o ra l d isc rimina tio n ta sk, with ha lf the tria ls la sting 14 s, a nd the o the r ha lf la sting

e ithe r zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA10 s o r 20 s No re info rc e me nt w a s g ive n o n a ny tria l, i.e the te sting w a s d o ne in

e xtinc tio n Se ssio ns e nd e d a fte r 112 tria ls, a nd this c o nd itio n a fte r 2 se ssio ns

Fast testing context (2)

Whe n the se c o nd te st w a s c o mp le te d , c ho ic e re sp o nse s we re a g a in e lig ib le fo r re in-

fo rc e me nt: a ll c o rre c t re sp o nse s to the sho rt (IO s) a nd lo ng (20 s) stimuli we re re info rc e d

As b e fo re , re sp o nse s o n p ro b e (14 s) tria ls we re no t re info rc e d Se ssio ns e nd e d a fte r 60

re info rc e rs, a nd this c o nd itio n a fte r 1 se ssio n

In a ll c a se s, the p rima ry me a sure o f p e rfo rma nc e c o nsiste d o f the p ro b a b ility o f ma king

a ‘ lo ng ’ c ho ic e re sp o nse This p ro b a b ility w a s c a lc ula te d se p a ra te ly o n sho rt (IO s), lo ng (20 s), a nd p ro b e (14 s) tria ls This me a sure w a s use d b e c a use it p e rfo rms a s well as other measures of bias suc h a s / 3, a nd is e a sy to inte rp re t (Ma c Milla n a nd C re e lma n, 1991)

Whe re a s re sp o nse s to the p ro b e stimuli sho uld sho w the g re a te st shifts, re sp o nse s to the tra ining stimuli sho uld a lso b e a ffe c te d b y o ur b ia sing ma nip ula tio ns, a nd so the re sp o nse

p ro b a b ilitie s ma y a lso b e a ve ra g e d o ve r a ll thre e typ e s o f tria l

Results

Fig ure 2 sho w s d a ta fro m the fina l 15 b a se line se ssio ns p o o le d a c ro ss sub je c ts a nd a c ro ss the first a nd se c o nd b a se line c o nd itio ns The fig ure d isp la ys the p ro b a b ility o f re sp o nd ing

‘ lo ng ’ a fte r the IO s (c irc le s), 14 s (sq ua re s), a nd 20 s (tria ng le s) stimuli; the to p c e ntra l line sho w s the me a n p ro b a b ility o ve r a ll typ e s o f tria ls The ve rtic a l ma rks ind ic a te sta nd a rd

e rro rs a c ro ss sub je c ts The a ve ra g e p ro b a b ility o f b e ing c o rre c t whe n p re se nte d with o ne o f the tra ining stimuli w a s 89%, c o rre sp o nd ing to a d’ o f 1.25, a nd a We b e r Fra c tio n (c o e ffic ie nt o f va ria tio n) o f 0.27 This is c o mp a ra b le to va lue s fo und in o the r stud ie s (e g Stub b s, 1979), a ltho ug h it is so me wha t la rg e r tha n tha t o b ta ina b le fo r p ig e o ns und e r the

b e st o f c irc umsta nc e s (0.22; Fe tte rma n a nd Kille e n, 1992) The p re se nc e o f unre info rc e d

p ro b e tria ls o f inte rme d ia te d ura tio n ma y b e o ne re a so n fo r the le ss tha n p e rfe c t

d isc rimina tio n The se d a ta sho w tha t d isc rimina tio n w a s: (1) c o ntro lle d b y the d ura tio n o f the c e nte r ke y stimulus, a nd (2) ve ry sta b le a c ro ss ind ivid ua ls a nd b a se line te st se ssio ns Fig ure 3 d isp la ys the p ro p o rtio n o f ‘ lo ng ’ re sp o nse s to the 14 s p ro b e stimulus in 8

p ro b e -tria l b lo c ks (i.e e a c h d a ta p o int summa rize s re sp o nse s to 8 p re se nta tio ns o f the

p ro b e stimulus) fo r the two c o nse c utive te sting se ssio ns The d a ta a re a ve ra g e d o ve r sub je c ts a nd o ve r the o rig ina l (first te st) a nd re ve rse d (se c o nd te st) c o nd itio ns, a ll o f whic h sho w e d the sa me e ffe c ts whe n a na lyse d se p a ra te ly So me a nima ls sto p p e d re sp o nd ing

d uring the la st b lo c k o f tria ls, a nd so the fig ure p o rtra ys o nly the d a ta fro m the first 13 tria l

b lo c ks

We se e tha t the p ro p o rtio n o f ‘ lo ng ’ c ho ic e s w a s hig he r a fte r sub je c ts e xp e rie nc e d the hig h ra te o f re info rc e me nt (fa st b ia s), tha n a fte r the y e xp e rie nc e d the lo w ra te o f

re info rc e me nt (slo w b ia s): in the first fe w b lo c ks o f tria ls the sta nd a rd e rro r b a rs fo r

p ro p o rtio n o f ‘ lo ng ’ re sp o nse s d o no t o ve rla p This is e xa c tly the re sult p re d ic te d und e r o ur

a ssump tio n tha t ra te o f re info rc e me nt a ffe c ts the sp e e d o f the p a c e ma ke r The inte rc e p ts o f the re g re ssio n line s te ll us ho w muc h the w a rp ing ma nip ula tio ns c ha ng e d the b ia s b e fo re the e xtinc tio n c o nd itio ns ha d a c ha nc e to w a sh o ut the e ffe c t Fo r the fa st b ia s c o nd itio n,

_

1

012345676 9 10 11 12 13 14 15

DAYS zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Fig 2 Ave ra g e p ro b a b ility o f re sp o nd ing ‘ lo ng ’ d uring b a se line Pro b a b ilitie s fro m the first a nd

se c o nd b a se line s w e re p o o le d C irc le s, sq ua re s, a nd tria ng le s re p re se nt p ro b a b ility m e a sure s o n sho rt,

p ro b e , a nd lo ng d ura tio n tria ls, re sp e c tive ly The line w itho ut symb o ls re p re se nts the a ve ra g e o ve r a ll

tria ls The ve rtic a l line s ind ic a te sta nd a rd e rro rs o f the m e a n a c ro ss e ig ht p ig e o ns

the intercept was 0.55, a nd for the slow bias condition it wa s 0.27 The 95% confidence

intervals for these intercepts do not overlap (the lower limit for the fast bias is 0.46,

whe re a s the up p e r lim it for the slow bias is 0.39) This provides strong confirmation of the

shift predicted by BeT, the primary claim tested in this experiment

BeT further predicts that under the extinction in the testing context the rate of the

pacemaker will slow, leading to a ‘choose short’ bias, and the distributions of adjunctive

behaviors will flatten, leading to poorer discrimination This also happened The slope for

the fast bias condition is -0.018, which is significantly less than zero: t(1 I) = 3.522,

1 l Fast Bias y = - 0.018x + 0.55 0.8 0 Slow Bias y = + 0.007x + 0 27

Blocks of 8 Probe Trials Fig 3 Pro b a b ility o f re sp o nd ing ‘ lo ng ’ to the p ro b e (14 s) stim ulus d uring te sting , a ve ra g e d o ve r

266 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

P < 0.05 The slo p e fo r slo w b ia s, 0.007, d o e s no t d iffe r sig nific a ntly fro m ze ro : t(ll> =

1 012, P > 0.05 The o ve ra ll p ro b a b ility o f sa ying ‘ lo ng ’ d e c re a se d fro m tha t fo und und e r

b a se line (0.47) to 0.36, c o nsiste nt with a d e c re a se in p a c e ma ke r sp e e d d ue to the a b se nc e

o f re info rc e me nt in this c o nd itio n

No te tha t Fig 3 d isp la ys the d a ta fro m two se ssio ns, with so me d isc o ntinuity visib le

b e twe e n the la st b lo c k o f the first d a y o f te sting (7) a nd the first b lo c k o f the se c o nd d a y o f

te sting (8) If we a na lyse the se d a ys se p a ra te ly, fo r the first d a y we find a hig he r inte rc e p t

for the fast bias condition (0.641, a nd a g re a te r d e c re a se in the p ro b a b ility o f a ‘ lo ng ’

c ho ic e thro ug h the c o urse o f the se ssio n The slo w b ia s c o nd itio n inte rc e p ts the o rd ina te a t

0.24, a nd sho w s no c ha ng e thro ug h the se ssio n O n the se c o nd d a y the a ve ra g e p ro b a b ility

o f a ‘ lo ng ’ re sp o nse re ma ins re la tive ly c o nsta nt fo r b o th g ro up s a t a me a n va lue o f 0.36,

sig nific a ntly lower than found under baseline (t(5) = 5.25, 2.92 for fast and slow; both

P’s < ,051

Afte r the te st se ssio ns, a ll sub je c ts we re e xp o se d to a ‘ fa st te sting c o nte xt’ , a n e le va te d

ra te o f re info rc e me nt (a ll c o rre c t re sp o nse s to sho rt a nd lo ng stimuli re info rc e d , yie ld ing

a p p ro xima te ly 3 re info rc e rs p e r minute ) Exc e p t fo r the hig h d e nsity o f re w a rd , c o nd itio ns

we re o the rwise the sa me a s to tho se d uring te st se ssio ns (i.e the mixture o f p ro b e a nd

tra ining tria ls w a s the sa me ) Fig ure 4 sho w s d a ta fro m the fa st te sting c o nte xt (I) p lo tte d

a c ro ss b lo c ks o f 16 tria ls Fo r e c o no my o f p re se nta tio n, the se d a ta a re b a se d o n re sp o nse s

to a ll stimulus typ e s (sho rt, p ro b e , a nd lo ng d ura tio ns) Diffe re nt symb o ls sho w whe the r a

sub je c t ha d b e e n in the fa st b ia s o r slo w b ia s c o nd itio ns p rio r to the te st The d a she d line

ind ic a te s the a ve ra g e p ro b a b ility o f re sp o nd ing ‘ lo ng ’ fo r a ll stimuli fo r the fina l se ssio n o f

the first b a se line c o nd itio n; the ve rtic a l b a rs g ive s the sta nd a rd e rro r o ve r the e ntire se ssio n,

a nd fo r e a c h b lo c k

‘0

1 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

08

1

0.6

n Slow Bias

0 Fa st Bias

- Average

00

BLOCKS OF 16 RESPONSES Fig 4 Probability of responding ‘long’ across blocks of 16 responses averaged over subjects and

stimuli during the first exposure to the fast testing context Filled squares indicate subjects who had

previously received the slow bias treatment and unfilled circles indicate subjects who had previously

received the fast bias treatment The dashed line indicates probability for the last session of baseline

267 zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Whe n shifte d to the e le va te d ra te s o f re info rc e me nt in the fa st te sting c o nte xt (I), a ll

sub je c ts sho w e d the p re d ic te d inc re a se in the a ve ra g e p ro p o rtio n o f ‘ lo ng ’ re sp o nse s (fro m

0.47 to 0.65); this inc re a se w a s sig nific a nt, t(7) = 5.02, P < 0.05 The fig ure sho w s tha t the

b ia s inc re a se d o ve r the first ha lf o f the se ssio n (p re suma b ly a s the p a c e ma ke r c a me up to

sp e e d ) a nd d e c re a se d d uring the se c o nd ha lf (p re suma b ly a s the b e ha vio ra l c ue s we re

re c a lib ra te d ) Up o n se c o nd e xp o sure to this c o nd itio n in the fa st te sting c o nte xt (21, whic h

is no t sho w n in the fig ure , the inc re a se in the p ro p o rtio n o f ‘ lo ng ’ re sp o nse s (to 0.51) w a s

no t sig nific a nt It is p o ssib le tha t the p rio r histo ry o f re c a lib ra tio n p e rmitte d a d justme nt to

p ro c e e d mo re q uic kly und e r this se c o nd e xp o sure to the fa st te sting c o nte xt

Discussion

We ha ve d e mo nstra te d tha t shifts in the ra te o f re info rc e me nt, b o th b e fo re a nd d uring

te sting , re lia b ly b ia s time e stima te s in the d ire c tio n p re d ic te d b y Be T The se shifts a re

c o nsiste nt with the e ffe c ts o f e le c tric a l b ra in stimula tio n (Me e k, 19881, a nd with d rug

a d ministra tio ns whic h e ithe r e xc ite o r se d a te sub je c ts (Me e k, 1983) The y a re a lso

c o nsiste nt with p he no me no lo g ic a l o b se rva tio ns ma d e e ve ry d a y b y unto ld numb e rs o f

huma n o b se rve rs

The c ha ng e s in b ia s w a she d o ut in the c o urse o f o ne o r two se ssio ns This w a s a s

e xp e c te d We c ho se to run the te sts in e xtinc tio n to a vo id the p o ssib ility o f re c a lib ra ting the

b e ha vio ra l c ue s b y p a iring ne w (shifte d ) a d junc tive b e ha vio rs with the c o rre c t c ho ic e

re sp o nse s But we a lso re c o g nize tha t the sp e e d o f the p a c e ma ke r va rie s with the ra te o f

re info rc e me nt in the e xp e rime nta l c o nte xt, a nd tha t it a p p ro a c he s p ro p o rtio na lity with it a s

a line a r a ve ra g e o f its o ld me a n va lue a nd the c urre nt va lue In the te sting c o nte xt, the ra te

o f re info rc e me nt is ze ro , so tha t we e xp e c t a c o ntinua l d e c re a se in the sub je c tive ra te o f

re info rc e me nt to wa rd ze ro As it d e c re a se s, the me a n a nd sta nd a rd d e via tio n o f the

me d ia ting b e ha vio rs will inc re a se p ro p o rtio na te ly, c a using the c urve s in Fig 1 to mo ve to

the rig ht a nd fla tte n As the b e ha vio rs tha t me d ia te the ‘ lo ng ’ re sp o nse mo ve o ut o f ra ng e ,

the p ro b a b ility o f c ho o sing ‘ lo ng ’ will d e c re a se to wa rd ze ro To sho w ho w this w o rks, we

ma y se t up a simp le mo d e l in whic h the me a n o f the d istrib utio n o f the a d junc tive

b e ha vio r tha t me d ia te s the ‘ lo ng ’ re sp o nse (~~1 inc re a se s line a rly fro m its va lue b e fo re

e xtinc tio n b e g ins, kb , a s a func tio n o f the time in e xtinc tio n (t):

The sta nd a rd d e via tio n o f the se d istrib utio ns is g ive n b y We b e r’ s La w , with a c o nsta nt o f

p ro p o rtio na lity (‘ We b e r c o nsta nt’ ) o f w :

Fro m the se two e q ua tio ns we ma y fo rm a Z-sc o re :

C umula tive no rma l d istrib utio ns p ro vid e a g o o d fit to the d istrib utio ns o f te rmina l

re sp o nse s sho w n in Fig 1, a s the y d o to a d junc tive b e ha vio r in g e ne ra l (Fe tte rma n a nd

Kille e n, 1991) We ma y the re fo re e va lua te the Z-sc o re a s a no rma l d e via te In the p re se nt

0 L., zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

Blocks of 8 Probe Trials Fig 5 The data are zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBAthe sa me a s tho se sho w n in Fig 3 The smooth lines are derived from the

assumption that, because the speed of the pacemaker varies with the rate of reinforcement in the

experimental context, its period must lengthen during extinction This causes the distribution of

adjunctive behavior that serves as the criteria1 count for the ‘long’ response to shift to the right and

flatten, according to Equations 1-3

c a se , we ha ve a d d itio na l info rma tio n a b o ut the p a ra me te rs: x is the time a t whic h the

p ro b e d isc rimina tio ns a re me a sure d , whic h is a lwa ys 14 s The We b e r fra c tio n fo r the

b a se line c o nd itio ns is w = 0.27 The ind e p e nd e nt va ria b le t is p ro p o rtio na l to the block

number, and fo r simp lic ity we use the b lo c k numb e r a s the p re d ic to r This le a ve s us two

p a ra me te rs to e va lua te : the inte rc e p t a nd slo p e o f Eq ua tio n 1 Fig ure 4 sho w s tha t if we se t zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

k = 0.8, p r, fo r the fa st b ia s c o nd itio n to 11 s, a nd p c Lb fo r the slo w b ia s c o nd itio n to 19 s,

the p re d ic te d shifts in b ia s (c urve s) p ro vid e a g o o d summa ry o f the d a ta

A mo re e la b o ra te mo d e l w o uld inc re me nt the p e rio d o f the p a c e ma ke r a s a n e xp o ne n-

tia lly-we ig hte d mo ving a ve ra g e o f the e la p sing time witho ut re info rc e me nt, a nd ma ke a n

e xp lic it c o mmitme nt to the p o ints a t whic h up d a ting o c c urre d , a nd tre a t the c urre nc y

c o e ffic ie nt se p a ra te ly fro m the sc a le fa c to r fo r time But it w o uld b e unlike ly to d o b e tte r

with the se d a ta , a nd o ur ma jo r p o int - re c o nc ilia tio n o f the ‘ c ho o se sho rt’ b ia s with time in

e xtinc tio n, a s p re d ic te d b y Be T - is se rve d a s we ll b y this simp le mo d e l

No te tha t the infe rre d sta rting va lue s fo r the me a ns o f the me d ia ting d istrib utio n, p b ,

we re shifte d in the d ire c tio ns p re d ic te d b y the the o ry: to sho rte r va lue s fo r the fa st b ia s

c o nd itio n, a nd lo ng e r va lue s fo r the slo w b ia s c o nd itio n The shift w a s no t p ro p o rtio na l to

the shifte d in ra te s o f re info rc e me nt und e r the b ia sing c o nd itio ns Altho ug h Be T a ssume s

p ro p o rtio na lity, e xp e rime nta l ma nip ula tio ns suc h a s the se mo re o fte n yie ld a line a r ra the r

tha n p ro p o rtio na l c ha ng e (Bizo a nd White , 1993; Fe tte rma n a nd Kille e n, 1991) The re a re

se ve ra l re a so ns why this ma y b e the c a se Ea rly in the shift the old values fo r the p a c e ma ke r

ma y no t ha ve w a she d o ut A mo re c o mmo n re a so n ma y b e the c o nd itio ning o f the

e xp e rime nta l c o nte xt Entry into the e xp e rime nta l c ha mb e r ha s been associated with a

base rate of reinforcement for scores of se ssio ns Altho ug h the re ma y b e sub sta ntia l shift in

the sp e e d o f the p a c e ma ke r within the c o urse o f a se ssio n, we e xp e c t so me sp o nta ne o us

re c o ve ry o f its o ld va lue a t the b e g inning o f the ne xt se ssio n It will ta ke multip le se ssio ns

fo r this c o nd itio ning o f the c o nte xt to re a c h a symp to te Fina lly, the re ma y b e struc tura l

limits to the sp e e d o f the p a c e ma ke r If e a c h p ulse ta ke s a finite a mo unt o f time , the n a t

hig h ra te s o f re info rc e me nt so me p ulse s will b e b lo c ke d b y the e missio n o f o the rs The

a p p ro p ria te ‘ b lo c ke d -c o unte r’ mo d e l p re d ic ts tha t p a c e ma ke r sp e e d sho uld inc re a se a s a

hyp e rb o lic func tio n o f ra te o f re info rc e me nt tha t a p p ro a c he s line a rity o nly a t lo w ra te s o f

re info rc e me nt (Kille e n, 1993)