Independent And Stationary Sequences Of Random Variables - Chapter 18 pdf

We shall therefore study processes satisfyingthe strong or uniform mixing conditions, and functionals of such processes .There is one other sort of trivial behaviour which must be exclud

Chapter 18

THE CENTRAL LIMIT THEOREM FOR STATIONARYPROCESSES

© 1 Statement of the problem

This chapter contains the main objective of the second part of the book,the investigation of the limiting behaviour of the distributions of sums orintegrals of the form

a

as T-* co, where X, is a stationary process

If no assumptions except stationarity are made, it is not generallypossible to prove anything stronger than an ergodic theorem Thus forinstance we may take Xt = X for all t, A T = O, B T = T, and obtain anydistribution as the limiting distribution of (18 1 1) However, in this exam-ple, there is strong dependence between Xt , and Xtz even for very largevalues ofIt,- t2l This shows that, to obtain theorems of interest, we mustimpose conditions of weak dependence between the past andfuture (R) of the process We shall therefore study processes satisfyingthe strong or uniform mixing conditions, and functionals of such processes There is one other sort of trivial behaviour which must be excluded, whicharises when the sums E "Xtdo not grow as Tincreases Suppose for exam-ple that (0 is a sequence of independent, identically distributed randomvariables ; then

Xt = fit+ 1 ~tdefines a stationary process which is, in any reasonable sense, weaklydependent But

T

Y, Xt= T+1 - 1 t=1

(18 1 1)

so that (18 1 1) converges in distribution in a trivial way, taking B T =1

To exclude such behaviour, we always require thatlim B T = xx

T With

or-these restrictions, it is possible to find all the possible limit tions of ( 18.1 1

distribu-Theorem 18 1 1 Let F n (x) be the distribution function of

Before proving the theorem, we make a general remark about the methods

of proof of this and other limit theorems for dependent variables used inthis book They are all based on a very fruitful idea introduced intoprobability theory by Bernstein [8] We represent the sum

S n = X1+X2+ +Xn

in the form

k

k

Sn = Lr ~j+ Z Ylk, j=0

s=(j+1)p+jq+1

Any two random variables c ;, ~ j (i #j) are separated by at least one able q j containing q terms Ifq is sufficiently large, the mixing condition

will ensure that the ~j are almost independent, and the study of E ~j may

be related to the well understood case of sums of independent randomvariables If, however, q is small compared with p, the sum E q j will besmall compared with S, Thus Bernstein's method permits us to reducethe dependent case to the independent case

Proof of theorem 18 1 1 From (18 1 3) we conclude, as in © 2 1, thatlim Bn + 1/B n = 1

We can also choose a sequence r(n) increasing so slowly that

r

B n 1 Y X j 0

j=1

in probability as n * oo Consider the sum

n+r ((a iBn )_1 E Xj -C n - (a1Bn)-1 E Xj

By virtue of the strong mixing condition (17 2 2), the distribution function

of the left-hand side of (18 1 5) differs from

Fn (alx+bl) * Fm a l Bn x+b2

B M

by at most o(1) as r oc Because of the choice of r, the right-hand sidehas the limiting distribution F (ax + b), where a> 0 and b are constants Consequently,

F(a 1 x+b 1 ) * F(a 2 x+b 2 ) = F(ax+b),

and F(x) is stable

The variables cj have the same distribution, so that

H E(etit4;) = {E (e

l`1')}k j

Let r (n) * oo so slowly that the limiting distribution of the sum

coincides with that of the sum

18 1

STATEMENT OF THE PROBLEM

a

Q

= (b-a)EIX(0)I < co ,and since X (t, w) is measurable in(t, co), Fubini's theorem shows that

Ja b

X (t, co) dtexists for almost all w c Q, and that

E J

b X(t)dt = Ja EX(t)dt

b

a

Theorem 18 1 2 Let FT (x) be the distribution function of

320

CENTRAL LIMIT THEOREM FOR STATIONARY PROCESSES

Chap 18

Proof The stability of the limiting distribution is proved exactly as in thediscrete case To prove (18 1 10) it suffices to prove that, for all 'r>0,lim BT,/BT = T'/a .

Thus (18 1 11) is proved for integral T

If r= p/q is rational, then with T'= T / q , we have

Jim (BTP/9/BT) = lim

(

BT

P BT / = pa/qa T-oo

T,-oo BT, BT'qThus (18 1 11) is proved for rational 'r Now let T be any positive number, and choose 'r' so that (T + , C)is rational From what has already been proved,

BT r+t'

BT t+r' (

18 2

T

The inequalities (18 1 14) and (18 1 15) complete the proof

Conditions are still not known for the convergence of the normed sums

of a stationary process to a given stable law with exponent a< 2 For theremainder of the chapter we shall therefore consider only the conver-gence to a normal distribution of

T

© 2 The variance of X 1 + + X„

Consider the stationary sequence

., X_ 1 , Xo, Xl , X2 with autocovariance function R (n) and spectral function F (,~) (and, asremarked, E Xj = 0) If there is a spectral density, it will be denoted by

f ()) We write

Sn =X 1 +X2+ .+X,,

322

CENTRAL LIMIT THEOREM FOR STATIONARY PROCESSES

/ =

(n - III)eijz = n + 2 Re n E eij l - 2 Re

nEeijA

Now letf (2) exist and be continuous at 2 = 0 Integrating (18 2 4)we have

f

7rsin2(2

[f(A) -f (0)] dA

_ sin (2A

18 2

THE VARIANCE OF X, + +X„

n -1 / 4 2(1max If (A)-f( 0)I

_n- 1/4 sing

2n), ) (2 k)

n -+ o0

then either lira V (S n) = 00 n-i w

or lira sup V (Sn) < 00 , n~a0

and the latter possibility holds if and only if

lim(S mj , ~) = lim E(S mj ~) _ -( Y, ) _ -E(Y~) But then

lim (US m.,1) _ - ( UY, c) ,

323

(18 2 6)

n- oo

then for all ~ cL (X) , (UY- Y, ~) = E(UY- Y, ) = E(X1 ) , E(UY-Y-X1)~=0

Taking, in particular, = UY - Y - X1 , we have with probability 1,

Hence the theorem is proved

Remark The theorem shows that, if R(n)=-+0, then V(Sn) necessarilyconverges to a, possibly infinite, limit

Corollary 18 2 1 Under the conditions of Theorem 18 2 2,

- n

Rlim V (S,,) = 2E (Yo) = 2 f

and the argument just given gives (18 2.7)

Let (Xn) be a regular stationary sequence We have proved that, for any

Y e H,,, (X), the stationary sequence Yn = U" Y has a spectral density, andthus by the Riemann-Lebesgue lemma,

Theorem 18 2 3 If a stationary sequence Xn is uniformly mixing, and iflim V (S n) = oo ,

where h(n) is a slowly varying function of the integral variable n Moreover,

h (n) has an extension to the whole real line which is slowly varying

The theorem therefore asserts that V (S,,) is either bounded or almostlinear

Proof. This is quite long, and will therefore be divided into several parts

326 CENTRAL LIMIT THEOREM FOR STATIONARY PROCESSES Chap 18

(I) Writing 0(n)= V (S,,), we have first to prove that, for every integer k,

lim (kn) = k n-~ ao ~ l j1 )

We write

n

~j = I X(j- 1)n+ (j- 1)r+s , S=1

r 1j = I Xjn+(j-1)r+s S=1

and

(nk) = V (Snk)

k

_ E E~ +2 E E~ j ~ j+ Y~ E~ j g i+ E Egj gi j=1 i # j i,j i,j

so that 0 (n) is of the form (18 2 9), where h (n)is slowly varying

(II) We now list the properties of h (n) which admit its extension to aslowly varying function of a continuous variable

Lemma 18 2 1 For fixed k,

lim h(n+k)/h(n) = 1

(18 2 16) n- o0

Proof. Since /(n)-+co as n-+oo, the stationarity gives

~ Xj +E I Xj +2E j Xi E Xj = J=1

so that

h(n+k) _ n Ilr(n+k) _

(1+0(1))= 1+0(1) h(n)

lim n - Eh(n) = 0

(18 2 17) n-+ o0

Proof. Sincelim h (2n) / h (n) = 1 , n- 00

and using (18 2 16), we havelog h(n) _

log {h([2-jn])/h([2-j-1 n])} = o(log n) j

328

CENTRAL LIMIT THEOREM FOR STATIONARY PROCESSES

n

n+m

101 < 2[10(m) 0(n) 0(r-n)}'+ {/ (n) (m)} I+

-{t/i(r-n)~(m)}2]+~(m)Since

2{1/i(n)i(r-n)}2 <, l//(n)+l/i(r-n)=nh(n)+(r-n)h(r-n),

we have, for large n, h(r+m)=01h(n)+02h(r-n)+O(n-*),where 0 1 > 32, 02 > 0 Consequently, for large n,

01

h(n) < 2 ,

h(r) < 4 (r+m)

()Lemma 18 2 4 For all sufficiently small c and all sufficiently large n,

h (cn) h(n) < c

(18 2 19)(Of course, (18 2 19) only holds if cn is an integer )

Proof. From what has been proved about h (n),

18 2

THE VARIANCE OF X,+ +X

329

We remark that (18 2.19) holds for all c < c o , where co does not depend

on n (III) Using Theorem 18.2 1, we now extend the functions i/r (n), h (n) tothe interval (0, oo) by the equations

Px )

so that i i1(a)=02(a) = a for rational a It thus suffices to prove that 01

and /2 are continuous But

j 0((a + E) x) -0 (ax)j / i/r (x) _

lx

sinlti

(axe)f(,~) d2

n

where C > 0 and E(u)-+0 (u -+ oo)

© 3 The variance of the integral fo X(t) dt

In this section we extend the results of © 2 to the continuous time case,setting

lim R (t) = 0,

t- ao

then eitherlim V {S(T)} = (Do T-

332

CENTRAL LIMIT THEOREM FOR STATIONARY PROCESSES

Chap 18

orlim sup V {S(T)} < oo ,

c L (X), lira (S (T.), ) _ (Y, )

Tn

J i X (t) dt , )

0

and since R (t) is continuous,

tlimEli-1 ~ X(t)dt-X(0) _

-r-0

2 -2 dF(A,)

T- c0

- 00

Theorem 18 3 3 If thestationary process X (t) satisfies the uniform mixing

condition, and iflim V {S(T)} = co ,

T- oo

then

V {S(T)} = Th(T) ,

where h(T) is slowly varying at infinity

© 4 The central limit theorem for strongly mixing sequences

Let (Xj ) be a stationary sequence with E (Xj) = 0, E (Xj ) < oc, and set

n+m

Sn = I Xj , 6n = V (Sn)

j=m

334

CENTRAL LIMIT THEOREM FOR STATIONARY PROCESSES

It is of course sufficient to prove this for the sequence S n = S,1

,-Theorem 18 4 1 Let the sequence Xj satisfy the strong mixing conditionwith mixing coefficient a(n) In order that the sequence satisfy the centrallimit theorem it is necessary that

(1) Qn= nh (n), where h (x) is a slowly varying function of the continuousvariable x>0,

(2) for any pair of sequences p = p (n), q = q (n) such that

(a) p->oo, q->co, q=o(p), p=o(n) as n->oo,(b) limn'-aq'+ap-2 =0 for all f3>0,

func-Proof We first establish the necessity of (1) From Theorem 18 1 1 itfollows that h (n) is slowly varying in its integral argument Let the distri-bution function

Iz~EN

so that

18 4

STRONGLY MIXING SEQUENCES

From the remark at the end of © 18 2, we have only to show that, for each

s > 0, there exists p = p (s) such that

IE(~q)I < sE(~ 2 )

(18 4 4)Using the arguments of Theorem 17 2 2 it is easy to show that

IZI > ae(p) - lea

The strong mixing condition shows that, by suitable choice of p, we canmake IE (~ri) Ismaller than SU for sufficiently largen Thus we have provedthe necessity of the condition (1), which will henceforth be assumed The remaining parts of the proof are more complicated, but proceed inoutline as follows We represent the sum S n in the form

k-1

k

Sn = E, ~i + E, qi = Sn + Sn ,

i=0

We first have to verify that the conditions imposed on pandq can indeed

be satisfied ; otherwise the theorem would be somewhat trivial To dothis we set

To show that Z' is negligible, we need the following lemma

Lemma 18.4 1 If the distribution function F„(x) of the random variable ~nconverges weakly as n > oo to the distribution function F(x), and if ri n con- verges to zero in probability, i e.

lim P(IgnI > E)-+O

n-oo

for all e > 0, then the distribution function ofCn=~n + q n converges weakly

to Proof. Letf (t) be the characteristic function of F (x), so thatlim E(ei"In) =f(t)

18 4

STRONGLY MIXING SEQUENCES

337

Thuslim sup IEeit(~n+nn~_f(t)I

for any positive E

To continue the proof along the lines suggested, we show thatlim E IZn12 = 0 ,

n - + oo

which, since

P(IZ'I >E) < E -2 EIZn1 2 ,shows that Zn-+O in probability We haveEIZnl 2 = Qn 2

()

() {k(q/n)4}'{k(q'ln)4}2 0 ,

(18 4 10)and

"z")

-On (t) kI -+ 0

IEe"zn

-4n(t)ki < 16ka (q)which tends to zero by (18 4 1), and proves (18 4 12)

Now consider a collection of independent random variables

cn j

(n=1, 2, ; j=1, 2, , k=k(n)) ,where ~nj has the same distribution as on-1 ~ 0 Then (18 4 12) asserts thatthe limiting distribution of Z ;, isthe same as that of

bnl+bn2+ +~nk(which has characteristic function O n (t)k ) The results of © 1 7 show that

16a (q) ,

z 2 dP (c ;,j< z) _ n-oo j=1 IzI>E

= lim k

z 2dP(a 1 ~o < z)n-oo

J IzI>E

Butk

Iz! > E 1 n n

^, 2

z 2 dFp(z) ,

an P fIZI > EOn

and the theorem is proved

We remark that the only part of the proof in which (18 4 1(b)) was usedwas in the proof that E IZn1 2 = 0

The theorem simplifies if we assume that V(Sn) is asymptotically linear, as

it will be, for instance, if the spectral density f (A) exists and is continuous

at A=O, with f(O)O0

Theorem 18 4 2 If Xj is strongly mixing, and V (Sn)=a2n(1+o(1)) as n-+ co (a > 0), then Xjsatisfies the central limit theorem if and only if

lim lim sup

IzI>N

where Fn(z) is the distribution function of the normalised sum

n Zn=an 1

IxISN

as n -+ oo, since k-*oo, p- +oo

© 5 Sufficient conditions for the central limit theorem

In this section we investigate some conditions on the moments of Xj ,and on the mixing coefficients a (n), 0 (n),which guarantee thatXjsatisfiesthe central limit theorem

Theorem 18 5 1 Let the uniformly mixing sequence Xj satisfy E jXjI2+s < oo for some b > 0 If

an = E (Xl + X2 + + Xn ) 2 -> 00

as n-> oo, then X3 satisfies the central limit theorem

Proof. We show that all the conditions of Theorem 18 4.1 are satisfied

2+a

< 2+a aan

Proof. We denote constants by c 1 , c 2 , , and write

18 5

SUFFICIENT CONDITIONS

< {2+80(k) l i(2+a) }an+4an+s

To prove (18 5 1) it suffices to take k so large that 80(k) 11(2+a) <

Using the theorem again, but with p = 2 + 6,

EISn) ISn11+b < 20(k)li(2+a)an+ElSnl EISnIl+a (18 5 4)

By Lyapunov's inequality (© 1 4),

E ISnI < Un ,

EISnIl+S' 6,n+a (18 5 5)

as n + oo If we choose N so large that, for n >, N,

(1 +E')(2+E 1 ) < 2+E 2 ,

cl (1+E') < 2c 1 ,

then (18 5 6) holds forn> N, with c'2 = 2c 1 in place of c2 But we can choose

c2 so that (18 5 6) holds also for n< N, and so (18 5 6) is proved Because of (18 5 6), for any integer r,

r a2r < (2+E) r a l +C2 E (2+E)j-1 a2rS

IfE 3 and e are chosen so small that

C6 2h r + a 1 21 ++b) h

< c7 U2r

(18 5 7) (2 )

2+b

r U2r

2+b C7 Un

Y, -J

=

j=0 Un r

is bounded, which is true since thejth term is bounded by c 8 p i for some

p 1 < 1 Thus the lemma is proved

It is now not difficult to complete the proof of the theorem We have toprove that

lim n2 ~

z 2 dFP(z) = 0 n- ° D P6n „ I > Ea

By Lemma 18 5 1,

r

2+b 2+b < C7

(

E U2r-,

_ j=0

as n-* cc, by (18 4 1(a) and Theorem 18 2 3

If the mixing coefficient 0(n) is required to decrease reasonably fast, wecan remove the moment condition imposed on the Xj

Theorem 18 5 2 Let the stationary sequence Xj satisfy the uniform mixing condition, and let the mixing coefficient 0 (n) satisfy

{4)(n)}2 < oo

(18 5 8) Then the sum

~2 = E(X 2 )+2 > E(Xo Xj)

j=1 converges, and if a =A 0, as n-* oc,

n

P

Xj < z -° (27r) 6n j=1

Proof By Theorem 17 2 3,

IR(1) I=IE(XoXj)I < 24)(n)-'{E(X0)E(X2)}2whence the convergence of (18 5 9) follows As in (18 2 1),

n

2

6n = E

Xj = (j=1

(18 5 10)

18 5

SUFFICIENT CONDITIONS

X j =Z;,+Z„, j=1

where

Zn =

n

7 [fN (Xj) - EfN (X j) ] j=1

(N) a(N)

n

L, [fN ( Xj) -E fN (Xj) ]

Z „ = j -1 n

fN (x) = x - fN (x)

an t