Independent And Stationary Sequences Of Random Variables - Chapter 19 ppsx

The central limit theorem for homogeneous Markov chains Consider a homogeneous Markov chain with a finite number of stateslabelled 1, 2,.. For an arbitrary initial distribution weproceed

Chapter 19

EXAMPLES AND ADDENDA

The separate sections of this chapter are not related to one another except

in so far as they illustrate or extend the results of Chapter 18

© 1 The central limit theorem for homogeneous Markov chains

Consider a homogeneous Markov chain with a finite number of states(labelled 1, 2, , k) and transition matrix P = (p i ;) (see, for instance,Chapter III of [47] ) If Xn is the state of the system at time n, we have thesequence of random variables

X 1 , X 2 , , X n ,

(19 1 1)

We denote byp~~ )the probability of moving from state ito statejinnsteps

If for some s > 0, p( ;)> 0 for all i,j,then Markov's theorem [47] states thatthe limits

p j = lim p(n ) n-+ ooexist for all i andj and do not depend on i, and that, for constants C, p(0<p<1),

max l pt~!)-p

j.1 < Cpn

(19 1 2)i,,

The numbers p l , p 2 , , p,, form a stationary probability distribution inthe sense that, if P(X 1 = j) = p; for all j, then the variables X n form astationary sequence It then follows from (19 1 2) that Xn is uniformlymixing, since, if

A = {X1=i1, X2='2, , X r=ir { ,

B = {Xn+r - in+r, , Xs i s{ ,

= I'(A)Pinln+rPi„+rin+r+l Pi,- 1is ,

P (A) P (B) = P (A) p to+rpto+rin+r+1 PG-ii"

so that JP(AB)-P(A)P(B)I < P(A)IP~"n+r-Pin+rI < P(A)Cpn Let f (~) be any real function defined on the states of the chain Application

of Theorem 18 5 2 shows that the central limit theorem applies to the sequence f (Xj) whenever

62 = E{f(X1)-Ef(X1)}2 +

n +2 Z E{f(Xj+1)-Ef(Xj+1)} If(X1)-Ef(X1)} # 0 j=1

If n= (n1, it2,

nk) is any other initial distribution, we denote the

cor-responding probability and expectation by P,, and En

Theorem 19 1 1 Assume that (19 1 2) holds, and that a :~4-0 Then, for any initial distribution n,

lim Pn Qn2

°

E exp it 2 [f (Xj) - Ef (Xj)] -1 1+

J 1 it

< 2C r + 2 Its log np

Using the Kolmogorov extension theorem, we can find a sequence of random variables

X1, X2, , Xn,

(19 1 4) with values in 3`, such that

P(X1EA1, X2EA2, , XneAn)=

JA1 n(d~I) f p(~1, d~2)

f P(~n-1, din)

(19.1 5)AZ

An The n-step transition probabilities p(n) (c, A) are given by

PGV, A) = p (~, A) , p(n) (~, A) = f Pcn -1 > (q, A) P (~, dri) and

7r(A) EA) =

(n ~ 2) Under reasonably weak conditions, there exists a stationary measure

p (A), such that sup I p(") (x, A) - p (A) I < C pn ,

(19.1 7)4,A

where C, p are constants, 0 < p < 1 This is true, for instance (see [23]) if (1) there is a finite measure m on j~, with m (3) > 0, an integer v and a positive number e such that p(V) (~, A) < 1- c whenever m (A) < a (Doeblin's condition), and

(2) there is only one ergodic set

= E{f(Xl) - Ef(XJ)}2 +00

+2 E E{f(X1) - Ef(X1)}{f(Xj+1) - Ef(Xj+1)} =AO ,

j=1 then for any initial distribution 7r (A),

19 2

m-DEPENDENT SEQUENCES

3 69

special case of Theorem 18 5 2 For an arbitrary initial distribution weproceed exactly as in the proof of Theorem 19 1 1, estimating the differ-ence E n - E

In a similar way we may use Theorem 18 6.1 to prove the following result

Theorem 19 6 1 Let (19 1 7) hold, and let f=f (~1, ~2, ) be a real-valued function on the infinite product E x ¢ x , measurable with respect to the product 6-algebra tR, x a x Write

n

z P,+- 1n_I Z (fj _Efj)<z =(2it) -2~ - e - J"2 duu

if mt•_ 0 and 93 are independent when b - a > m

The latter form of the definition has an obvious extension to the case of aprocess X(t) with continuous time parameter

A simple method of constructing m-dependent sequences (see [25] forexamples occurring in statistics) is as follows Let

sta-An in-dependent sequence is trivially uniformly mixing with 0(n) =0 for

n > m Hence the following result is a special case of Theorem 18 5 2

Theorem 19 2 1 Let Xjbe a stationary m-dependent sequence with EX0 <

oo Then

00

~Z = EX0+2 1 EXOXJ

j=1 converges, and if u :AO,

n lim e 0 -l

© 3 The distribution of values of sums of the form Ef (2kx)

Letf (t) be a periodic function of the real argument t, with period 1, andconsider the distribution of the values of the sum

Such sums are of considerable importance in the metric theory of numbers,and as such have been studied by a number of authors An importantspecial case is the function

f (t) = {t}

the fractional part of t

The reason for discussing the problem here is that it is a special case ofthose discussed in © 18 6 Indeed, for 0 <t < 1,

n

n Sn(t) = E, f({ 2k t}) = E f(T k t),

k=1

k=1

where T is the mapping of [0, 1) into itself defined by

T t = {2t}

(We leave this for the reader to verify )

We now study the probability space formed by the segment [0, 1), withthe Lebesgue measurable sets, and probability measure A Then equation

(19 3 2) means that the sequence of random variables f k = f (2k t) is

stationary We shall see that much more can in fact be said Any t E [0, 1) has an expansion of the form

fk = f (2k t) = f (Ek, Ek + 1, )

is obtained from the independent random variables E jin the way discussed

in © 18.6 Theorem 18 6 1 therefore gives sufficient conditions for theasymptotic normality of Sn (t), i.e for the limit

lim A t ; S fit)SES.(t)

< z = (P (z)

n o0

1 n( )}

to hold

372

EXAMPLES AND ADDENDA

Chap 19

These conditions may be stated in a different, and more natural, form inthe present application We must first, of course, require that f have afinite variance, i e

A Jk

so that

fJ2 k [f]k(t) = 2k `

The special case of Theorem 18 6 1 can now be stated as follows

Theorem 19 3 1 Let f (t) be a function in L2(01 1) and with period 1, and let fof(t)dt=0 If

-(Y

0 then

00

2 =f 1

f (t) 2 dt + 2 Z

J 1 f (t)f (2k t) dt 0

k=1

Remark 19 3 1 The condition (19 3 3) will be satisfied if either

(1) f (t) is a function of bounded variation, or

1 (2)

J If (t)-f(t+h)12dt= 0 log-2-Eh , e>0

~ f(t+(j - 1 ) 2-k )] 2dv)

Substituting condition (2), we have

IIf-[ f]kII 2 = O(k -2-L) ,

so that00

I II f- [ f]kli < 00 k=1

© 4 Application to the metric theory of continued fractions

Each real number t in the interval (0, 1) has a unique continued fractionexpansion of the form

of computing the measures of sets of values of t defined by conditions onthe sequencea„ (t) We shall see that, if a suitable measure is placed on theinterval (0, 1), then the sequence a„ (t) can be made stationary, and that itthen satisfies the uniform mixing condition Many of the results of thetheory then follow from the known properties of mixing sequences Define then a probability space with (0, 1) as the set of elementary events,endowed with the 6-algebra of Lebesgue measurable sets The appropriateprobability measure turns out to be that defined by the equation

y (A) = (log 2)-1

prop-If t >, 0, [t] = a o we write

t = [a o ; a 1 , a 2 , ] ,

19 4

APPLICATION TO THE METRIC THEORY

or

Pk \ t \ Pk + Pk- 1 qk

Then t= [a o ; a 1 , a 2 , , r n+1 ] and rn= [an ; an+l, an+2, ] The number

rn is called the remainder of order n of the continued fraction expansion

of t From (19 4 4) it follows that, for all k in 1 <, k ,<n,

[a0 ; a1 , , an] = Pk-1rk+Pk-2 ,

qk-1rk+qk-2

and thus for all n,

t = Pn-1 rn+Pn-2 qn-1rn+qn-2

Pn qn Proof. The set A~1 ", consists of the numbers of the form

and Pn + Pn-1

qn+qn-1

as rn+1 varies on the interval [1, cc)

The fundamental result is then the following theorem

Theorem 19 4 1 With respect to the measure §, the sequence a n (t) is stationary, and satisfies the uniform mixing condition with

0(n) < Ke-~" , where K and A are absolute constants

We note that a n cannot be stationary in the wide sense, since

Ea 1 (t) _ -101

1

1 +t) dt =

19 4

APPLICATION TO THE METRIC THEORY

377

Proof (1) To prove that a n(t) is stationary we have only to show that

§(AI ; gin) = §(Ail+s °

in+')

We first show that

§(AS ",) = l-c(A2 ..

.gin + 1) )

By Lemma 19.4.1,A! , : : :n is an interval with end-pointsp n / q nand(pn+ P„-1)/ (q n + qn _ 1 ),wherep n /q n = [i 1 , i 2 , , in] For the sake of argument supposethat

1 +An

1 n l092 log 1 + An - §(A i, in

Continuing in this way,

(n+s)) = Y(A

7

(19 4 9)Finally, the general case is obtained from (19 4.9) and the equation

378

EXAMPLES AND ADDENDA

Chap 19

since any sets A, B measurable with respect to (a1 , , a,) and (a,,,,,ak+n+s) respectively may be written as disjoint unions

of sets A l , B mof the type occurring in (19 4 11), and (19 4 11) will imply that

l § (AB) - u (A) u (B) I < E l§ (A 1 B.) - u (Al)§ (Bm) I

u(B) < K e-.inIy(A)

To prove (19 4 11) we need a theorem of Kyz'min, whose proof may befound in [68]

Theorem (Kyz'min) Let (fn (x) ; n =1, 2, ) be a sequence of functions

on [0, 1] satisfying

00

(0<x<1, 101< 1 ),where

1

a = log 2 J ofo(z) dz ,) is an absolute positive constant, and K depends only on M, m

We setMn(x) = N {t ; a1 (t) = i1, , ak(t) = ak(t) = ak , Zk+n(t) < x}

where z,,(t) denotes the continued fractionZs(t) = [as+ 1(t), as+ 2 (t), ]

In order that a 1 (t)= i1 , , a k (t) = ik , Zk + n (t) < x, it is necessary and cient that, for some integer r,

suffi-A= J AI, B U BM ,

I

m

To calculate the integral of the left-hand side, note that by Lemma 19 4 1,

A + n° ° °Jk + n +s is the interval (a, /3) on which the first s coefficients of thecontinued fraction of t arejk+n, ° ° °,jk+n+s° The difference Mn

$~

Hence the integrated version of (19 4 14) is equivalent to (19 4.11)

It remains to prove the admissibility of Kyz'min's theorem, so that wehave to show that

Therefore

MO(x) _

0<X0'o(X)<c1 IXg (x)I < c 2 ,

where c l , c 2 are constants For t e Ail : : : k ,

t = Pk + Zk Pk- 1 qk+Zkgk-1

whereA

19 4

APPLICATION TO THE METRIC THEORY

Theorem 19 4 2 Let f (t) be an absolutely integrable function on (0, 1) Then for all points t of (0, 1) except possibly for the elements of a set of measure zero,

lim n-1 n

E 1

1 f(Tit) = (log 2)-1

t d n- oo

'=o

f o +

Corollary 19 4 1 Let f (r) be a function of the integral variable r, and let

f (r) = 0 (r1- a), S > 0 Then for almost all t c- (0, 1),

a0lim n-1

r=1

Many similar results can be proved in the same way

To formulate a central limit theorem for the sequence f(Tit), we have to

be able to compute expressions of the form[f]k(t) = E(f Ia l , a2, , a k)

This is constant on A l : : : k, and

f 1 k [f]k(t)§(dt) = fA1 k f(t)§(dt) ,

lI ik

t ; u- ' n-Z I { f(Tkt-a)} < z , = O(z) , n- cx

k=0

where AA denotes Lebesgue measure on (0, 1)

Proof. It follows at once from Theorems 18 6.1 and 19 4.1 that

n-1

lim § t ; a-1 n

-1-E If(Tk t)-a} < z = 0(Z) n-c%

k=0

Hence we have only to show that § can be replaced by Lebesgue measure,,

1 E' (f) =

If f (t)is a bounded function measurable with respect to 93? 1 ,then Lemma

where the BJ are disjoint sets in + 1 , and for such f, (19 4 20) gives

IE u (f) - EA(f)I < sup Ia j l I Ii(B)-)(Bj)I <

t

384

EXAMPLES AND ADDENDA

n- 00Using (19 4 20), as n-+oo, with r= [log n],

IE,,(e``zn) -E1(eirz n) I

1- exp {

any k 0 [f (Tk t) -a]

n - 1 + (E § -E 2 ) exp it

O (e -~ rl/z ) = 0(1)

Hence the theorem is proved

Remark 19 4 1 Condition (19 4 18) is satisfied if either of the conditions

of Remark 19 3 1 is satisfied.

The proof is almost the same as before, using the fact that, fort, u e A k ,

by (19 4 5) and (19 4 6),It-uI G Pk

19 5

then the distribution of Zn does not converge to the normal distribution

In view of the results of © 17 3, Y, and hence Xj , is regular Thus we have

an example of a regular stationary process not conforming to the centrallimit theorem Moreover,Xjis generated by independent random variables

in the sense of © 18 6 The reader will be able to verify without difficultythat

E{ [Xo -E(XoI ~-k, , ~k)] 2 } = 0(k'-2o) ,which shows that the condition of Theorem 18 6.1 cannot be significantlyweakened

We now go on to investigate the limiting distribution of Z n In order not

to overload the argument, the justification for some of the analyticaloperations is left to the reader

From the results of © 16 7, it follows that the sequence Y has spectraldensity

00

f (~) = Lr

k-ae`k i

2 k=1

By partial summation,00

00

385

(19 5 1)

386

EXAMPLES AND ADDENDA

Lemma 19 5 1 If (Y1 , Y2 , , Yn) is a non-degenerate Gaussian random vector, then the characteristic function ofE"=1 y2 is

n

rp(t) _ [111-2it§j { - ,

j=1

where the pj are the eigenvectors of the matrix R = (EYYk)

Proof We denote by y the vector (y 1 , Y2, , Yn), by A the matrix (aij),

with inverse A-1 and determinant {A{, and associated quadratic form

(Ay, y) = E aiiyjyj

i,jThe identity matrix is I

If A is any complex matrix such that Re A = (Re a i j ) is positive-definite,then (see for example [22])

(t) = E {exp(it E Y2 )} = E(e`t ("• Y))_

00

00(2~)-2n {R{-2

f -00

f -00eit(y,y)e-2(R-1Y,Y)dy1 dy n =

00 _ (2,r)-2n {R{-2f

Remark 19.5 1 The function {1-2it{-2 is the characteristic function of

ri g , where ri c N (0, 1) Lemma 19 5 1 therefore asserts that EY2 has thesame distribution asE§ j rl ,3, wherethep jare independent N (0, 1) variables

19 5

EXAMPLE OF A SEQUENCE

nf _" sin 2(i~) f (A) &

T(1-a)2 E sin 2 (in).)

E(U1U2U3U4) = E(U1U2)E(U3U4)+E(U1U3)E(U2U4)+

+E(Ul U4)E(U2 U3)

Hence

n

k=1and

0 < b = lim inf b(n) < lim sup b(n ) = B < ac ,

and 4'n(t) is a characteristic function If the distribution of Z n converges

to N (0, 1) as n-+oo, then we may go to infinity in (19 5 7) along a quence (ii i )onwhich bothbin' ) and0,,j (t) converge, to arrive at the equation

subse-e -2` 2 = 11-2itb ol-2cp(t),

(19 5 8)where 0 < b o < oo and 4 (t) is a characteristic function

It is however, a famous result of Cramer (see, for instance, [23] or [105]),that if

e zt2 1 (t)M'2(t)

19.5

EXAMPLE OF A SEQUENCE

where 0 , and 02 are characteristic functions, then both 01 and 02 must

be normal This contradicts (19 5 8), and shows that Z n cannot converge

in distribution to N(0, 1) In fact, more detailed analysis shows that