Independent And Stationary Sequences Of Random Variables - Chapter 4 ppt

A local limit theorem is an asymptotic expression for Pn k as n--*00.. If the distribution of the XXbelongs to the domain of attraction of a stable law with density g x, the natural way

Chapter 4

LOCAL LIMIT THEOREMS

§ 1 Formulation of the problem

Suppose that the independent, identically distributed random variables X1 , X2 , have a lattice distribution with interval h, so that the sum

Zn= X1 + X2 + + X„takes values in the arithmetic progression {na + kh ;

k = 0, ± 1, } The distribution of Z„ is completely determined by the numbers

P„ (k) = P {Zn = na + kh}

A local limit theorem is an asymptotic expression for Pn (k) as n *00

If the distribution of the XXbelongs to the domain of attraction of a stable law with density g (x), the natural way to obtain an asymptotic expres-sion is to associate with the stable law a discrete distribution on the lattice {khn}, where h„ = h/Bn, and the Bn are the usual normalising con-stants, assigning to khn the probability

(k+-)h„

Pn(k) =

_

g (x) dx - hn g(khn)

(k

)hn

The theorems of § 2 give conditions which ensure that

Pn(k) Pn(k) Another sort of local limit theorem arises when the distribution of the

Xj, belonging to the domain of attraction of a stable law with density

g (x), has a density p (x) The problem then is to give asymptotic expres-sions for the density p n(x) of the normalised sum

Zn = (XI + X2 + + Xn -A ) 1B ,

dr.d in particular to give conditions under which pn(x) converges (in some sea se) to g (x) These problems are examined in § 3

4 2.

LOCAL LIMIT THEOREMS FOR LATTICE DISTRIBUTIONS

1 2 1

The first local limit theorem to emerge was that of de Moivre and Laplace

In the last fifteen years local limit problems have been studied by many authors, notably Gnedenko, whose work on the subject was motivated

by the work of Khinchin [74] on the analytic foundations of statistical mechanics

§ 2 Local limit theorems for lattice distributions Let the independent random variables

X 1 , X2 , , Xn ,

(4.2 1) have the same distribution, concentrated on the arithmetic progression {a+kh}, and write

Z n =X1 +X2+ .+Xn ,

P (Zn = an + kh) = Pn (k)

Theorem 4.2 1 In order that, for some choice of constants A n , Bn ,

lim sup IB n Pn (k) - g an+

B

-A n

n-~ oc k

n

where g (x) is the density of some stable distribution G with exponent a (0<a<,2), it is necessary and sufficient that

(1) the common distribution function F of the X X should belong to the domain of attraction of G, and

(2) the interval h be maximal

Proof The transformation X,'= (X;-a)/h

permits us to confine attention, as we shall, to the case a = 0, h = 1

(1) Necessity If h =1 is not maximal, there is some integer b > 1 such that Pn (k) = 0 unless b divides k Since this clearly contradicts (4 2 2), the necessity of (2) is proved Moreover, (4.2 2) implies that

= 0 ,

(4 2 2)

LOCAL LIMIT THEOREMS

Chap 4

F n (x) = P

CZ"B"An < x -> G (x)

as n + oo, so that (1) is also necessary

(ii) Sufficiency Choose A n , B n so that Fn + G The characteristic func-tion of Zn is given by

{f (t) }" = I e itkPn(k)

k where f is the characteristic function of the X;, and therefore

n

Pn ( k )

= 2 -n

e -itk {f(t)} n dt=

27LB n J -RBn

where

Z = Znk = ( k - An) / Bn

nB„

e-izt-itAn/Bnf(t/Bn

) n dt l

If v is the characteristic function of the stable distribution G, then g(z) = 2- J

-e-i~t v(t)dt

(4 2 4)

00

From (4.2 3) and (4 2 4), for any k,

IBn

P n (k)-g kB-)A" I < h+I 2 +I 3 +I4 ,

(4 2 5)

n

where

A

h = I-A Ie-ItAnIBnf(t/Bn)"-v(t)I dt,

I2 = J

I f(t/Bn)I "dt , ASItISEB n

f

I3 =

If(t/Bn)Indt , EBn ItI 7CBn

I4=

JItI>A Iv(t)Idt,

and A and E are constants, to be determined

We turn now to the estimation of the integrals Ij

(4 2 3)

4 2.

LOCAL LIMIT THEOREMS FOR LATTICE DISTRIBUTIONS

1 2 3

(1) Condition (1) implies that, uniformly in [-A, A], the integrand in

Ii converges to zero as n * oo Hence lim Il = 0

n-oo

(2) We remark that, for any 6 <a, there is a positive number c (6) (not depending on n) such that in some neighbourhood of t = 0 (also indepen-dent of n),

Ifn(t)I < e-(a*la

(4.2 7)

To prove this, use the results of § 2 6 to show that f satisfies If(t)I =exp{ - cltl"h(It1 -1)},

where c > 0 and h is a slowly varying function with

lim nBn "h (B,,) = 1 n~co

By Karamata's theorem (Appendix 1) there exists a function e(u)-•0 (u-+co) such that, as n-*co,

h (Bn/Itl)

=exp

h (Bn)

If therefore n is sufficiently large, Ifn( t)I = If( t/Bn)I"=

= I exp

-

h (Bn )Itl" h(BItl) I

< exp { -c (6)Itl b}

"

Bn

(Bn) for some c (6) > 0

Consequently, for sufficiently large n, e > 0 can be chosen so that

I2 <

, sE

-B„/ tl

du (1+o(1)=

B„

U

exp { -c(-!a)ItI 2"}dt <

exp { - c (4a) Itl 2"} dt-*0f2

tI A

as A *oo

JAItI Bfl

(4 2 6)

124

LOCAL LIMIT THEOREMS

Chap 4

(3) Because h = 1 is the maximal interval, the results of § 1 4 show that there is a positive constant c such that, for E Iti < n,

i f (t) i < e

(4 2 9)

Since B n = o (en`) (Theorem 2 1 1), we have

I3 =

I f (t/Bn) I n dt -<- 27re-n` Bn-~ 0

(4 2 10)

fEB n -<It I-< rzBn

as n oo

(4) Finally, since v (t) is integrable on (- oo, oo ), we have

lira

1 4 = 0

(4 2.11)

A - ao

Thus we have proved that each

iican be made arbitrarily small, and (4.2.2) follows

Theorem 4 2 2 Let the conditions of Theorem 4 2 1 be fulfilled Then, with the same choice of normalising constants An , B n ,

Jim Z JP(k) n

-B gCan+ B

-An

= 0

(4 2 12) n

n

Proof Denote by Gn(x) the distribution on the lattice {(an + kh -A n )/B n } obtained by grouping the distribution G in the manner described in § 1 Denote by F n (x) the distribution function of

(X1+X2+ +Xn-An)/Bn Then (4 2 12) asserts that the variation distance pl (F n , Gn) tends to zero

as n * oo

To prove this, restrict attention as before to the case h = 1, a = 0 By Theorem 4 2 1,

A(n) =,J = sup BnPn(k) - g k -An -+ 0

k

n

as n-* cc, and consequently

~., 1/2

1 P,(k)-Bn 1g((k-An)/B,)I < 2A - 1 = o(1)

~k - A n I<B n A - / z

4 3

A LIMIT THEOREM FOR DENSITIES

125

From the analytic properties of the density g (x) established in Chapter 2, there exists a constant C such that

lg(x)-g(y)I < clx-yl/(1+Ixla+1) when Ixl < lyl Consequently

_

A 1 /2

Bn 1g ((k -An)lBn

J

g (x) dx

Ik - AnIS B n A- 1 2

- A -1 /z

and since

Ik - AnI , <B„A -1 /2 (k-An J- )IBn

= 0 (Bn 1) ,

I

Ik - AnI -<BnA - 1 /2 Similarly,

E

Ik - AnI > BnA - 1 /2

J (k

A n

+

J)IBn

g(x)dx = 1+0(A-f") ,

g (x) g

k -An An )jdx+0(B')=n n

Bn 1g((k-An)/Bn)= 1+0(A-Ia+B„ 1 )

(4 2 14)

Bn 1g((k-An)/Bn) =0(,A-"+B„ 1 ) (4 2 15)

Finally, since the probabilities Pn (k) sum to unity, it follows from (4 2.13) and (4 2 14) that

P,, (k) = 0(Bn 1 +A-f°+A ) = o(1)

(4.2 16)

Ik-A n I>Bn A -1 /2

Combining (4.2 13), (4.2.15) and (4 2.16) proves (4.2.12)

§ 3 A limit theorem for densities

In this section we shall assume that the common distribution of the X ; has the property that, for some value N ofn, the random variable

Zn = (X 1+X2+ +Xn- An) / Bn

(4 3 1) has a density p,, (x) This clearly implies the existence of pn(x) for all n 3 N

126

LOCAL LIMIT THEOREMS

Chap 4

Theorem 4 3 1 In order that for some choice of the constants A n , B n , lim sup Ipn (x) - g (-x) I -+ 0 ,

(4 3 2)

n - oo x where g (x) is the density of some stable distribution with exponent a (0 <

a < 2), it is necessary and sufficient that the following conditions be fulfilled (1) the common distribution function F(x) of the Xj should belong to the domain of attraction of the stable law, and

(2) there exists N with sup p N (x) < oo

Proof. Condition (2) is clearly necessary, and implies that the densities

p n (x) (n > N)are uniformly bounded To see that (1) is necessary, note that (4 3 2) implies that, for x > y,

I {Fn (x) - Fn (y) } - { G (x) - G (y) } I < J

x

I pn (z) - g (z) I dz

Y

Sup IN (Z) - g (z)I (x -y) - 0

Z

as n + oo, from which it follows easily that lim F n (x) = G (x)

n- 00

Assume therefore that (1) and (2) are satisfied, and choose appropriate constantsAn, Bn Because of(2) the densityp N (x),and thus also its Fourier transform, is square integrable, and

f

co

I fN(t)1 2 dt = ~~ IpN(x)12 dx .

It follows that

fn (t) = e-itAn/Bnf( t/Bn)n

is integrable for all n >, 2N, whence

f

co -00

e - `tXf

n (t)dt = 1

00

e-itx-itAn/Bn

f(t/Bn)n dt

27r f-00

4 3

A LIMIT THEOREM FOR DENSITIES

127

Denoting by v (t) the characteristic function of G, we have

Rn = sup IPn(X) - g (x) I =

x

f

0 1

-itx{ fn(t)-v(t)}dt , 1

J

I fn (t)-v(t)I dt,

= sup 127r

-00

e

27r

2~

{I 1 +12+1 3 +14} ,

where

A

I1

= J _ Ifn( t) -v (t) I dt, A

I2= J

ItI

Iv(t)Idt,

~A

13 -

Ifn(t)I dt ,

fA5ItIsEB„

14=

J

Ifn(t)Idt,

ItI ? cB n

and A and s are positive numbers to be determined Condition (1) implies that f,-*v uniformly in every bounded interval,

lim I1 = 0

n -ao

Since v is integrable,

(4 3 3)

(4 3 4)

lim T2 = 0 A- x

The estimate (4.2 7) shows that, for sufficiently small E, there exists c o > 0 such that

13 =

I fn (t) dt

A<ItIsEB„

,

e-c o ld'/2 dt,

e -c0 It l lY`dt >0

(4 3 5)

A

f

Sltl <EB„

fItl ,A

as A >oo Since FN has a density,

sup I f (t) I = Cc < 1 ,

(4 3 6) ItIiE

and sincefn is integrable for n >, 2N,

LOCAL LIMIT THEOREMS

Chap 4

14

- I

I fn( t )l dt = ItI~eB„

fItl%tB„ If(t/Bn)Indt,<Bne-(n-2N) J- If

as n-* oc Thus each of the integrals ii can be made arbitrarily small, and

so therefore can Rn

Remark 1 It is not difficult to give examples of densities p(x) for Xj ,

for which each p n (x) is unbounded Such is the case for example [48] when

a density belonging to the domain of attraction of the normal law

Remark 2 Condition (2) of the theorem will be satisfied if for some N, the density P N (x) belongs to L P for some p > 1 Indeed, if 1 <, p <2 and

f

00

PN (x)IP dx < oc , then Titchmarsh's inequality (Appendix 2) shows that

00

00

1 fN ( t ) I P'(P - 1) dt <- (27r)(P- 2)/(P - 1)

f J

- 00- 00 showing that fn is integrable, and pn therefore bounded, whenever n>Np/(p-1)

§ 4 Limit theorem in the L 1 metric

In the last section the discrepancy between pn (x) and its limit g (x) was measured by the uniform metric

sup IPn(x) - g (x)I

X

However, pn (x) is determined only up to a set of measure zero, and it is therefore more natural to use the L 1 metric

(t)1 2N dt ,0

IPN(x)IPdx 1/(P-1)

(4 3 7)

> e-1 ,

P (x) _

{2 Ixl log log Ixl} -1 Ixl < e -1 ,

4 4

LIMIT THEOREMS IN THE L, METRIC

129 Go

HPn -g I I l= _ IPn(x) - g (x) I dx ,

or more generally the L P metric

11P - gMI P ={ ~ : IPn(x) - g(x)IP dxc l/P for 1 p < co

It turns out to be unnatural to restrict attention to absolutely continuous distributions F, and we shall accordingly describe the derivative F(x) as the density p (x), without presupposing that

f

x F(x) =

F(y) dy Each distribution function F may be represented in the form

F (x) = a R (x) + b S (x) ,

(4 4.1) where a, b >, 0, a + b =1, R (x) is an absolutely continuous distribution function

R (x) = J x p (z) dz ,

and S (x) is a singular distribution function (corresponding to a distribu-tion concentrated on a set of zero Lebesgue measure) with

S' (x) = 0 for almost all x Then the density of x is

p (x) = F' (x) = ap (x) Now let X1, X2 , be a sequence of independent random variables with distribution F, and denote as before by Fn(x) the distribution function of the normalised sum

Z,, = (X1 + X2 + + Xn - An)/ B,, Then Fn has a similar decomposition

Fn (x) = anRn (x) + b nSn(x)

(4.4 2) into absolutely continuous and singular components, and p„ (x) = F;, (x) _

an R;, (x) will denote the corresponding density

LOCAL LIMIT THEOREMS

Chap 4

Theorem 4.4 1 Letg(x) be the density of a stable law G In order that, as n-+00,

00 IIPn-gII1

-S_ 00

IPn(x) - g(x)Idx >0

it is necessary and sufficient that (1) F belongs to the domain of attraction of G, and (2) for some N, a N > 0

Proof. From (4 4 3) it follows that

a n = ~- a nR„ (x) dx =

00

<

(4 4.3)

00

_

p n (x)dx -+ J _

00

g (x) dx = 1

(4 4 4)

00

as n-+oo, whence (2) is certainly necessary Moreover, 4 4 3) and (4.4 4) imply that, for each x,

x

I F n (x) - G (x)I

~_ 00

I Pn ( x ) -g (x)Idx + bn Sn (x) IIPn glll+( 1-an) -+ 0,

so that (1) is also necessary Conversely, suppose that (1) and (2) are satisfied To prove (4.4 3) we re-quire a number of lemmas

Lemma 4 4 1 For any a > 0, b ,> 0, a + b =1, /3 > 0,

(fl) ambn-m = o(n-6)

(4.4 5)

m-na<-n'/2logn m

Proof. Let ~ 1 , b2, be independent and identically distributed random variables taking only the values 0 and 1, with respective probabilities b and a Bernstein's inequality (cf § 7 5) shows that

(

n) ()a m bn-m =

m-na<-n'/2logn m

=PR1+~2+ +~n-na < nl- log n} _ P{c1+~2+ +fin-na < -2(nab)f- log n}

< e-(log n)2 = o(n -°) (4 4 6)

4 4

LIMIT THEOREMS IN THE L, METRIC

131

Lemma 4 4 2 If N is the integer referred to in the statement of Theorem

4 4 1, then FN can be written in the form

FN (x) = a H,(x) + b H2 (x) , where a>0, b,> 0, a + b =1, H1 and H2 are distribution functions, and H 1

is absolutely continuous with bounded derivative ess sup Hi (x) < co

Proof. Choose positive numbers k and K so that {x ; k < p N (x) < K} has positive measure, and define u (x)to be equal to PN (x) on this set and zero elsewhere Determine a and H, by

a =

u (x)dx >0 ,

H,(x) = a - 1

~

X

u (x) dx •

( 4 4 7)

We now proceed to the proof of the theorem Any integer n > N can be written n = mN + r, where m and r are integers and 0 < r < N By Lemma 4.4 2,

F,,(x) = F*" (Bn x + A n B n) =

{aHl

Bnx+AnBn-ANBN

+ bH Bnx+AnBn-ANBNI *m * BN

2

B N

M

m ajbm-jH*j B n x+A"Bn - A NBN (J)

1

B j=0

N

*HZ (m_j)

(Bnx+AnBn-ANBN *

F *r (Bn x+A n Bn ) _

BN

*

* F *r (Bn X +An Bn ) =

1 +

m ajbm-j Hlj*

j-ma

m l/2logm

j-ma> -m'/2logm

I

E'1n(x )+ Y- 2n( x ) , say

* H *(m-j) * F *r1 =

LOCAL LIMIT THEOREMS

By Lemma 4 4 1,

Y-2n(°o) =Var 1 2n = Var (Fn-Z1n) <

< j-ma < -I

m/2 Iogm

(7) ajbm-j=o(n-1)

Chap 4

(4 4 8)

By virtue of Lemma 4 4 2, the distribution

Hij* H2 (m-j) *Fr

is absolutely continuous, with bounded density pmj(x) (cf § 1 2) If

h1(t), h2(t) andf (t) are the characteristic functions of H1, H2 and F, then

by Parseval's theorem and Lemma 4 4 2,

oo

- 00

_ co

1h, (t)1 2 dt = ~ 00 IHi(x)12dx <

K J 00

Hl(x) dx < oo

(4 4 9)

Go

Therefore for all j > 2 the function

hi h2 'f

is absolutely integrable, and

rn(x) = 1In(x)-9(x)=

(m) ajbm-j

pmj(x) -9(x)=

j-ma ml/2logm

J

f

°°

m

j ajbm_jh1 t BN X

21 1 - oo

j - ma _< m/2 log m

J

Bn

B m j

(t)T

x h2

(tBN f B exp it AN-A - rAn - V(t

)

e - `tx dt,

n

n

N

)I)

(4 4 10) where v (t) is the characteristic function of G We shall prove that

lim sup Jr,, (x) l =0

(4 4 11)

n-oo x

To prove this, write (4 4.10) as

fo

rn (x)

= 1

%(t)-v(t)}e-`txdt=

27r

-Go

= Zn (I1-I2+13+14),