Independent And Stationary Sequences Of Random Variables - Chapter 16 ppsx

± 1 .' Definition and general properties A random process Xt t c- T is called stationary in the strict sense if the distribution of the random vector Xtt +h, Xt 2 +h,.. The random proces

Chapter 16

SOME RESULTS FROM THE THEORY OF STATIONARY PROCESSES

In this chapter an account is given of those results from the theory of stationary processes which will be required in the sequel This chapter has much in common with Chapter 1, but here the proofs will, as a rule,

be given in full, although the discussion will be rather condensed For a more complete and detailed account we refer to chapters X and XI of [31], as well as [163]

± 1 ' Definition and general properties

A random process Xt (t c- T) is called stationary (in the strict sense) if the distribution of the random vector

(Xtt +h, Xt 2 +h, , Xt s +h)

does not depend on h, so long as the values t i + h belong to T (a subset of the real line)

The random process is called stationary (in the wide sense) ifE (Xr) < 00 for all t, and ifE(XX) and E(XSXX+t) do not depend on s Without loss of generality we can (and will) take E (XS ) = 0 (otherwise we can replace

XtbyXt - E (Xt) ) If no confusion can be caused, the qualifying parentheses (in the strict (or wide) sense) will be omitted

The parameter set T will be taken to be either the whole line or the set of integers (positive or negative), except when it is specifically stated that only non-negative values of t are considered We distinguish the two cases

as those of continuous time and discrete time ; stationary processes with

a discrete time parameter are often called stationary sequences It is nota-tionally convenient to write continuous time processes as X (t) or X (s), and discrete time processes asX§ orXj ;when both are considered together

we use the notation Xt or XS

16 1 DEFINITION AND GENERAL PROPERTIES 2 8 5

In the case of continuous time we assume that the process is stochastically continuous in the sense that, for all e > 0,

S-0

When dealing with wide-sense stationary processes, however, we shall assume the stronger condition

S-0

These conditions are very weak ; they are fulfilled in all cases of interest

A random process Xt is a function Xt(w) of two variables t c- T and w E 0, where (Q, R, P) is the underlying probability space (see ± 1 1) We shall assume that, as a function defined on (T x Q), Xt(w) is measurable with respect to the product a-algebra RT X R, where RT is the a-algebra of Lebesgue sets in T Every stochastically continuous process may be modified to satisfy this condition without altering the finite-dimensional distributions [31]

Example 1 The sequence of independent, identically distributed random variables

., X-1, XO, X1, X2, forms a stationary process in the strict sense

Example 2 Let , ~ _ 1, ~0 ' b11 ~2, be a sequence of independent, identically distributed random variables with E (~j)= 0, E (~ ;) =62 < cc

If the sequence ajis such that E'- cc a; < oc, then the equations

Xi = I, ak bk+ j = I ak- j ~k

determine a stationary sequence If it is assumed that the cj are not in-dependent, but merely orthogonal in the sense that E (~i~j) = 0 for i 0j, then the sequence X j is stationary only in the wide sense The verification

of these assertions is left to the reader

2 8 6

THEORY OF STATIONARY PROCESSES : SOME RESULTS

Chap 16

± 2 Stationary processes and the associated measure-preserving transformations

With each random process Xt (- oo < t< oo) we can associate a-algebras

9J1a (X) = 9J1' (- oo < a < b < oo ), where 9J1a is the a-algebra generated by the events of the form

A = {(Xt1 , X 2 , , Xts)EA}

(16 2 1) for a<,t l < t 2< < is<b and s-dimensional Borel sets A This a-algebra can be regarded as the closure of the set of events (16 2 1) with respect

to the metric

p (A, B) = P { (A - B) u (B - A) }

( 16 2 2)

A special role will be played by the a-algebras 9Nt

00

9

JJ1_ cc =nmt.

t clearly, for s>0,

Every stationary process defines a family of mappings Ttof 9N 0,, into itself given by the following rule

For events A of the form (16 2 1),

T t A= T t {(Xt1 , , Xrs)EA} = {(Xt1+t, , Xrs +t)EA} The set 91 of events of the form (16 2 1) is dense in 9J1 cc in the metric (16 2 2)

We can therefore extend Tt from 91 to T1., as its unique continuous ex-tension

The family of mappings Tt has the following properties (1) Tt is well-defined up to events of zero probability (2) P(PA)=P(A), AE9J2 cc

(3) Up to events of probability zero,

Tt( U A k = U T t (Ak) ,

T t

(n Ak = n T t (Ak) ,

A kE9X 0 , k

k

(Itl < oo) ,

1 6 2 STATIONARY PROCESSES 287

V(q) = V(A) A c- 9W , , Tt(T`A) = T - `(TtA)=A AE9J * (4) T t,+t2

=Tt, T` 2

i e T tl +t2 (A)= V , (T t2(A)) Transformations satisfying conditions (1)-(3) are called measure-preserv-ing transformations of 93t" Property (4) indicates that the transforma-tions Tt form a group (a semigroup if only non-negative values of the time parameter are considered)

Thus to any process X, stationary in the strict sense, there corresponds a group (Tt) of measure-preserving transformations on the o algebra 9i1

In the discrete time case, (Tt) is the cyclic group of powers of T = T 1 Conversely, every measure-preserving group of transformations, con-tinuous in the sense of (16 1 1), on a o- -algebra SJJZ c a, generates a family

of stationary processes Xt, and every stationary process is so generated

To prove this, associate with Tt a transformation Ti on the class of random variables measurable with respect to fit, defined by the conditions (1 1 ) Ti is well-defined up to differences on sets of zero probability (2 1 ) If X (A) is the indicator function of the event A e fit, then

Ti x (A) = x (Tt A) (3 1 ) The transformations Tl are linear ; for random variables measurable with respect to 9J t, and constants a, (3,

Ti (a~ + A) = a T1(~) + flTit (n) (4 1 ) The transformations Ti are continuous ; if n ~ with probability 1, then Ti (fin)-+Ti (~) with probability 1

Since each random variable ~, measurable with respect to M, is the limit

as n ' co with probability 1, of the random variables

~n

j

x

C

there exists one and only one transformation Ti (up to events of

probabili-ty 0), satisfying the conditions (1 1 )-{4 1 )

If now c is any random variable measurable with respect to SW, the ran-dom process defined by 4t = Ti ~ will be stationary in the strict sense Moreover, if Tt is the group,of transformations defined by the stationary

* This assumes the process defined for both positive and negative values oft

THEORY OF STATIONARY PROCESSES : SOME RESULTS

Chap 16

process X, then it is easy to see that

TitXt = Xt+ t ,

and in particular,

Xt = T' X0

We note two properties of T1 which follow from (3) and (2 1 )-(4 1 ) : T1(~q) = T1W T1('1)

(16 2 3) and

E(TT ~) = E(~)

( 16.2.4)

It is always possible to work, not with the transformation Tt of 9.N , into itself, but with a one-to-one point transformation of 0 itself In fact, by Kolmogorov's extension theorem for measures on product spaces, we can always take Q to be the set of finite real-valued functions w t defined

on T, with Xt (w) = wt We define transformations Tt of Q into itself by the equation

T t (CO), = wt + t

The stationarity implies that Tt preserves the measure of all sets in 2I, and thus of all in 9J2 C, The transformations Ti are given by

Ti(c (w)) = c(T-tw)

An event A E9N C is called invariant if, for all t, T' A = A (mod 0),

i.e p(TtA, A) = 0

Clearly all events with probability 0 or 1 are invariant If the group of transformations Tt has no other invariant events it is said to be metrically transitive From a probabilistic point of view, the absence of metric trans-sitivity implies a dependence between the distant past (JJL_ ,,,) and the future (D(J

© 3 Hilbert spaces associated with a stationary process

On a given probability space (0, R, P), consider the collection of all complex random variables ~, with EIz2l<oo It is easy to check (cf [2])

that

16 3 HILBERT SPACES 2 8 9

(~, h) = E(~i) has all the properties of a scalar product, endowed with which the collec-tion is a Hilbert space L 2 (Q)

Let Xtbe a stationary process (in the strict sense) defined on this probability space Then the a-algebra 99)1 determines the subspace H a(X) consisting

of those ~ eL2(Q)which are measurable with respect to 9J2Q With the obvious notation, for s>0,

H_ 00 cH`_ 00 cH`_+ JcH1 The restrictions U` of Tl to L 2 (Q) form a group of unitary operators (if only non-negative values of t are considered, a semigroup of isometric operators) In the discrete time case, this is the cyclic group of powers of the restriction U of Ti In fact, because of (16 2 3) and (16 2 4),

(U , U77)=E(U Urn)=EU( r~)=E( r7)_( ,r7)

We remark that the space H,,,, is separable In the discrete time case, the set of indicators of events of the form (16.2 1), with A an s-dimensional rectangle with rational vertices, forms a countable set which is contained

in no proper closed subspace ofHam In the continuous time case, we take the indicators of (16 2 1) with !as before and the t;rational

If the time parameter is continuous, the group (Ut ) is continuous in the sense that, for all ~, i , the scalar function

(U t ~,') = E(Ut~ q)

is continuous in t To prove this, it is sufficient to show that, for all ~ EH" ' , limJIUt+s~-Ut ~Jj =lim{EIU + s~_Ut~J 2}=0, (16 3 1)

and it is only necessary to prove this for a set of elements ~ dense in H,-,, Because of (16 1 1) the indicators ~ of events of the form (16 2 1) satisfy (16 3 1)

If the process Xt is stationary only in the wide sense, we can define as before the spaces Hs(X), but these are not very helpful because we cannot define the measure-preserving transformation Ut Instead we define the smaller subspace L'(X), the closed subspace generated by the variables

t

X„ (s ,< u ,< t) T>0,It is clear that, for

_'O L_ 00 _00

-00

- 11

THEORY OF STATIONARY PROCESSES : SOME RESULTS

Chap 16

We define the group (Ui) of unitary operators on L am, by

Ui (~cjxtj' _ c j XtJ+t J

J

If the process is stationary in both the wide and strict senses, then Ui is the restriction of Ut to L,, ; we shall use the symbol Ut for both operators

A family of projection operators E 2 (a < ~< b) is called a partition of the operator I if

(1) Ea =O, Eb =I, (2 ) E ;-0 = E2

(a<A,<b) , (3) E ; E,u = Emin (d,§)

We state two theorems on the representation of groups of unitary oper-ators as Fourier-Stieltjes transforms of partitions of the identity (the proof of which can be found in [2])

Theorem 16 3 1 (von Neumann-Wintner) To each unitary operator U there corresponds a partition of the identity E , (-?L < ), < n), such that

n U" =

J -n e`Z"dE

The integral is to be understood as the strong limit, as max (A 1 - ;~;) >0,

of the sums

e`.Z'" (E

E ) i

Theorem 16 3 2 (Stone) Let U t be a group of unitary operators in the Hilbert space H such that, for all , ri c- H, (Ut , ri) is a continuous function oft Then there exists a partition of the identity E 2 (- oo <A < oc) such that

Ut =

f~ e i t' dE,

(16.3 3)

- 00

If we apply these theorems to the operators Ut on L,,,, (X), we obtain the representation

Ut =

J `t ),dE 2

(16 3 4) where the integration is over [-it, 7t] in the discrete time case and over

(16 3 2)

16 4 AUTOCOVARIANCE AND SPECTRAL FUNCTIONS 2 9 1

(- oo, co) in the continuous time case We shall see in © 5 that this equa-tion has an important probabilistic meaning

© 4 Autocovariance and spectral functions of stationary processes

Let X X be a stationary process in the wide sense By definition, E (Xt XS ) depends only on (t - s)

E(XtXs) = Rt-5 The function R t _ s is called the autocovariance function of the process

It has two properties, apart from the obvious fact that R t = R - t (a) R t is continuous,

(b) R t is positive-definite [47]

In fact, by (16 1 2), IRt - RS) = JEX t X 0 -EXS X 0 < { EIX0 1 2 EIXt -XS I 2 }=->0

as s *t, and if z 1 , , z,, are arbitrary complex numbers, and t 1 , , t o points of the parameter set T,

R tk _ ti Z k Z j = Zk 2 E(XtkXt j) E E ZkXt k 2 i0

These properties (a), (b) imply that R t /R O is the characteristic function of some probability distribution In the continuous time case, the Bochner-Khinchin theorem shows that

R t = J

_ 00

e`t A dF(A) , while in the discrete time case Herglotz's theorem asserts that

R t =

n

J n eirz dF (A)

where in either case F is bounded and non-decreasing The function F ()) is called the spectral function of the process Xt If F ( ;!) is absolutely continuous, its derivative f (.?) = F'(),) is called the spectral density of the process The relation between the autocovariance and spectral functions is the same as that between the characteristic and distri-bution functions ; in particular they determine one another uniquely

292

THEORY OF STATIONARY PROCESSES : SOME RESULTS

Chap 16

It is a consequence of Kolmogorov's extension theorem that every posi-tive-definite continuous function R I defines a Gaussian process XI stationary in both senses (© 17 3) Consequently, every continuous posi-tive-definite function is the autocovariance function of some stationary process, and every bounded non-decreasing function is the spectral func-tion of some stafunc-tionary process

© 5 The spectral representation of stationary processes

In order to ascertain the probabilistic significance of the theorems of

© 3 about the spectral expansion of the family Ut, consider the random process

Z (~) = E z Xo From (16.3 4),

X t = UtX 0 = Je i tdE A Xo

'= Je i t 2 dZ(2)

(16 5 1)

To understand (16 5.1) we must find an intrinsic description of the pro-cess Z, and to this end we construct stochastic integrals of which (16 5.1)

is a particular case For a detailed account of such integrals see Chapter

IX of [31]

A random process Y (A) is said to have orthogonal increments if, for any values 21 <A21<4 < 24,

E{(Y( ) 14) - Y(23))(Y(L2)- Y 011))} = 0

To each such process there corresponds a non-decreasing function

Fy (2) = F (2) such that

E I Y( 2 2) - Y(2 1 )I 2 = F( 22)- F(21) ,

(22 >21)

It is convenient to write this relation in the symbolic form

E I d Y (x,)12 = dF (~)

(16 5.2)

In fact, one can set for example

E IY (2) - Y (20)1 2 ,

%A0) ,

F (a ) - E I Y (2) - Y(2o) 1 2 , (2<2O)'

16 5 THE SPECTRAL REPRESENTATION 293

where io is an arbitrary value of ? It is easy to see that (16 5.2) defines F uniquely up to an additive constant

Let Y (2) be a process with orthogonal increments, defined on some inter-val [a, b], and F(,1) the function associated with it in (16.5 2) Denote by

L 2 (dF) the complex Hilbert space of functions with

= f

b If (A)I2

IIfIIF dF(A)<

a

We define the stochastic integral

IA f (~) d Y ( ) for allfEL 2(dF) and all Borel sets A c [a, b] In fact it is sufficient to define

b

I(f) = f f(A) d Y (A)

a since we can then define

IAf(A)dY(A) = I (fx(A))

We first defineI (f)when f is a step function Ifa < a1 <a2< < an< band

0, (i <a 1 ), f(i) = c;, (a ;_1,<A<aj), (16 5.3)

0, (A>an) , then we define

n I(f) _ Z cJ[Y(aj-0)-Y(aj_1-0)],

j=2 where Y (A ± 0) denotes the limit, in the metric of L2(Q) of Y (A+ t) as t ).+O (Clearly Y(2 + 0) = Y (A) = Y (A- 0) at points of continuity of F(A) )

The integral I (f) defined in this way on the step function has immediately the following properties

(1) For any complex numbers a, /3 and any step functions f, g,