Long range dependence and self similarity

These topics concern, but are not limited to, physical models that give rise to long-range dependence and self-similarity; cen-tral and non-cencen-tral limit theorems for long-range depe

This modern and comprehensive guide to long-range dependence and self-similarity

starts with rigorous coverage of the basics, then moves on to cover more specialized,

up-to-date topics central to current research These topics concern, but are not limited

to, physical models that give rise to long-range dependence and self-similarity;

cen-tral and non-cencen-tral limit theorems for long-range dependent series, and the limiting

Hermite processes; fractional Brownian motion and its stochastic calculus; several

celebrated decompositions of fractional Brownian motion; multidimensional models

for long-range dependence and self-similarity; and maximum likelihood estimation

methods for long-range dependent time series Designed for graduate students and

researchers, each chapter of the book is supplemented by numerous exercises, some

designed to test the reader’s understanding, while others invite the reader to consider

some of the open research problems in the field today

V L A D A S P I P I R A S is Professor of Statistics and Operations Research at the

University of North Carolina, Chapel Hill

M U R A D S T A Q Q U is Professor of Mathematics and Statistics at Boston

University

C A M B R I D G E S E R I E S I N S TAT I S T I C A L A N D

P RO BA B I L I S T I C M AT H E M AT I C S

Editorial Board

Z Ghahramani (Department of Engineering, University of Cambridge)

R Gill (Mathematical Institute, Leiden University)

F P Kelly (Department of Pure Mathematics and Mathematical Statistics,

University of Cambridge)

B D Ripley (Department of Statistics, University of Oxford)

S Ross (Department of Industrial and Systems Engineering,

University of Southern California)

M Stein (Department of Statistics, University of Chicago)

This series of high-quality upper-division textbooks and expository monographs covers all aspects of stochastic applicable mathematics The topics range from pure and applied statis- tics to probability theory, operations research, optimization, and mathematical programming.

The books contain clear presentations of new developments in the field and also of the state of the art in classical methods While emphasizing rigorous treatment of theoretical methods, the books also contain applications and discussions of new techniques made possible by advances

in computational practice.

A complete list of books in the series can be found at www.cambridge.org/statistics.

Recent titles include the following:

19 The Coordinate-Free Approach to Linear Models, by Michael J Wichura

20 Random Graph Dynamics, by Rick Durrett

21 Networks, by Peter Whittle

22 Saddlepoint Approximations with Applications, by Ronald W Butler

23 Applied Asymptotics, by A R Brazzale, A C Davison and N Reid

24 Random Networks for Communication, by Massimo Franceschetti and Ronald Meester

25 Design of Comparative Experiments, by R A Bailey

26 Symmetry Studies, by Marlos A G Viana

27 Model Selection and Model Averaging, by Gerda Claeskens and Nils Lid Hjort

28 Bayesian Nonparametrics, edited by Nils Lid Hjort et al.

29 From Finite Sample to Asymptotic Methods in Statistics, by Pranab K Sen, Julio M Singer and Antonio

C Pedrosa de Lima

30 Brownian Motion, by Peter Mưrters and Yuval Peres

31 Probability (Fourth Edition), by Rick Durrett

33 Stochastic Processes, by Richard F Bass

34 Regression for Categorical Data, by Gerhard Tutz

35 Exercises in Probability (Second Edition), by Lọc Chaumont and Marc Yor

36 Statistical Principles for the Design of Experiments, by R Mead, S G Gilmour and A Mead

37 Quantum Stochastics, by Mou-Hsiung Chang

38 Nonparametric Estimation under Shape Constraints, by Piet Groeneboom and Geurt Jongbloed

39 Large Sample Covariance Matrices and High-Dimensional Data Analysis, by Jianfeng Yao, Shurong

Zheng and Zhidong Bai

40 Mathematical Foundations of Infinite-Dimensional Statistical Models, by Evarist Giné and Richard

Nickl

41 Confidence, Likelihood, Probability, by Tore Schweder and Nils Lid Hjort

42 Probability on Trees and Networks, by Russell Lyons and Yuval Peres

43 Random Graphs and Complex Networks (Volume 1), by Remco van der Hofstad

44 Fundamentals of Nonparametric Bayesian Inferences, by Subhashis Ghosal and Aad van der Vaart

45 Long-Range Dependence and Self-Similarity, by Vladas Pipiras and Murad S Taqqu

Long-Range Dependence and

University Printing House, Cambridge CB2 8BS, United Kingdom One Liberty Plaza, 20th Floor, New York, NY 10006, USA

477 Williamstown Road, Port Melbourne, VIC 3207, Australia 4843/24, 2nd Floor, Ansari Road, Daryaganj, Delhi – 110002, India

79 Anson Road, #06–04/06, Singapore 079906 Cambridge University Press is part of the University of Cambridge.

It furthers the University’s mission by disseminating knowledge in the pursuit of education, learning, and research at the highest international levels of excellence.

www.cambridge.org Information on this title: www.cambridge.org/9781107039469

DOI: 10.1017/9781139600347 c

 Vladas Pipiras and Murad S Taqqu 2017 This publication is in copyright Subject to statutory exception and to the provisions of relevant collective licensing agreements,

no reproduction of any part may take place without the written permission of Cambridge University Press.

First published 2017 Printed in the United States of America by Sheridan Books, Inc.

A catalog record for this publication is available from the British Library.

ISBN 978-1-107-03946-9 Hardback Cambridge University Press has no responsibility for the persistence or accuracy

of URLs for external or third-party internet websites referred to in this publication and does not guarantee that any content on such websites is, or will remain,

accurate or appropriate.

to Rachelle, Yael, Jonathan, Noah, Kai and Olivia

List of Abbreviations page xv

1 A Brief Overview of Time Series and Stochastic Processes 1

2 Basics of Long-Range Dependence and Self-Similarity 15

3 Physical Models for Long-Range Dependence and Self-Similarity 113

6 Fractional Calculus and Integration of Deterministic Functions with

List of Abbreviations xv

1.1.3 Weak or Second-Order Stationarity (of Increments) 4

1.4.1 Representations of a Gaussian Continuous-Time Process 12

2.2 Relations Between the Various Definitions of Long-Range

2.2.1 Some Useful Properties of Slowly and Regularly Varying Functions 19

2.3 Short-Range Dependent Series and their Several Examples 30

2.4 Examples of Long-Range Dependent Series: FARIMA Models 35

x Expanded Contents

2.5 Definition and Basic Properties of Self-Similar Processes 43

2.8 Connections Between Long-Range Dependent Series and

2.9 Long- and Short-Range Dependent Series with Infinite Variance 76

2.9.1 First Definition of LRD Under Heavy Tails: Condition A 76

2.9.2 Second Definition of LRD Under Heavy Tails: Condition B 82

2.9.3 Third Definition of LRD Under Heavy Tails: Codifference 82

2.11 Generation of Gaussian Long- and Short-Range Dependent Series 99

3 Physical Models for Long-Range Dependence and Self-Similarity 1133.1 Aggregation of Short-Range Dependent Series 113

3.3 Infinite Source Poisson Model with Heavy Tails 120

3.3.2 Workload Process and its Basic Properties 123

3.3.4 Limiting Behavior of the Scaled Workload Process 131

3.8 Motion of a Tagged Particle in a Simple Symmetric Exclusion

3.11.2 Correlations, Dimers and Pfaffians 184

3.11.5 Long-Range Dependence at Critical Temperature 212

4.1 Hermite Polynomials and Multiple Stochastic Integrals 229

4.2 Integral Representations of Hermite Processes 232

4.2.1 Integral Representation in the Time Domain 232

4.2.2 Integral Representation in the Spectral Domain 233

4.2.3 Integral Representation on an Interval 234

4.3 Moments, Cumulants and Diagram Formulae for Multiple Integrals 241

4.3.3 Relation Between Diagrams and Multigraphs 246

4.3.4 Diagram and Multigraph Formulae for Hermite Polynomials 251

5.1 Nonlinear Functions of Gaussian Random Variables 282

5.6.1 Direct Approach for Entire Functions 306

5.6.2 Approach Based on Martingale Differences 311

5.7.4 Multivariate Limits of Multilinear Processes 3245.8 Generation of Non-Gaussian Long- and Short-Range Dependent

xii Expanded Contents

5.8.2 Relationship Between Autocorrelations 331

6.1.1 Fractional Integrals on an Interval 345

6.1.2 Riemann–Liouville Fractional DerivativesD on an Interval 348

6.1.3 Fractional Integrals and Derivatives on the Real Line 352

6.1.4 Marchaud Fractional Derivatives D on the Real Line 354

6.2 Representations of Fractional Brownian Motion 359

6.2.1 Representation of FBM on an Interval 359

6.2.2 Representations of FBM on the Real Line 3686.3 Fractional Wiener Integrals and their Deterministic Integrands 369

6.3.1 The Gaussian Space Generated by Fractional Wiener Integrals 369

6.3.2 Classes of Integrands on an Interval 372

6.3.6 Classes of Integrands on the Real Line 383

6.3.7 Connection to the Reproducing Kernel Hilbert Space 384

6.4.3 Elementary Linear Filtering Involving FBM 392

7 Stochastic Integration with Respect to Fractional Brownian

7.1 Stochastic Integration with Random Integrands 397

7.1.1 FBM and the Semimartingale Property 397

7.2.1 Stochastic Differential Equations Driven by FBM 413

7.2.3 Numerical Solutions of SDEs Driven by FBM 418

7.2.4 Convergence to Normal Law Using Stein’s Method 426

8 Series Representations of Fractional Brownian Motion 437

8.1.1 The Case of General Stochastic Processes 437

8.2.3 Fractional Conjugate Mirror Filters 450

8.2.4 Wavelet-Based Expansion and Simulation of FBM 452

8.3.1 Complex-Valued FBM and its Representations 455

8.3.2 SpaceL aand its Orthonormal Basis 456

9.5.3 Special Subclasses and Examples of OFBFs 523

10.2.1 Whittle Estimation in the Spectral Domain 542

xiv Expanded Contents

10.6.3 Bias Reduction and Rate Optimality 565

A.1.1 Fourier Series and Fourier Transform for Sequences 575

A.2 Fourier Series of Regularly Varying Sequences 579

A.3.2 The Case of Locally Finite Measures 584A.4 Stable and Heavy-Tailed Random Variables and Series 585

B.1 Single Integrals with Respect to Random Measures 588

B.1.1 Integrals with Respect to Random Measures with Orthogonal

B.1.2 Integrals with Respect to Gaussian Measures 590

B.1.3 Integrals with Respect to Stable Measures 592

B.1.4 Integrals with Respect to Poisson Measures 593

B.1.5 Integrals with Respect to Lévy Measures 596B.2 Multiple Integrals with Respect to Gaussian Measures 597

C.4 Generator of the Ornstein–Uhlenbeck Semigroup 608

ACVF autocovariance function

FARIMA fractionally integrated autoregressive moving average

xvi List of Abbreviations

Numbers and sequences

x+, x− max{x, 0}, max{−x, 0}, respectively

,  real and imaginary parts, respectively

x ∧ y, x ∨ y min{x, y}, max{x, y}, respectively

[x], x (floor) integer part and (ceiling) integer part of x, respectively

a Fourier transform of sequence a

a∨ time reversion of sequence a

a ∗ b convolution of sequences a and b

↓2, ↑2 downsampling and upsampling by 2 operations, respectively

 p (Z) space of sequences{a n}n∈Zsuch that∞

n=−∞|a n|p < ∞

Functions

f Fourier transform of function f

f ∗ g convolution of functions f and g

f ⊗ g tensor product of functions f and g

f ⊗k kth tensor product of a function f

1A indicator function of a set A

log, log2 natural logarithm (base e) and logarithm base 2, respectively

e i x complex exponential, e i x = cos x + i sin x

B (·, ·) beta function

H k (·) Hermite polynomial of order k

C k (I) space of k–times continuously differentiable functions on an interval I

C b k (I) space of k–times continuously differentiable functions on an interval I

with the first k derivatives bounded

L p (R q ) space of functions f : Rq → R (or C) such thatRq | f (x)| p d x < ∞

 f  L p (R q ) (semi-)norm 

Rq | f (x)| p d x

1/p

L p (E, m) space of functions f : E → R (or C) such thatE | f (x)| p m (dx) < ∞

f←, f−1 (generalized) inverse of non-decreasing function f

xviii Notation

Matrices

Rp ×p,Cp ×p collections of p × p matrices with entries in R and C, respectively

A, A T transpose of a matrix A

A∗ Hermitian transpose of a matrix A

det(A) determinant of a matrix A

A ⊗ B Kronecker product of matrices A and B

A1/2 square root of positive semidefinite matrix A

ξ p (E|ξ| p )1/pfor a random variableξ

N (μ, σ2) normal (Gaussian) distribution with meanμ and variance σ2

χ(X1, , X p ) joint cumulant of random variables X1, , X p

D1,p , D k ,p domains of the Malliavin derivative

I n (·),  I n (·) multiple integrals of order n

a.e., a.s almost everywhere, almost surely, respectively

∼ asymptotic equivalence; that is, a (x) ∼ b(x) if a(x)/b(x) → 1

o (·), O(·) small and big O, respectively

B(A) σ –field of Borel sets of a set A

We focus in this book on range dependence and self-similarity The notion of

long-range dependence is associated with time series whose autocovariance function decays

slowly like a power function as the lag between two observations increases Such time series

emerged more than half a century ago They have been studied extensively and have been

applied in numerous fields, including hydrology, economics and finance, computer science

and elsewhere What makes them unique is that they stand in sharp contrast to

Markovian-like or short-range dependent time series, in that, for example, they often call for special

techniques of analysis, they involve different normalizations and they yield new limiting

objects

Long-range dependent time series are closely related to self-similar processes, which by

definition are statistically alike at different time scales Self-similar processes arise as large

scale limits of range dependent time series, and vice versa; they can give rise to

long-range dependent time series through their increments The celebrated Brownian motion is

an example of a self-similar process, but it is commonly associated with independence and,

more generally, with short-range dependence The most studied and well-known self-similar

process associated with long-range dependence is fractional Brownian motion, though many

other self-similar processes will also be presented in this book Self-similar processes have

become one of the central objects of study in probability theory, and are often of interest in

their own right

This volume is a modern and rigorous introduction to the subjects of long-range

depen-dence and self-similarity, together with a number of more specialized up-to-date topics at

the center of this research area Our goal has been to write a very readable text which will

be useful to graduate students as well as to researchers in Probability, Statistics, Physics and

other fields Proofs are presented in detail A precise reference to the literature is given in

cases where a proof is omitted Chapter 2 is fundamental It develops the basics of

long-range dependence and self-similarity and should be read by everyone, as it allows the reader

to gain quickly a basic familiarity with the main themes of the research area We assume that

the reader has a background in basic time series analysis (e.g., at the level of Brockwell and

Davis [186]) and stochastic processes The reader without this background may want to start

with Chapter 1, which provides a brief and elementary introduction to time series analysis

and stochastic processes

The rest of the volume, namely Chapters 3–10, introduces the more specialized and

advanced topics on long-range dependence and self-similarity Chapter 3 concerns physical

models that give rise to long-range dependence and/or self-similarity Chapters 4 and 5 focus

on central and non-central limit theorems for long-range dependent series, and introduce the

xxii Preface

limiting Hermite processes Chapters 6 and 7 are on fractional Brownian motion and itsstochastic calculus, and their connection to the so-called fractional calculus, the area of realanalysis which extends the usual derivatives and integrals to fractional orders Chapter 8 pur-sues the discussion on fractional Brownian motion by introducing several of its celebrateddecompositions Chapter 9 concerns multidimensional models, and Chapter 10 reviews themaximum likelihood estimation methods for long-range dependent time series Chapters 3through 10 may be read somewhat separately They are meant to serve both as a learningtool and as a reference Finally, Appendices A, B and C are used for reference throughoutthe book

Each chapter starts with a brief overview and ends with exercises and bibliographicalnotes In Appendix D, “Other notes and topics” contains further bibliographical information.The reader will find the notes extremely useful as they provide further perspectives on thesubject as well as suggestions for future research

A number of new books on long-range dependence and self-similarity, listed in Appendix

D, have been published in the last ten years or so Many features set this book apart First,

a number of topics are not treated elsewhere, including most of Chapter 8 (on positions) and Chapter 9 (on multidimensional models) Second, other topics provide amore systematic treatment of the area than is otherwise presently available; for example,

decom-in Chapter 3 (on physical models) Third, some specialized topics, such as decom-in Chapters

6 and 7 (on stochastic analysis for fractional Brownian motion), reflect our perspective.Fourth, most specialized topics are up to date; for example, the classical results of Chap-ters 4 and 5 (on Hermite processes and non-central limit theorems) have been supplemented

by recent work in the area Finally, the book contains a substantial number of early andup-to-date references Though even with this large number of references (more than athousand), we had to be quite selective and could not include every single work that wasrelevant

We would like to conclude this preface with acknowledgments and a tribute A ber of our former and present students have read carefully and commented on excerpts

num-of this book, including Changryong Baek, Stefanos Kechagias and Sandeep Sarangi atthe University of North Carolina, Shuyang Bai, Long Tao and Mark Veillette at BostonUniversity, Yong Bum Cho, Heng Liu, Xuan Yang, Pengfei Zhang, Yu Gu, Junyi Zhang,Emilio Seijo and Oliver Pfaffel at Columbia University Various parts of this volume havealready been used in teaching by the authors at these institutions As the book was tak-ing its final shape, a number of researchers read carefully individual chapters and providedinvaluable feedback We thus express our gratitude to Solesne Bourguin, Gustavo Didier,Liudas Giraitis, Kostas Spiliopoulos, Stilian Stoev, Yizao Wang We are grateful to DianaGillooly at Cambridge University Press for her support and encouragement throughoutthe process of writing the book We also thank the National Science Foundation and theNational Security Agency for their support We are responsible for the remaining errors andtypos, and would be grateful to the readers for a quick email to either author if such arefound

Finally, we would like to pay tribute to Benoit B Mandelbrot,1 a pioneer in the study

of long-range dependence and self-similarity, among his many other interests He was the

1 A fascinating account of Benoit B Mandelbrot’s life can be found in [681].

first to greatly popularize and draw attention to these subjects in the 1970s It was Benoit

B Mandelbrot who introduced one of the authors (M.S.T.) to this area, who in turn passed

on his interests and passion to the other author (V.P.) This volume, including the presented

specialized topics, are direct fruits of the work started by Benoit B Mandelbrot With his

passing in 2010, the scientific community lost one of its truly bright stars

A Brief Overview of Time Series and Stochastic

Processes

This chapter serves as a brief introduction to time series to readers unfamiliar with this topic

Knowledgeable readers may want to jump directly to Chapter 2, where the basics of

long-range dependence and self-similarity are introduced A number of references for the material

of this chapter can be found in Section 1.6, below

1.1 Stochastic Processes and Time Series

A stochastic processes {X(t)} t ∈T is a collection of random variables X (t) on some

proba-bility space( , F , P), indexed by the time parameter t ∈ T In “discrete time,” we typically

choose for T ,

Z = { , −1, 0, 1, }, Z+= {0, 1, }, {1, 2, , N}, , and denote t by n In “continuous time,” we will often choose for T ,

R, R+= [0, ∞), [0, N],

In some instances in this volume, the parameter space T will be a subset ofRq , q ≥ 1,

and/or X (t) will be a vector with values in R p , p ≥ 1 But for the sake of simplicity, we

suppose in this chapter that p and q equal 1 We also suppose that X (t) is real-valued.

One way to think of a stochastic process is through its law The law of a stochastic process

{X(t)} t ∈T is characterized by its finite-dimensional distributions (fdd, in short); that is, the

Here, the prime indicates transpose, and all vectors are column vectors throughout Thus,

the finite-dimensional distributions of a stochastic process fully characterize its law and, in

particular, the dependence structure of the stochastic process In order to check that two

stochastic processes have the same law, it is therefore sufficient to verify that their

finite-dimensional distributions are identical Equality and convergence in distribution is denoted

by= andd → respectively Thus {X(t)} d t ∈T = {Y (t)} d t ∈T means

P(X(t1) ≤ x1, , X(t n ) ≤ x n ) = P(Y (t1) ≤ x1, , Y (t n ) ≤ x n ), t i ∈ T, x i ∈ R, n ≥ 1.

2 A Brief Overview of Time Series and Stochastic Processes

A tilde∼ indicates equality in distribution: for example, we write X ∼ N (μ, σ2) if X is

Gaussian with meanμ and variance σ2

A stochastic process is often called a time series, particularly when it is in discrete time

and the focus is on its mean and covariance functions

1.1.1 Gaussian Stochastic Processes

A stochastic process {X(t)} t ∈T is Gaussian if one of the following equivalent conditions

holds:

(i) The finite-dimensional distributions Z = (X(t1), , X(t n ))are multivariate Gaussian

N (b, A) with mean b = EZ and covariance matrix A = E(Z − EZ)(Z − EZ);

(ii) a1X (t1) + · · · + a n X (t n ) is a Gaussian random variable for any a i ∈ R, t i ∈ T ;

(iii) In the case when EX(t) = 0, for any a i ∈ R, t i ∈ T ,

E exp {i(a1X (t1) + · · · + a n X (t n ))} = exp −1

ance function is defined as Cov(X(t), X(s)) = EX (t) − EX(t)X (s) − EX(s), s , t ∈ T

Together with the mean function EX(t), t ∈ T , the covariance determines the law of the

Gaussian stochastic process

Example 1.1.1 (Brownian motion) Brownian motion (or Wiener process) is a Gaussian

stochastic process{X(t)} t≥0with1

EX(t) = 0, EX(t)X(s) = σ2min{t, s}, t, s ≥ 0, σ > 0, (1.1.2)

or, equivalently, it is a Gaussian stochastic process with independent increments X (t k ) −

X (t k−1), k = 1, , n, with t0 ≤ t1 ≤ · · · ≤ t n such that X (t) − X(s) ∼ σ N (0, t − s),

t ≥ s > 0 Brownian motion is often denoted B(t) or W(t) Brownian motion {B(t)} t∈Ron

the real line is defined as B (t) = B1(t), t ≥ 0, and B(t) = B2(−t), t < 0, where B1and B2are two independent Brownian motions on the half line

Example 1.1.2 (Ornstein–Uhlenbeck process) The Ornstein–Uhlenbeck (OU) process is a

Gaussian stochastic process{X(t)} t∈Rwith

EX(t) = 0, EX(t)X(s) = σ2

2λ e −λ(t−s) , t > s, (1.1.3)

1 One imposes, sometimes, the additional condition that the process has continuous paths; but we consider here

only the finite-dimensional distributions.

0 t h t + h

Figure 1.1 If the process has stationary increments, then, in particular, the

increments taken over the bold intervals have the same distributions

whereλ > 0, σ > 0 are two parameters The OU process is the only Gaussian stationary

Markov process It satisfies the Langevin stochastic differential equation

d X (t) = −λX(t)dt + σ dW(t),

where{W(t)} is a Wiener process The term −λX(t)dt in the equation above adds a drift

towards the origin

1.1.2 Stationarity (of Increments)

A stochastic process{X(t)} t ∈T is (strictly) stationary if T = R or Z or R+ orZ+, and for

any h ∈ T ,

{X(t)} t ∈T = {X(t + h)} d t ∈T (1.1.4)

A stochastic process{X(t)} t ∈T is said to have (strictly) stationary increments if T = R or

Z or R+orZ+, and for any h ∈ T ,

{X(t + h) − X(h)} t ∈T = {X(t) − X(0)} d t ∈T (1.1.5)See Figure 1.1

Example 1.1.3 (The OU process) The OU process in Example 1.1.2 is strictly stationary

Its finite-dimensional distributions are determined, for t > s, by

EX(t)X(s) = σ2

2λ e −λ(t−s)=

σ2

2λ e −λ((t+h)−(s+h)) = EX(t + h)X(s + h).

Thus, the law of the OU process is the same when shifted by h∈ R

An example of a stochastic process with (strictly) stationary increments is Brownian

motion in Example 1.1.1

There is an obvious connection between stationarity and stationarity of increments If

T = Z and {X t}t∈Zhas (strictly) stationary increments, thenX t = X t − X t−1, t ∈ Z, is

(strictly) stationary Indeed, for any h∈ Z,

{X t +h}t∈Z = {X t +h − X t +h−1}t∈Z= {X t +h − X h − (X t +h−1 − X h )} t∈Z

d

= {X t − X0− (X t−1− X0)} t∈Z = {X t − X t−1}t∈Z = {X t}t∈Z.

Conversely, if{Y t}t∈Z is (strictly) stationary, then X t =t

k=1Y kcan be seen easily to have(strictly) stationary increments

If T = R, then the difference operator  is replaced by the derivative when it exists and

the sum is replaced by an integral

4 A Brief Overview of Time Series and Stochastic Processes

1.1.3 Weak or Second-Order Stationarity (of Increments)

The probabilistic properties of (strictly) stationary processes do not change with time Insome circumstances, such as modeling, this is sometimes too strong a requirement Instead

of focusing on all probabilistic properties, one often requires instead that only order properties do not change with time This leads to the following definition of (weak)stationarity

second-A stochastic process{X(t)} t ∈T is (weakly or second-order) stationary if T = R or Z and

for any t , s ∈ T ,

EX(t) = EX(0), Cov(X(t), X(s)) = Cov(X(t − s), X(0)). (1.1.6)

The time difference t − s above is called the time lag Weakly stationary processes are often called time series Note that for Gaussian processes, weak stationarity is the same as strong

stationarity This, however, is not the case in general

1.2 Time Domain Perspective

Consider a (weakly) stationary time series X = {X n}n∈Z In the time domain, one focuses

on the functions

γ X (h) = Cov(X h , X0) = Cov(X n +h , X n ), ρ X (h) = γ X (h)

γ X (0) , h, n ∈ Z, (1.2.1)called the autocovariance function (ACVF, in short) and autocorrelation function (ACF, in short) of the series X , respectively ACVF and ACF are measures of dependence in time

series Sample counterparts of ACVF and ACF are the functions

ρ(h − 1) have a nontrivial (asymptotic) dependence structure.

1.2.1 Representations in the Time Domain

“Representing” a stochastic process is expressing it in terms of simpler processes The

following examples involve representations in the time domain.

Example 1.2.1 (White Noise) A time series X n = Z n , n ∈ Z, is called a White Noise,

denoted{Z n } ∼ WN(0, σ2

Z ), if EZ n= 0 and

Example 1.2.2 (MA(1) series) A time series{X n}n∈Zis called a Moving Average of order

one (MA(1) for short) if it is given by

Example 1.2.3 (AR (1) series) A (weakly) stationary time series {X n}n∈Z is called

Autoregressive of order one (AR (1) for short) if it satisfies the AR(1) equation

X n = ϕX n−1+ Z n , n ∈ Z,

where{Z n } ∼ WN(0, σ2

Z ) To see that AR(1) time series exists at least for some values of

ϕ, suppose that |ϕ| < 1 Then, we expect that

X n = ϕ2X n−2+ ϕZ n−1+ Z n

= ϕ m X n −m + ϕ m−1Z

n −(m−1) + · · · + Z n=

∞

for|ϕ| < 1 One can easily see that it satisfies the AR(1) equation and is (weakly) stationary.

Hence, the time series{X n}n∈Zis AR(1).

When|ϕ| > 1, AR(1) time series is obtained by reversing the AR(1) equation as

X n = ϕ−1X n+1− ϕ−1Z n+1, n ∈ Z,

and performing similar substitutions as above to obtain

2A random variable is well-defined in the L2( )–sense if E|X|2< ∞ A series∞n=1X nis well-defined in the

L2( )–sense ifN

n=1X n converges in L2( )–sense as N → ∞, that is, E|N2

n =N1X n| 2 → 0 as

N1, N2 → ∞.

6 A Brief Overview of Time Series and Stochastic Processes

X n = ϕ−1X n+1− ϕ−1Z n+1

= ϕ−2X n+2− ϕ−2Z n+2− ϕ−1Z n+1 = −∞

m=0

ϕ −(m+1) Z n +m+1

When |ϕ| = 1, there is no (weakly) stationary solution to the AR(1) equation When

ϕ = 1, the AR(1) equation becomes X n − X n−1 = Z n and the non-stationary (in fact,

stationary increment) time series satisfying this equation is called Integrated of order one (I(1) for short) When Z n are i.i.d., this time series is known as a random walk.

For|ϕ| < 1, for example, observe that for h ≥ 0,

ρ X (h) = ϕ |h|

Example 1.2.4 (ARMA (p,q) series) A (weakly) stationary time series{X n}n∈Z is called

Autoregressive moving average of orders p and q (ARMA (p, q), for short) if it satisfies the

equation

X n − ϕ1X n−1− · · · − ϕ p X n −p = Z n + θ1Z n−1+ · · · + θ q Z n −q ,

where{Z n } ∼ WN(0, σ2

Z ).

ARMA(p, q) time series exists if the so-called characteristic polynomial 1 − ϕ1z− · · · −

ϕ p z p = 0 does not have root on the unit circle {z : |z| = 1} This is consistent with the

AR(1) equation discussed above where the root z = 1/ϕ1of the polynomial 1− ϕ1z= 0 is

on the unit circle when|ϕ1| = 1

Example 1.2.5 (Linear time series) A time series is called linear if it can be written as

X n=

∞

k=−∞

where {Z n } ∼ WN(0, σ2

Z ) and∞k=−∞|a k|2 < ∞ Time series in Examples 1.2.1–1.2.3

are, in fact, linear Observe that

k=−∞

a k +h a k = σ2

Z

∞

k=−∞

a h −k a −k = σ2

Z (a ∗ a∨) h ,

where ∗ stands for the usual convolution, and a∨ denotes the time reversal of a Since

EX n +h X h depends only on h and EX n = 0, linear time series are (weakly) stationary The

variables Z n entering the linear series (1.2.2) are known as innovations, especially when

they are i.i.d

Remark 1.2.6 Some of the notions above extend to continuous-time stationary processes

{X(t)} t∈R For example, such process is called linear when it can be represented as

X (t) =



Ra (t − u)Z(du), t ∈ R, (1.2.3)

where Z (du) is a real-valued random measure on R with orthogonal increments and control

measureE(Z(du))2= du (see Appendix B.1, as well as Section 1.4 below), and a ∈ L2(R)

is a deterministic function

1.3 Spectral Domain Perspective

We continue considering (weakly) stationary time series X = {X n}n∈Z The material of this

section is also related to the Fourier series and transform discussed in Appendix A.1.1

h=−∞

e −ihλ γ X (h), λ ∈ (−π, π], (1.3.1)

called the spectral density of the time series X The variable λ is restricted to the domain

(−π, π] since the spectral density f X (λ) is 2π–periodic: that is, f X (λ + 2πk) = f X (λ),

k ∈ Z Observe also that the spectral density f X is well-defined pointwise whenγ X ∈ 1(Z).

The variableλ enters f X (λ) through e −ihλ, or sines and cosines Whenλ is close to 0, we

will talk about low frequencies (long waves), and when λ is close to π, we will have high

frequencies (short waves) in mind Graphically, the association is illustrated in Figure 1.2.

Example 1.3.1 (Spectral density of white noise) If{Z n } ∼ WN(0, σ2

Z ), then

f Z (λ) = σ Z2

2π , that is, the spectral density f Z (λ), λ ∈ (−π, π], is constant.

Figure 1.2 Low (left) and high (right) frequencies.

8 A Brief Overview of Time Series and Stochastic Processes

Example 1.3.2 (Spectral density of AR (1) series) If {X n}n∈Z is AR(1) time series with

The spectral density has the following properties:

● Symmetry: f X (λ) = f X (−λ) This follows from γ X (h) = γ X (−h) In particular, we can

follows from observing that

γ Y (h) = EY h Y0=

∞

Example 1.3.3 (Spectral density of AR (1) series, cont’d) Applying (1.3.3)–(1.3.4) to the

AR(1) equation X n − ϕX n−1 = Z nwith{Z n } ∼ WN(0, 1) yields

I X (λ) is known as the periodogram, and has the following properties:

● Computational speed: I X (λ k ) can be computed efficiently by Fast Fourier Transform

(FFT) in O (N log N) steps, supposing N can be factored out in many factors.

● Statistical properties: I X (λ) is not a consistent estimator for 2π f X (λ), but is

asymptoti-cally unbiased The periodogram needs to be smoothed to become consistent

Warning: Two definitions of the periodogram I X are commonly found in the literature

One definition appears in (1.3.5) The other popular definition is to set the whole left-hand

side of (1.3.5) for the periodogram; that is, to incorporate the denominator 2π into the

peri-odogram Since the two definitions are different, it is important to check which convention

is used in a given source With the definition (1.3.5), we follow the convention used in

Brockwell and Davis [186]

where Z X (dλ) is a complex-valued random measure such that Z X (−dλ) = Z X (dλ); that

is, Z X is Hermitian Moreover,

EZ X (dλ)Z X (dλ) = 0 (1.3.8)

10 A Brief Overview of Time Series and Stochastic Processes

when d λ = dλ(i.e., having orthogonal increments), and

Remark 1.3.4 In writing the spectral representation (1.3.7) we assumed implicitly that

the series X has a spectral density Spectral representations, however, exist for all (weakly)

stationary time series They are written more generally as

the property (1.3.9) is replaced by

E|Z X (dλ)|2= F X (dλ) for the so-called spectral measure F X on (−π, π] When the spectral measure F X has a

density f X (with respect to the Lebesgue measure), f X is the spectral density of the series

X and the relation (1.3.9) holds.

Example 1.3.5 (Spectral density of AR (1) series, cont’d) The spectral density of AR(1)

series was derived in Examples 1.3.2 and 1.3.3 Typical plots of AR(1) time series and their

0 0.5 1 1.5 2

f( λ ) f( λ )

0 0.5 1 1.5

n n

X n

X n

–5 –5

0 5

Figure 1.3 Typical plots of AR(1) spectral densities and of their sample paths in

the casesϕ > 0 and ϕ < 0.

spectral densities in the casesϕ > 0 and ϕ < 0 are given in Figure 1.3 These plots are

consistent with the idea behind spectral representation described above

Remark 1.3.6 If{X n}n∈Zis given by its spectral representation, then by using (1.3.8) and

which is the relation (1.3.2) connecting the spectral density to the ACVF

Remark 1.3.7 Suppose that f X (λ) = |g X (λ)|2with g X (λ) = g X (−λ), which happens in

many examples Then,

Remark 1.3.8 Many notions above extend to continuous-time stationary processes

{X(t)} t∈R The spectral density of such a process is defined as

with the difference from (1.3.1) that it is a function forλ ∈ R, and the sum is replaced by an

integral The inverse relation is

control measureE|Z X (dλ)|2= f X (λ)dλ.

1.4 Integral Representations Heuristics

The spectral representation (1.3.7) has components which are summarized through the

first two columns of Table 1.1; that is, the dependence structure of X n is transferred

into the deterministic functions e i n λ But one can also think of more general (integral)

12 A Brief Overview of Time Series and Stochastic Processes

Table 1.1 Components in representations of time series and stochastic processes.

Components in representation Time series Stochastic process

t (u)

Uncorrelated (or independent) random measure Z X (dλ) M(du)

Example 1.4.1 (Linear time series) The linear time series in (1.2.2) is in fact definedthrough an integral representation since

Various random measures and integral representations are defined and discussed inAppendix B.2 The following example provides a heuristic explanation of Gaussian randommeasures and their integrals

Example 1.4.2 (Gaussian random measure) Suppose E = R and set M(du) = B(du), viewing B (du) as the increment on an infinitesimal interval du of a standard Brownian

motion{B(u)} u∈R Since Brownian motion has stationary and independent increments, and

EB2(u) = |u|, one can think of the random measure B(du) as satisfying

Rh (u)B(du), each B(du) is weighted by the nonrandom factor h(u), and since the B(du)s

are independent on disjoint intervals, one expects that

Formally, the integral I (h) =E h (u)M(du) is defined first for simple functions h and then,

by approximation, for all functions satisfying

E h2(u)du < ∞.

1.4.1 Representations of a Gaussian Continuous-Time Process

Let h ∈ L2(R, du); that is, Rh2(u)du < ∞ The Fourier transform of h is  h (x) =



Re i ux h (x)du with the inverse formula h(u) = 1

2π



Re −iuxh (x)dx (see Appendix A.1.2).

It is complex-valued and Hermitian; that is, h (dx) =  h (−dx) Introduce a similar formation on B (du), namely, let  B (dx) =  B (−dx) be complex-valued, with  B (dx) =

trans-B1(dx) + i B2(dx), where B1(dx) and B2(dx) are real-valued, independent N (0, dx/2),

and require B to be Hermitian; that is,  B (dx) =  B (ưdx) Then, E| B (dx)|2= dx and

h (x) B (dx) =:  I ( h ).

See Appendix B.1 for more details

Example 1.4.3 (The OU process) Consider a stochastic process

X (t) = σ

 t

ư∞e

ư(tưu)λ B (du), t ∈ R,

whereσ > 0, λ > 0 are parameters, and B(du) is a Gaussian random measure on R with the

Lebesgue control measure du The process X (t) is Gaussian with zero mean and covariance

The integral representation above is in the time domain; that is, I (h) =Rh t (u)B(du)

with h t (u) = σ e ư(tưu)λ1{u<t} Observe that



1(0,∞) (t ư u) ư 1 (0,∞) (ưu)B (du), where B (du) is a Gaussian random measure with the control measure du With h t (u) =

e i xt ư 1

i x B (dx).

14 A Brief Overview of Time Series and Stochastic Processes

A reader wishing to learn more about time series analysis and stochastic processescould consult the references given in Section 1.6 below These references are helpful inunderstanding better the material presented in subsequent chapters

1.5 A Heuristic Overview of the Next Chapter

In the next chapter, we introduce basic concepts and results involving long-range dence and self-similarity Because the precise definitions can be rather technical, we providehere a brief and heuristic overview

depen-There are several definitions of long-range dependence which are, in general, not alent Basically, a stationary series{X n}n∈Z is long-range dependent if its autocovariancefunctionγ X (k) = EX k X0− EX k EX0behaves like k 2d−1as k → ∞, where 0 < d < 1/2 This range of d ensures that∞

equiv-k=−∞γ X (k) = ∞ From a spectral domain perspective, the spectral density f X (λ) of {X n}n∈Zbehaves asλ −2d as the frequencyλ → 0 Since d > 0,

note that the spectral density diverges as λ → 0 A typical example is FARIMA(0, d, 0)

series introduced in Section 2.4.1

We also define the related notion of self-similarity A process{Y (t)} t∈Ris H –self-similar

if, for any constant c, the finite-dimensional distributions of {Y (ct} t∈Rare the same as those

of{c H Y (t)} t∈R, where H is a parameter often related to d In fact, if the process {Y (t)} has stationary increments, then X n = Y (n) − Y (n − 1), n ∈ Z, has long-range dependence with

H = d + 1/2 Conversely, we can obtain Y (t) from X nby using a limit theorem

Fractional Brownian motion is a typical example of Y (t) It is Gaussian, H–self-similar,

and has stationary increments We provide both time-domain and spectral-domain tations for fractional Brownian motion We also give additional examples of non-Gaussianself-similar processes, such as the Rosenblatt process and also processes with infinite vari-ance defined through their integral representations, for instance, linear fractional stablemotion and the Telecom process

represen-1.6 Bibliographical Notes

There are a number of excellent textbooks on time series and their analysis The monograph

by Brockwell and Davis [186] provides a solid theoretical foundation The classic by ley [833] has served generations of scientists interested in the spectral analysis of time series.For more applied and computational aspects of the time series analysis, see Cryer and Chan[271], Shumway and Stoffer [909] Nonlinear time series are treated in Douc, Moulines, andStoffer [327]

Priest-On the side of stochastic processes, Lindgren [635] provides an up-to-date treatment ofstationary stochastic processes The basics of Brownian motion and related stochastic cal-culus are treated in Karatzas and Shreve [549], Mörters and Peres [733] A number of otherfacts used in this monograph are discussed in Appendices B and C

Basics of Long-Range Dependence and

Self-Similarity

This chapter serves as a general introduction It is necessary for understanding the more

advanced topics covered in subsequent chapters We introduce here basic concepts and

results involving long-range dependence and self-similarity

There are various definitions of long-range dependence, not all equivalent, because of the

presence of slowly varying functions These functions vary slower than a power; for

exam-ple, they can be a constant or a logarithm We give five definitions of long-range dependence

As discussed in Section 2.2, these definitions can be equivalent depending on the choice of

the slowly varying functions The reader should read the statement of the various

proposi-tions of that section, but their proofs which are somewhat technical may be skipped in a first

reading

Short-range dependent time series are introduced in Section 2.3 A typical

exam-ple is the ARMA(p, q) Examples of time series with long-range dependence, such as

FARIMA(0, d, 0) and FARIMA(p, d, q) are given in Section 2.4 Self-similarity is defined

in Section 2.5 Examples of self-similar processes are given in Section 2.6 These include:

● Fractional Brownian motion

● Bifractional Brownian motion

● The Rosenblatt process

● S αS Lévy motion

● Linear fractional stable motion

● Log-fractional stable motion

● The Telecom process

● Linear fractional Lévy motion

The Lamperti transformation, relating self-similarity to stationarity is stated in Section 2.7

Section 2.8 deals with the connection between long-range dependence and self-similarity

The infinite variance case is considered in Section 2.9 Section 2.10 focuses on heuristic

methods of estimation of the self-similarity parameter These include:

● The R/S method

● The aggregated variance method

● Regression in the spectral domain

● Wavelet-based estimation

Finally, we indicate in Section 2.11 how to generate long- and short-range dependent series,

using the

16 Basics of Long-Range Dependence and Self-Similarity

● Cholesky decomposition

whose complexity is O (N3) and hence relatively slow, and the

● Circulant matrix embedding

which is an exact method with the complexity O (N log N).

2.1 Definitions of Long-Range Dependent Series

We start with the notion of long-range dependence, also called long memory or strong dence It is commonly defined for second-order stationary time series Recall from Sections 1.1.3 and 1.2 that a second-order stationary time series X = {X n}n∈Zhas a constant mean

depen-μ X = EX nand an autocovariance function

γ X (n − m) = EX n X m − EX n EX m = EX n X m − μ2

X , where “n”, “m” are viewed as time and γ X depends only on the time lag|m − n| One can characterize the time series X by specifying its mean μ X and autocovarianceγ X (n), n ∈ Z,

or one can take a “spectral domain” perspective by focusing on the spectral density f X (λ),

λ ∈ (−π, π], of the time series, if it exists (see Section 1.3) It is defined as

 π

−π e

i n λ f

X (λ)dλ = γ X (n), n ∈ Z, that is, the spectral density f X (λ) is a function whose Fourier coefficients are the

autocovariances γ X (n) (up to a multiplicative constant following the convention of

and a slowly varying function L.

Definition 2.1.1 A function L is slowly varying at infinity if it is positive on [c, ∞) with

c ≥ 0 and, for any a > 0,

models that give rise to long- range dependence and/ or self- similarity Chapters and focus

on central and non-central limit theorems for long- range dependent series, and introduce the