Fixed domain asymptotics and consistent estimation for gaussian random field models in spatial statistics and computer experiments

Chapter 2 studiesthe fixed domain asymptotic behavior of the tapered MLE for the microergodicparameter of isotropic Mat´ern class covariance function when the taper support is ap-allowed

CONSISTENT ESTIMATION FOR GAUSSIAN RANDOM FIELD MODELS IN SPATIAL

STATISTICS AND COMPUTER

EXPERIMENTS

WANG DAQING

(B.Sc University of Science and Technology of China)

A THESIS SUBMITTED FOR THE DEGREE OF DOCTOR OF PHILOSOPHY DEPARTMENT OF STATISTICS AND APPLIED

PROBABILITY NATIONAL UNIVERSITY OF SINGAPORE

2010

I am so grateful that I have Professor Loh Wei Liem as my supervisor He istruly a great mentor not only in statistics but also in daily life I would like to thankhim for his guidance, encouragement, time, and endless patience Next, I wouldlike to thank my senior Li Mengxin for discussion on various topics in research Ialso thank all my friends who helped me to make life easier as a graduate student

I wish to express my gratitude to the university and the department for supporting

me through NUS Graduate Research Scholarship Finally, I will thank my familyfor their love and support

2.1 Introduction 11

2.2 Main Results 18

2.3 Some Probability Inequalities 21

2.4 Spectral Analysis 28

2.5 Tapered Covariance Functions 40

2.6 Proofs 47

2.7 Simulations 68

2.7.1 Precision of Theorem 2.3 approximations for finite 𝑛 68

2.7.2 Precision of Theorem 2.1 and 2.2 approximations for finite 𝑛 71 Chapter 3 Multiplicative Covariance Function 89 3.1 Introduction 89

3.2 Quadratic Variation 96

3.3 Spectral Analysis 108

3.3.1 Spectral Density Function 108

3.3.2 Mean of the Periodogram 124

3.3.3 Covariance and Variance of the Periodogram 136

3.3.4 Consistent Estimation 148

3.4 Multiplicative Mat´ern Class 151

3.5 Simulations 152

Our work has two parts The first part (Chapter 2) deals with isotropic variance function Maximum likelihood is a preferred method for estimating the

co-covariance parameters However, when the sample size 𝑛 is large, it is a burden

to compute the likelihood Covariance tapering is an effective technique to proximating the covariance function with a taper (usually a compactly supportedcorrelation function) so that the computation can be reduced Chapter 2 studiesthe fixed domain asymptotic behavior of the tapered MLE for the microergodicparameter of isotropic Mat´ern class covariance function when the taper support is

ap-allowed to shrink as 𝑛 → ∞ In particular when 𝑑 ≤ 3, conditions are established

in which the tapered MLE is strongly consistent and asymptotically normal

The second part (Chapter 3) establishes consistent estimators of the covarianceand scale parameters of Gaussian random field with multiplicative covariance func-

tion When 𝑑 = 1, in some cases it is impossible to consistently estimate them simultaneously under fixed domain asymptotics However, when 𝑑 > 1, consistent

estimators of functions of covariance and scale parameters can be constructed byusing quadratic variation and spectral analysis Consequently, they provide theconsistent estimators of the covariance and scale parameters

List of Tables

Table 2.1 Percentiles (standard errors in parentheses), Means and dard Deviations (SD) of√ 𝑛(ˆ𝜎2

Stan-1𝛼 2𝜈 1,𝑛 − 𝜎2𝛼 2𝜈 )/( √ 2𝜎2𝛼 2𝜈), and Biases

and Mean Square Errors (MSE) of estimator ˆ𝜎2

1,𝑛 𝛼 2𝜈

1 for 𝜎 = 1, 𝛼 = 0.8, 𝛼1 = 1.3, 𝜈 = 1/4 74

Table 2.2 Percentiles (standard errors in parentheses), Means and dard Deviations (SD) of√ 𝑛(ˆ𝜎2

Stan-1𝛼 2𝜈 1,𝑛 − 𝜎2𝛼 2𝜈 )/( √ 2𝜎2𝛼 2𝜈), and Biases

and Mean Square Errors (MSE) of estimator ˆ𝜎2

1,𝑛 𝛼 2𝜈

1 for 𝜎 = 1, 𝛼 = 0.8, 𝛼1 = 1.3, 𝜈 = 1/2 75

Table 2.3 Percentiles (standard errors in parentheses), Means and dard Deviations (SD) of√ 𝑛(ˆ𝜎2

Stan-1𝛼 2𝜈 1,𝑛 − 𝜎2𝛼 2𝜈 )/( √ 2𝜎2𝛼 2𝜈), and Biases

and Mean Square Errors (MSE) of estimator ˆ𝜎2

1,𝑛 𝛼 2𝜈

1 for 𝜎 = 1, 𝛼 = 0.8, 𝛼1 = 2, 𝜈 = 1/4 76

Table 2.4 Percentiles (standard errors in parentheses), Means and dard Deviations (SD) of√ 𝑛(ˆ𝜎2

Stan-1𝛼 2𝜈 1,𝑛 − 𝜎2𝛼 2𝜈 )/( √ 2𝜎2𝛼 2𝜈), and Biases

and Mean Square Errors (MSE) of estimator ˆ𝜎2

1,𝑛 𝛼 2𝜈

1 for 𝜎 = 1, 𝛼 = 0.8, 𝛼1 = 2, 𝜈 = 1/2 77

Table 2.5 Percentiles (standard errors in parentheses), Means and

Stan-dard Deviations (SD) of√ 𝑛(ˆ𝜎2

1𝛼 2𝜈 1,𝑛 − 𝜎2𝛼 2𝜈 )/( √ 2𝜎2𝛼 2𝜈), and Biases

and Mean Square Errors (MSE) of estimator ˆ𝜎2

1,𝑛 𝛼 2𝜈

1 for 𝑑 = 1, 𝜎 =

1, 𝛼 = 5, 𝛼1 = 7.5, 𝜈 = 1/4 and 𝜙 1,1 (𝑥/𝛾 𝑛) 78Table 2.6 Percentiles (standard errors in parentheses), Means and Stan-

dard Deviations (SD) of√ 𝑛(ˆ𝜎2

1𝛼 2𝜈 1,𝑛 − 𝜎2𝛼 2𝜈 )/( √ 2𝜎2𝛼 2𝜈), and Biases

and Mean Square Errors (MSE) of estimator ˆ𝜎2

1,𝑛 𝛼 2𝜈

1 for 𝑑 = 1, 𝜎 =

1, 𝛼 = 5, 𝛼1 = 7.5, 𝜈 = 1/4 and 𝜙 1,1 (𝑥/𝛾 𝑛) 79Table 2.7 Percentiles (standard errors in parentheses), Means and Stan-

dard Deviations (SD) of√ 𝑛(ˆ𝜎2

1𝛼 2𝜈 1,𝑛 − 𝜎2𝛼 2𝜈 )/( √ 2𝜎2𝛼 2𝜈), and Biases

and Mean Square Errors (MSE) of estimator ˆ𝜎2

Figure 2.6 Histograms of√ 𝑛(ˆ𝜎2

1𝛼 2𝜈 1,𝑛 −𝜎2𝛼 2𝜈 ) for different 𝑛 and tapering range 𝛾 𝑛 = 𝐶𝑛 −0.03 with 𝑑 = 1, 𝜎 = 1, 𝛼 = 5, 𝛼1 = 7.5, 𝜈 = 1/2 86

Figure 2.7 Histograms of√ 𝑛(ˆ𝜎2

1𝛼 2𝜈 1,𝑛 −𝜎2𝛼 2𝜈 ) for different 𝑛 and tapering range 𝛾 𝑛 = 𝐶𝑛 −0.02 with 𝑑 = 2, 𝜎 = 1, 𝛼 = 5, 𝛼1 = 7.5, 𝜈 = 1/4 87

Figure 2.8 Histograms of√ 𝑛(ˆ𝜎2

1𝛼 2𝜈 1,𝑛 −𝜎2𝛼 2𝜈 ) for different 𝑛 and tapering range 𝛾 𝑛 = 𝐶𝑛 −0.02 with 𝑑 = 2, 𝜎 = 1, 𝛼 = 5, 𝛼1 = 7.5, 𝜈 = 1/8 88

Figure 2.9 Histograms of √ 𝑛(ˆ𝜎2

1,𝑛 𝛼 2𝜈

1 − 𝜎2𝛼 2𝜈) for different taper with

𝑑 = 1, 𝜎 = 1, 𝛼 = 0.8, 𝛼1 = 2, 𝜈 = 1/2 88

LIST Of NOTATIONS

∣ ⋅ ∣ the Euclidean norm i.e ∣x∣ = (𝑥2

≲ for real functions 𝑎(𝑥) and 𝑏(𝑥), 𝑎(𝑥) ≲ 𝑏(𝑥) means

that there exists a constant 0 < 𝐶 < ∞ such that

∣𝑎(𝑥)∣ ≤ 𝐶∣𝑏(𝑥)∣ for all possible 𝑥

that there exist constants 0 < 𝐶1 < 𝐶2 < ∞ such

that 𝐶1∣𝑏(𝑥)∣ ≤ ∣𝑎(𝑥)∣ ≤ 𝐶2∣𝑏(𝑥)∣ for all possible 𝑥

CHAPTER 1

Introduction

Computer modeling is having a profound effect on scientific research In terministic computer experiments, unlike physical experiments, no random errorexists, that is, repeating an experiment using the same set of inputs gives the same

de-response Suppose 𝑆 ⊂ ℝ 𝑑is the set of all possible computer inputs, usually called

design space Let t ∈ 𝑆 denote the 𝑑-dimensional vector of input values and 𝑌 (t)

denote the deterministic response when an experiment is run at t It has become

common practice to approach the deterministic response 𝑌 (t) as a realization of a

spatial process In this regard, Sacks, Welch, Mitchell and Wynn (1989) modeled

𝑌 (t) as a realization of a Gaussian spatial process (random field), that includes a

regression model, called Kriging model,

where 𝑓 𝑗 (t)’s are known functions, 𝛽 𝑗 ’s are unknown coefficients and 𝑋(t) is

as-sumed to be Gaussian random field with mean zero and covariance

co-There are two common asymptotic frameworks in spatial statistics, ing domain asymptotics and fixed domain asymptotics or called infill asymptotics(Cressie, 1993) In increasing domain asymptotics, the distance between neighbor-ing observations is bounded away from zero so that the observation region grows

increas-as the number of observation increincreas-ases In fixed domain increas-asymptotics, the ber of observations increases in a given fixed and bounded domain so that the

num-observations will be increasingly dense Not surprisingly, the asymptotic behavior

of covariance parameter estimators can be quite different under these two works For example, some covariance parameters are not consistently estimatedunder fixed domain asymptotics (Ying, 1991; Zhang, 2004), whereas these sameparameters can be consistently estimated and their maximum likelihood estimatorsare asymptotically normal, subject to some regularity conditions, under increasingdomain asymptotics (Mardia and Marshall, 1984)

frame-We are interested in processes on a given fixed region of ℝ𝑑 and focus on fixeddomain asymptotics In the next two sections, we will describe some asymptoticproperties under fixed domain asymptotics for two classes of Gaussian processeswith the following covariance functions: Mat´ern class and powered exponentialclass respectively

1.1 Mat´ern Class

There are two kinds of Mat´ern class covariance function We first introduce

Gaussian process 𝑋(t), t ∈ 𝑆 = [0, 1] 𝑑 with mean zero and isotropic Mat´ern classcovariance function, which was first proposed by Mat´ern in 1960 This covariance

function is defined by

𝐶𝑜𝑣(𝑋(x), 𝑋(y)) = 𝜎2(𝛼∣x − y∣) Γ(𝜈)2 𝜈−1 𝜈 K 𝜈 (𝛼∣x − y∣), (1.1)

∀ x, y ∈ ℝ 𝑑 , where 𝜈 is a known strictly positive constant and 𝜎, 𝛼 are unknown positive parameters and K 𝜈 is the modified Bessel function of order 𝜈 [see Andrews

et al (1999), p.223] The larger 𝜈 is, the smoother 𝑋 is In particular, 𝑋 will be

𝑚 times mean square differentiable if and only if 𝜈 > 𝑚.

Results for parameter estimation under fixed domain asymptotics are difficult

to derive in general and little work has been done in this area Stein (1999) stronglyrecommended using Gaussian random field with isotropic Mat´ern class covariancefunction as a sensible class of models He investigated the performance of maximumlikelihood estimators for the parameters of a periodic version of the Mat´ern modelwith the hope that the large sample results for this periodic model would be similar

to those for non-periodic Mat´ern-type Gaussian random fields under fixed domainasymptotics

Zhang(2004) proved some important results about isotropic Mat´ern class He

first pointed out that the induced two mean zero Gaussian measures with (𝜎2

asymptotics for 𝑑 ≤ 3 But 𝜎2𝛼 2𝜈 can be consistently estimated In particular,

under some conditions the estimator of 𝜎2𝛼 2𝜈 is asymptotically efficient (Zhang,

2004) Furthermore, Kaufman et al (2008) showed that for any fixed 𝛼, the estimator of 𝜎2𝛼 2𝜈 obtained by tapered maximum likelihood estimator is strongly

consistent under fixed domain asymptotics for 𝑑 ≤ 3 when the taper support is

fixed Du et al (2009) showed that such estimator is also asymptotically efficient

for 𝑑 = 1 However, it is unknown whether such estimators lose any asymptotic efficiency for 𝑑 > 1.

Another kind of Mat´ern class covariance function, called multiplicative Mat´ernclass covariance function, is defined by

By the definitions, we can see that when 𝑑 = 1, multiplicative Mat´ern class

and isotropic Mat´ern class are the same Thus for multiplicative Mat´ern class, it

follows from Zhang’s (2004) result that 𝜎2 and 𝜃 cannot be consistently estimated respectively when 𝑑 = 1 However, the structure of multiplicative Mat´ern class

for one dimension may be quite different from that for high dimension When

𝑑 ≥ 2, Loh (2005) showed that for 𝜈 = 3/2 the maximum likelihood estimator

(MLE) of (𝜎2, 𝜃1, ⋅ ⋅ ⋅ , 𝜃 𝑑) is consistent using observations on a regular grid under

mild conditions This result implies that 𝜎2 and 𝜃1, ⋅ ⋅ ⋅ , 𝜃 𝑑 can be identified for

multiplicative Mat´ern class when 𝑑 ≥ 2 and 𝜈 = 3/2 That also gives an example

to show that the structure of multiplicative Mat´ern class may be quite different

from that of isotropic Mat´ern class for 𝑑 ≥ 2 Thus we cannot directly apply the

results for isotropic Mat´ern class under fixed domain asymptotics to multiplicative

Mat´ern class when 𝑑 ≥ 2.

1.2 Powered Exponential Class

In this section, we introduce Gaussian random field 𝑋(t), t ∈ [0, 1] 𝑑 withmultiplicative powered exponential class covariance function

𝐶𝑜𝑣(𝑋(t), 𝑋(s)) = 𝜎2∏𝑑

𝑖=1

exp{−𝜃 𝑖 ∣𝑡 𝑖 − 𝑠 𝑖 ∣ 𝛾 }, (1.3)

where t = (𝑡1, ⋅ ⋅ ⋅ , 𝑡 𝑑)𝑇 , s = (𝑠1, ⋅ ⋅ ⋅ , 𝑠 𝑑)𝑇 ∈ [0, 1] 𝑑 and 𝛾 ∈ (0, 2], 𝜃1, ⋅ ⋅ ⋅ , 𝜃 𝑑 and

𝜎2 are strictly positive parameters The parameter 𝜎2 is the variance of the process

and the scale parameters 𝜃1, ⋅ ⋅ ⋅ , 𝜃 𝑑control how quickly the correlation decays with

distance Here 𝛾 indicates smoothness of the process For example, 𝛾 = 1 implies

that the process is continuous but nowhere differentiable, while 𝛾 = 2 implies that

the process is infinitely differentiable in mean square sense

For 𝛾 = 1 and 𝑑 = 1, the process known as the Ornstein-Uhlenbeck process has Markovian properties that for 𝑡 > 𝑠, 𝑋(𝑡) − 𝑒 −𝜃∣𝑡−𝑠∣ 𝑋(𝑠) is independent of 𝑋(𝑢), 𝑢 ≤ 𝑠 Another interesting fact for 𝑋(𝑡) is that if 𝜎2𝜃 = ˜𝜎2˜𝜃, the in- duced Gaussian measures with (𝜎2, 𝜃) and (˜𝜎2, ˜𝜃) are equivalent [cf Ibraginov and

Rozanov (1978)] Thus (𝜎2, 𝜃) and (˜𝜎2, ˜𝜃) are not distinguishable with certainty

from the sample path 𝑋(𝑡), that is, it is impossible to estimate consistently both

𝜎2 and 𝜃 simultaneously However, Ying (1991) showed that the product ˆ𝜎2ˆ𝜃 of the maximum likelihood estimators ˆ𝜎2 and ˆ𝜃, as an estimator of 𝜎2𝜃, is strongly

consistent and asymptotically normal,

ˆ𝜎2ˆ𝜃 −→ 𝜎a.s 2𝜃, √ 𝑛(ˆ𝜎2ˆ𝜃− 𝜎2𝜃) −→ 𝑁(0, 2(𝜎 𝑑 2𝜃)2).

As we mentioned before, for 𝑑 = 1, 𝜎2 and 𝜃 are not identifiable However, for

𝑑 ≥ 2, under mild conditions Ying (1993) showed that the MLE (ˆ𝜎2, ˆ𝜃1, ⋅ ⋅ ⋅ , ˆ𝜃 𝑑) of

(𝜎2, 𝜃1, ⋅ ⋅ ⋅ , 𝜃 𝑑) is strongly consistent, i.e

(ˆ𝜎2, ˆ𝜃1, ⋅ ⋅ ⋅ , ˆ𝜃 𝑑)−→ (𝜎a.s. 2, 𝜃1, ⋅ ⋅ ⋅ , 𝜃 𝑑 ),

√ 𝑛(ˆ𝜎2∏𝑑

This implies that for 𝑑 ≥ 2, (1.3) gives quite a different structure.

Properties of MLE in other cases were also investigated by some researchers For

𝛾 = 1, 𝑑 = 2, van der Vaart (1996) showed that the maximum likelihood estimators

are also asymptotically efficient For 𝛾 = 2, Loh and Lam (2000) showed that sieve maximum likelihood estimators of 𝜃1, ⋅ ⋅ ⋅ , 𝜃 𝑑 are strongly consistent using a

regular sampling grid Unfortunately, when 𝛾 is not an integer, it is rather difficult

to handle likelihood functions analytically, because the covariance matrix of the

data is 𝑛 × 𝑛 and the computations of maximum likelihood can be formidable

as the sample size 𝑛 increases Also, Mardia and Marshall’s (1984) asymptotic

consistency of MLE is not applicable when the observations are taken from abounded region As a result, properties of MLE under fixed domain asymptoticsare not well understood

For covariance parameters, the preferred estimator is MLE However when the

sample 𝑛 is large, it is a burden to compute the likelihood The first objective of this

thesis is to use covariance tapering to approximate the covariance function so thatthe computation can be reduced Chapter 2 studies the fixed domain asymptotic

behavior of the tapered MLE for the parameter 𝜎2𝛼 2𝜈 of isotropic Mat´ern class

covariance function when the taper support is allowed to shrink as 𝑛 → ∞ The

conditions in Kaufman et al (2008) are difficult to check in practice In Chapter

2, conditions will be established in which the tapered MLE is strongly consistent

and asymptotically normal for 1 ≤ 𝑑 ≤ 3.

The second objective of this thesis is to investigate the asymptotic properties

of Gaussian random field with multiplicative covariance function under fixed main asymptotics We will focus on whether the variance and scale parameters

do-in multiplicative covariance function can be consistently estimated In Chapter 3,

we will use quadratic variation and spectral analysis to construct consistent timators As we mentioned before, the variance and scale parameters cannot beconsistently estimated respectively for one dimension But the product of thesetwo parameters can be consistently estimated The quadratic variation will give

es-a consistent estimes-ator of this product for 𝑑 = 1 using the observes-ations on es-a grid For 𝑑 > 1, spectral analysis may provide another consistent estimator for product

of variance and all scale parameters Then these consistent estimators can provide

the consistent estimators of variance and all scale parameters for 𝑑 > 1.

This thesis mainly concerns the consistent estimators of unknown parameters

It provides closed forms of the estimators Compared with MLE, these kinds ofestimator are easier to compute and may retain most of the information However,all these asymptotic results base on an essential assumption that the smoothness

parameter is known for technical issue Some researchers had studied the tion of smoothness parameter Stein (1993) studied the estimation of smoothnessparameter for a class of periodic Gaussian processes in one dimension Constantineand Hall (1994) studied the estimation of smoothness parameter in one dimensionunder a sort of mixture of increasing domain and fixed domain asymptotics How-ever, the estimation of smoothness parameter is not the subject to this study.

estima-CHAPTER 2

Isotropic Covariance Function

2.1 Introduction

Let 𝑋 : ℝ 𝑑 → ℝ be a mean-zero isotropic Gaussian random field with the

Mat´ern covariance function

𝐶𝑜𝑣(𝑋(x), 𝑋(y)) = 𝜎2𝐾 𝛼 (x − y)

= 𝜎2(𝛼∣x − y∣) Γ(𝜈)2 𝜈−1 𝜈 K 𝜈 (𝛼∣x − y∣), ∀ x, y ∈ ℝ 𝑑 , (2.1)

where 𝜈 > 0 is a known constant, 𝛼, 𝜎 are strictly positive but unknown parameters and K 𝜈 is the modified Bessel function of order 𝜈 [see Andrews et al.(1999), p.223].

It is well known that 𝑋 will be 𝑚 times mean square differentiable if and only if

𝜈 > 𝑚 We refer the reader to Stein (1999) for a comprehensive of Mat´ern-type

Gaussian random fields

We are concerned with the estimation of the (microergodic) parameter 𝜎2𝛼 2𝜈

using observations

{

𝑋(x1), 𝑋(x2) ⋅ ⋅ ⋅ , 𝑋(x 𝑛)},

where x1, ⋅ ⋅ ⋅ , x 𝑛 are distinct points in [0, 𝑇 ] 𝑑 for some (absolute) constant 0 <

𝑇 < ∞ Zhang (2004) observed it is this microergodic parameter and not the

individual parameters that matters more to interpolation We refer the reader toStein (1999), page 163, for the mathematical definition of microergodicity Forsimplicity, we write

X𝑛 = (𝑋(x1), 𝑋(x2) ⋅ ⋅ ⋅ , 𝑋(x 𝑛))𝑇

The covariance matrix of X𝑛 can be express as 𝜎2𝑅 𝛼 , where 𝑅 𝛼 is a 𝑛×𝑛 correlation matrix whose (𝑖, 𝑗)th is element 𝐾 𝛼(x𝑖 − x 𝑗 ) not depending on 𝜎 Since X 𝑛 ∼

𝑁 𝑛 (0, 𝜎2𝑅 𝛼 ), the log-likelihood 𝑙 𝑛 (𝛼, 𝜎) satisfies

𝑙 𝑛 (𝛼, 𝜎) = − 𝑛2log(2𝜋) − 𝑛 log(𝜎) − 12log(∣𝑅 𝛼 ∣) − 2𝜎12X𝑇

𝛼 X𝑛 (2.2)

It is generally acknowledged [e.g., Stein et al (2004), Furrer et.al (2006),Kaufman et al (2008) and Du et al (2009)] that in practice, the data set isusually very large and is irregularly spaced Computing the inverse covariance

where ˜𝐾 𝛼1,𝑛 : ℝ𝑑 → ℝ is a known isotropic correlation function, 𝛼1 > 0 is a

known constant and 𝜎2

1 = 𝜎2𝛼 2𝜈 /𝛼 2𝜈

1 ˜𝐾 𝛼1,𝑛 is allowed to, possibly, vary with

sample size 𝑛 Under (2.3), let 𝜎2

1𝑅˜𝛼1,𝑛 be the covariance matrix of X𝑛 and hence

X𝑛 ∼ 𝑁 𝑛 (0, 𝜎2

1𝑅˜𝛼

1,𝑛 ) The corresponding log-likelihood function ˜𝑙 𝑛 (𝛼1, 𝜎1) satisfies

˜𝑙 𝑛 (𝛼1, 𝜎1) = − 𝑛2log(2𝜋) − 𝑛 log(𝜎1) −12log(∣ ˜ 𝑅 𝛼1,𝑛 ∣) − 2𝜎12

For example, Zhang (2004) took ˜𝑅 𝛼1,𝑛 = 𝑅 𝛼1 where 𝛼1 > 0 is a known (arbitrarily

specified) constant This made the likelihood analysis simpler because (2.4) is a

function of only 𝜎1 Zhang (2004) proved that for 𝑑 ≤ 3, ˆ𝜎2

result is unlikely to hold if 𝑑 > 4 as Anderes (2010) recently proved in the latter case that the Gaussian measures defined by 𝜎2𝐾 𝛼 and 𝜎2

1𝐾 𝛼1 are orthogonal unless

(𝛼1, 𝜎1) = (𝛼, 𝜎) The case 𝑑 = 4 is still open.

Covariance tapering is an attractive method of constructing 𝜎2

1𝐾˜𝛼

1,𝑛 such that

it is an isotropic, positive definite and compactly supported function A way to

implement covariance tapering is as follows Let 𝐾tap : ℝ𝑑 → ℝ be an isotropic

correlation function with compact support, say, supp(𝐾tap) ⊆ [−1, 1] 𝑑 Define

𝐾 𝛼1,𝑛 is as in (2.3) and 𝐾 𝛼1 is as in (2.1) with 𝛼 replace by 𝛼1 (a known constant)

The motivation is that the covariance matrix 𝜎2

1𝑅˜𝛼1,𝑛 of X𝑛 corresponding to ˜𝐾 𝛼1,𝑛

is sparse (with many off-diagonal elements taking the value 0) and sparse matrixalgorithms are available to evaluate the log-likelihood (2.4) more efficiently [cf Pis-sanetsky (1984)] Isotropic, positive definite, compactly supported functions havebeen an intensively studied field The literature includes Wu (1995), Wendland(1995), (1998) and Gneiting (2002)

Assuming 𝛾 𝑛 ≡ 𝛾 is an absolute constant (independent of 𝑛), Kaufman, et al.

(2008) established conditions on the spectral density of 𝐾tap such that ˆ𝜎2

1,𝑛 𝛼 2𝜈

𝜎2𝛼 2𝜈 with 𝑃 𝛼,𝜎 probability 1 As in Zhang (2004), the theory of the equivalence

of Gaussian measures is used in a crucial manner

In the case 𝑑 = 1 and 𝛾 𝑛 ≡ 𝛾, Du, et al (2009) established conditions on

the spectral density of 𝐾tap such that √ 𝑛(ˆ𝜎2

𝑁(0, 2(𝜎2𝛼 2𝜈)2) as 𝑛 → ∞ under Gaussian measure 𝑃 𝛼,𝜎 As open problems, Du,

et al (2009) observed that their techniques cannot be extended from 𝑑 = 1 to

𝑑 = 2 or 3, and it would be practically important to obtain analogous asymptotic

normality results for higher dimensions They further noted that letting 𝛾 𝑛 → 0

as 𝑛 → ∞ is a natural scheme in the fixed-domain asymptotic framework and remarked that it is not obvious that their proofs can be adapted to varying 𝛾 𝑛

This chapter is organized as follows In Section 2.2, Theorem 2.1 generalizes

the strong consistency result of Kaufman, et al (2008) from 𝛾 𝑛 ≡ 𝛾 to a sequence

of 𝛾 𝑛 ’s which could vary with 𝑛, in particular where 𝛾 𝑛 → 0 as 𝑛 → ∞ It is noted

that even for covariance tapering with 𝛾 𝑛 ≡ 𝛾, the number of operations needed to

compute the inverse covariance matrix is still 𝑂(𝑛3) whereas if 𝛾 𝑛 → 0, the number

of operations is 𝑜(𝑛3) Clearly the latter will lessen the computational burden ofevaluating the likelihood and inverting the covariance matrix even more Theorem

2.2 extends the asymptotic normality results of Du, et al (2009) from 𝑑 = 1 and

𝛾 𝑛 ≡ 𝛾 to 1 ≤ 𝑑 ≤ 3 and 𝛾 𝑛 possibly varying with 𝑛 Theorem 2.3 deals with the case where the Mat´ern covariance function 𝜎2𝐾 𝛼 is mis-specified by another

Mat´ern covariance function 𝜎2

1𝐾 𝛼1 with 𝛼1 a known constant

Section 2.3 proves a number of Bernstein-type probability inequalities Theseinequalities are needed in the proof of Theorem 2.1 Section 2.4 is heavily motivated

by the equivalence of Gaussian measures theory (when 𝑑 = 1) as detailed in Chapter

3 of Ibragimov and Rozanov (1978) However in the case that 𝛾 𝑛 → 0 as 𝑛 → ∞,

the Gaussian measures in Theorems 2.1 and 2.2 are not equivalent (as 𝑛 → ∞).

As such, the results of Ibragimov and Rozanov (1978) have to be modified toaccommodate this fact The main result of Section 2.3 is (2.28) which is needed inthe proofs of Theorems 2.1 and 2.2

Lemma 2.5 in Section 2.5 establishes some bounds on the spectral density of

a tapered covariance function The proof of Lemma 2.5 is a slight refinement (in

order to accommodate a varying 𝛾 𝑛) of that found in Kaufman, et al (2008) Theproofs of Theorems 2.1, 2.2 and 2.3 are given in Section 2.6 Finally, Section 2.7provides simulation studies showing that how well the results are applied to finitesamples

We end this Introduction with a brief on notation ℝ and ℂ denote the sets of

real and complex numbers respectively i = √ −1, 𝐼{⋅} is the indicator function

and ∣x∣max= max1≤𝑗≤𝑑 ∣𝑥 𝑗 ∣, ∀ x = (𝑥1, ⋅ ⋅ ⋅ , 𝑥 𝑑)𝑇 ∈ ℝ 𝑑 If M is a vector or matrix, then M𝑇 is its transpose Finally, 𝑓(𝑥) ≍ 𝑔(𝑥) means that there exist constants 𝑐1and 𝑐2 such that 0 < 𝑐1 < 𝑐2 < ∞ and 𝑐1∣𝑏(𝑥)∣ ≤ ∣𝑎(𝑥)∣ ≤ 𝑐2∣𝑏(𝑥)∣ for all possible 𝑥.

2.2 Main Results

This section describes the main results of this chapter

Theorem 2.1 Let 0 < 𝑇 < ∞, 1 ≤ 𝑑 ≤ 3 and 𝜎2𝐾 𝛼 be the Mat´ern covariance function as in (2.1) Let 𝜖, 𝑀 be constants such that 𝜖 > max{𝑑/4, 1−𝜈} Suppose

𝐾 tap is an isotropic correlation function with supp(𝐾 tap ) ⊆ [−1, 1] 𝑑 whose spectral density

Let 𝛼1 > 0 and 𝜎1 > 0 be constants such that 𝜎2

𝜎2𝛼 2𝜈 and 2𝑏(2𝜈 + 2𝜖 + 𝑑)/ min{2, 4 − 𝑑, 4𝜖 − 𝑑, 4𝜈 + 𝑑} < (1 − 2𝑏𝑑)/(2𝑑) Define

1,𝑛 𝛼 2𝜈

1 − 𝜎2𝛼 2𝜈)−→ 𝑁(0, 2(𝜎 𝑑 2𝛼 2𝜈)2),

as 𝑛 → ∞ with respect to 𝑃 𝛼,𝜎 , the Gaussian measure defined by the covariance function 𝜎2𝐾 𝛼 in (2.1).

Remark 2.2 For 𝑏 = 0 or equivalently, 𝛾 𝑛 ≡ 𝛾 and 𝑑 = 1, Theorem 2.2 proves

the asymptotic normality of ˆ𝜎2

1,𝑛 under weaker conditions than Theorem 5(ii) of

Du, et al (2009)

Theorem 2.3 Let 0 < 𝑇 < ∞, 1 ≤ 𝑑 ≤ 3 and 𝜎2𝐾 𝛼 be the Mat´ern covariance function as in (2.1) Let 𝛼1 > 0 and 𝜎1 > 0 be constants such that 𝜎2

1𝛼 2𝜈 = 𝜎2𝛼 2𝜈 Define ˜ 𝐾 𝛼1,𝑛 (x) = 𝐾 𝛼1(x), ∀ x ∈ ℝ 𝑑 , and ˜ 𝑅 𝛼1,𝑛 = 𝑅 𝛼1 Let ˆ𝜎2

1,𝑛 be as in (2.5) Then

√ 𝑛(ˆ𝜎2

1,𝑛 𝛼 2𝜈

1 − 𝜎2𝛼 2𝜈)−→ 𝑁(0, 2(𝜎 𝑑 2𝛼 2𝜈)2),

as 𝑛 → ∞ with respect to 𝑃 𝛼,𝜎 , the Gaussian measure defined by the covariance function 𝜎2𝐾 𝛼 in (2.1).

Remark 2.3 For 𝑑 = 1, Theorem 2.3 reduces to Theorem 5(i) of Du, et al.

(2009) In the case 𝜈 = 1/2, i.e the Ornstein-Uhlenbeck process on [0, 𝑇 ], Ying

(1991) proved the strong consistency and asymptotic normality of the MLE for

𝜎2𝛼 while Du, et al (2009) obtained similar results for tapered MLE (obtained by

maximizing (2.4) with respect to both 𝛼1 and 𝜎1)

2.3 Some Probability Inequalities

This section proves a number of probability inequalities that are need in the

sequel Let 𝛼1, X 𝑛 and ˆ𝜎 1,𝑛 be defined as in (2.5) Define 𝜎2

1 = 𝜎2𝛼 2𝜈 /𝛼 2𝜈

1 Let

A = {∣ˆ𝜎2

1,𝑛 − 𝜎2𝛼 2𝜈 /𝛼 2𝜈

1 ∣ > 𝜀} for some constant 𝜀 > 0 and B ⊆ ℝ 𝑑 such that

A = {X 𝑛 ∈ B} For simplicity, we write 𝑃 𝛼,𝜎 and 𝑝 𝛼,𝜎 to denote probability and

probability density function of X𝑛 when (2.1) holds with parameters 𝛼, 𝜎, and

˜

𝑃 𝛼1,𝜎1,𝑛 and ˜𝑝 𝛼1,𝜎1,𝑛 to denote probability and probability density function of X𝑛

defined by the covariance function 𝜎2

1𝐾˜𝛼1,𝑛 in (2.3) Then for any constant 𝜏 𝑛 > 0

(which may depend on 𝑛), we have

𝑃 𝛼,𝜎 (A ) =

∫

B 𝑝 𝛼,𝜎 (x)𝑑x

Next we observe that there exists a 𝑛 × 𝑛 non-singular matrix 𝑈 such that

Lemma 2.2 With the notation of (2.8), suppose 𝜏 𝑛 > 0, 0 < 𝑐 𝑛 < 1, 𝐶 𝑛 and ˜ 𝐶 𝑛

are constants (which may depend on 𝑛) such that for 𝑛 = 1, 2, ⋅ ⋅ ⋅ ,

Proof We observe that

where 𝐸 𝛼,𝜎 denotes expectation with respect to the probability measure 𝑃 𝛼,𝜎 The

right hand side of the last equality is a minimum when 𝜆 𝑖,𝑛 = 1 for all 𝑖 = 1, ⋅ ⋅ ⋅ , 𝑛.

We further observe from (2.8) and (2.9) that

𝑖 , 1}) 𝑘]

Consequently it follows from Bernstein’s inequality [cf (7) of Bennet (1962)] that

𝑃 𝛼,𝜎(∑𝑛 𝑖=1

(𝜆 −1 𝑖,𝑛 − 1)[𝑌2

𝑖 )] > 𝜀

vuu

2.4 Spectral Analysis

This section is motivated by the equivalence of Gaussian measures theory asdeveloped in Chapter 3 of Ibragimov and Rozanov (1978) However, these ideashave to be modified because the Gaussian measures considered in Theorems 2.1

and 2.2 are not equivalent if 𝛾 𝑛 → 0 as 𝑛 → ∞.

Let 𝑑 ≤ 3 and observations be 𝑋(x1), ⋅ ⋅ ⋅ , 𝑋(x 𝑛), with x1, ⋅ ⋅ ⋅ , x 𝑛 ∈ [0, 𝑇 ] 𝑑

Define 𝜑 𝑘 (w) = 𝑒iw𝑇x𝑘 , ∀ w ∈ ℝ 𝑑 , 𝑘 = 1, ⋅ ⋅ ⋅ , 𝑛, where i = √ −1 Let 𝐿0

where 𝑐1, ⋅ ⋅ ⋅ , 𝑐 𝑑 are real-value constants, and 𝑓 𝛼,𝜎 be the spectral density defined

by the covariance function 𝜎2𝐾 𝛼 as in (2.1) That is