Multistep yule walker estimation of autoregressive models

The aim of this work is to ﬁt a ”wrong” model to an observed time series by employing higher order Yule-Walker equations in order to enhance the ﬁttingaccuracy.. Several parameter estima

Trang 1

OF AUTOREGRESSIVE MODELS

YOU TINGYAN

NATIONAL UNIVERSITY OF SINGAPORE

2010

Trang 2

OF AUTOREGRESSIVE MODELS

YOU TINGYAN

(B.Sc Nanjing Normal University)

A THESIS SUBMITTED FOR THE DEGREE OF MASTER OF SCIENCE

DEPARTMENT OF STATISTICS AND APPLIED PROBABILITY

NATIONAL UNIVERSITY OF SINGAPORE

2010

Trang 3

It is a pleasure to convey my gratitude to those who made this thesis possible all

in my humble acknowledgment In the ﬁrst place I am heartily thankful to my pervisor, Prof Xia Yingcun, whose encouragement, supervision and support fromthe preliminary to the concluding level enabled me to develop an understanding ofthe subject His supervision, advice, and guidance from the very early stage of thisresearch as well as giving me extraordinary experiences through the work are thecritical support to the completeness of this thesis Above all and the most needed,

su-he provided me sustaining encouragement and support in various ways His trulyscientist intuition has made him as a constant oasis of ideas and passions in science,which exceptionally inspire and enrich my growth as a student, a researcher and

a scientist want to be I am gratefully appreciating him more than he knows Ialso would like to record my gratitude to my classmates and seniors, Jiang Binyan,Jiang Qian, Liangxuehua, Zhu Yongting, Yu Xiaojiang, Jiang Xiaojun, for theirinvolvement with my research It’s so kind of them all always kindly to grant metheir time even for answering some of my unintelligent questions about time series

Trang 4

and estimation methods Many thanks go in particular to Fu Jingyu, who used herprecious time to read this thesis and gave her critical and constructive commentsabout it.

Lastly, I offer my regards and blessings to staffs in the general office of ment, and all of those who supported me in any respect during the completion ofthe project

Trang 5

1.1 Introduction 1

1.2 AR model and its estimation 2

1.3 Organization of this Thesis 4

2 Literature Review 6 2.1 Univariate Time Series Background 6

2.2 Time series Models 10

2.3 Autoregressive (AR) Model 12

2.4 AR model Properties 14

Trang 6

2.4.1 Stationarity 14

2.4.2 ACF and PACF for AR Model 15

2.5 Basic Methods for Parameter Estimation 18

2.5.1 Maximum Likelihood Estimation (MLE) 18

2.5.2 Least Square Estimation Method (LS) 20

2.5.3 Yule-Walk Method (YW) 23

2.5.4 Burg’s Estimation Method (B) 25

2.6 Monte Carlo Simulation 26

3 Multistep Yule-Walker Estimation Method 29 3.1 Multistep Yule-Walker Estimation (MYW) 30

3.2 Bias of YW method on Finite Samples 32

3.3 Theoretical Support of MYW 33

4 Simulation Results 36 4.1 Comparisons for Estimation Accuracy for AR (2) model 36

4.1.1 Percentage for Outperformance of MYW 36

4.1.2 Diﬀerence between the SSE of ACFs for YW and MWY methods 40

4.1.3 The Eﬀect of Diﬀerent Forward Step m 42

4.2 Estimation Accuracy for Fractional ARIMA Model 45

Trang 7

5.1 Data Source 525.2 Numerical Results 53

Trang 8

The aim of this work is to ﬁt a ”wrong” model to an observed time series

by employing higher order Yule-Walker equations in order to enhance the ﬁttingaccuracy Several parameter estimation methods for autoregressive models werereviewed, such as Maximum Likelihood method, Least Square method, Yule-Walkermethod, Burg’s method, etc Comparison of the estimation accuracy between thewell-known Yule-Walker method and our new multistep Yule-Walker method based

on the autocorrelation function (ACF) is made The eﬀect of diﬀerent number ofYule-Walker equations on the estimation performance is investigated Monte Carloanalysis and real data are used to check the performance of the proposed method

Keywords: Time series, Autoregressive Model, Least Square method, Yule-Walker

Method, ACF

Trang 9

List of Tables

4.1 Detailed Percentage for a Better Performance of MYW method 394.2 List of ”best” m for MYW method 44

Trang 10

List of Figures

4.1 Percentage for outpermance of MYW out of 1000 simulation

itera-tions for n=200, 500, 1000 and 2000 38

4.3 SSE of ACF for both method and its diﬀerence with n=200 41

4.6 SSE of ACF for MYW method with n=200, 500, 1000 and 2000 43

4.7 Diﬀerence of SSE of ACF with n=200, 500, 1000 and 2000 for p=2, d=0.2 47

4.9 Diﬀerence of SSE of ACF for n=500 with p=1 48

Trang 11

5.2 Diﬀerence between SSE of ACF for two methods with p=1 54

5.3 SSE of ACF for MYW method with p=1 54

Trang 13

ap-estimate the parameters of the ARMA model fall into two classes One is toconstruct a likelihood function and derive the parameters by maximizing it usingsome iterative nonlinear optimization procedure The other class of technique getsthe parameter in two steps: ﬁrstly obtain the coeﬃcients of autoregressive (AR)parameters, then derive the spectral parameters in moving-average (MA) part sub-sequently In the scope of our work, focus will be put on the method for parameterestimation for AR parameters After reviewing several commonly used AR modelparameter estimation methods, a new multistep Yule-Walker estimation method

is introduced which increases the equation number in the Yule-Walker method toenhance the ﬁtting accuracy The criteria used to compare the performance of themethods is the ACFs matching between model generated series and original series,which was detailed introduced by Xia and H.Tong( 2010)

Various models have been developed to mimic the observed time series ever, it is said that to some extend all the models are wrong due to certain reasons

How-No model could exactly reflects the observed series and inaccuracy is always isting for the postulated model The only effort we could make is to find a modelwhich can capture the characteristic of the series to the maximum extend and to

ex-ﬁt the ”wrong” model with a parameter estimation method which can reduce theestimation bias eﬀectively Our work will be focusing on the AR models and its

Trang 14

estimation methods in order to evaluate the performance of different parameterestimation methods for fitting the AR model The autoregressive (AR) model,which was developed by Box and Jenkins in 1970, represents a linear regressionrelationship of the current value of series against one or more past values of theseries Early in the mid seventies, autoregressive modeling was first introduced inthe nuclear engineering and widely used in other industries soon after Nowadays,autoregressive modeling is a popular means for identifying, monitoring, malfunc-tioning detecting and diagnosing system performance An autoregressive modeldepends on a limited number of parameters, which are estimated from time seriesdata There are a lot of techniques exist for computing AR coefficients, amongwhich the main two categories are Least Squares and Burg’s method We couldfind a wide range of supported techniques in MatLab for these methods Whenusing the various algorithms from different sources, there are two points to be paidattention to One is to check whether or not the series has already been takenout the mean, the other one is whether the sign of the coefficients are inverted

in the deﬁnition or assumptions Comparisons of the estimated ﬁnite-sample curacies within these methods have been made and these results provided someuseful insights into the behavior of these estimators It has already been provedthat these estimation techniques should lead to approximately the same parame-ter estimates in large data sample cases But either the Yule-Walker or the LeastSquares method is frequently used compared with other methods mostly due to

Trang 15

ac-some historical reasons Among all of the methods, the most common method is socalled Yule-Walker method which applies the least squares regression method onthe Yule-Walker equations system The basic steps to get the Yule-Walker equa-tions is ﬁrstly to derive the coeﬃcients by multiplying the AR model by its prior

values with lag n = 1, 2, · · · , p, and then to take the expectation of the multiple

values and normalize it (Box and Jenkins, 1976) However, some previous researchhas been done to show that in some occasions the Yule-Walker estimation methodleads to poor parameter estimates with large bias even for moderately sized datasamples In our study, we propose an improved method on the Yule-Walker methodwhich is to increase the equation numbers in the Yule-Walker system and try toﬁgure out whether this could help to enhance the parameter estimation accuracy.The Monte Carlo analysis will be used here to generate simulation results for thisnew method and real data will also be applied to check its performance

The outline of this work is as follows: In Chap 1, the aim and purpose of thiswork is presented and a general introduction on the approaches to the parameterestimation for autoregressive model is given In Chap 2, Literature review has beendone on the deﬁnition of univariate time series, background of time series modelclasses and properties of autoregressive model Emphasis has been given to themethods for estimating the parameters in the AR(p) model, including the Maxi-

Trang 16

mum Likelihood method, Least Square method, Yule-Walker method and Burg’smethod The Monte Carlo analysis which will be used in numerical examples in thefollowing section is also brieﬂy described In Chap 3, we will show the modiﬁcation

we proposed on the Yule-Walker method The bias of the Yule-Walker estimator

in finite sample which lead to the poor performance of the Yule-Walker method isdemonstrated Theoretical support for better estimation performance of MultistepYule-Walker method is given Simulation results of the autoregressive processes tosupport the modification are illustrated in Chap 4, while in Chap 5, we will illumi-nate our findings with the application of Multistep Yule-Walker modeling methodfor daily exchange rate of Japanese Yen for US Dollar Finally, conclusions for thiswork and some remarks for further study are presented in Chap 6

Trang 17

Chapter 2

Literature Review

Time series is a set of observations {x t } which is recorded at a speciﬁc time t

sequentially over equal time increments or continuous time If the set is of singleobservations, the series is called a univariate time series Univariate time seriescan be extended to deal with vector-valued data, which means more than oneobservations are recorded at a time This leads to the multivariate time-seriesmodels and vectors are used for the multivariate data Another extensions is theforcing time series, on which the observed series may not have a causal eﬀect Thediﬀerence between the multivariate series and the forcing series is that we couldcontrol the forcing series under experience design, which means it is deterministic,while the multivariate series is totally stochastic We will only cover the univariatetime series in this thesis, so hereinafter univariate time series is simply be put as

Trang 18

time series Time series can be either discrete or continuous A discrete-time timeseries is one in which the time for observation recording are is a discrete set, forexample, when observations are recorded at fixed time intervals Continuous-timetime series are obtained when time set recording the observations are continuous.Time series have been widely used in a wide range of areas It arise when mon-itoring engineering processes, recording stock price in financial market or trackingcorporate business metrics, etc Due to the fact that data points taken over timemay have an internal structure, such as autocorrelation, trend or seasonal vari-ation, time series analysis has been developed to accounted for these issues andinvestigate the information behind the series For example, in the financial in-dustry, time series analysis is used to observe the price changing trends on stock,bond, or other financial asset over time; it can also be used to compare the change

of these ﬁnancial variables with other comparable variables within the same timeperiod To be more speciﬁc, if you wanted to analyze how the daily closing stockprices for a given stock over a period of one year change, a list of all the closingprices for the stock over each day for the year should be obtained and recorded

in chronological order as a time series with daily interval and a one-year period.There are a number of approaches to modeling time series, from the simplest model

to more complicated model which take trend and seasonal and residual eﬀect intoaccount Decompositions is one approach is to decompose the time series into atrend, seasonal, and residual component Another approach, is to analyze the se-

Trang 19

ries in the frequency domain, which is the common method used in scientiﬁc andengineering applications We will not cover the complicated models in this workand only outline a few of the most common approaches below.

The simplest model for a time series is one in which there is no trends or seasonalcomponent The observations are simply independent and identically distributed

(i.i.d.) random variables with zero mean, which is referred as X1, X2, · · · We

deﬁne the series of random variables X t as time series if for any positive integer n

and real number x1, x2, · · · , x n,

where F (.) is the cumulative distribution function of the i.i.d random variables

observations Specially, for all h >> 1 and all x, x1, · · · , x n, if

we can say that X1, , X n contain no useful information when forecasting the

possible behavior of X n+h The function that minimizes the mean square error

property makes the i.i.d series quite uninteresting and limits its use for forecasting.However, it plays a very critical part as a building block for more complex timeseries models In other time series, trend is clear in the data pattern, thus, the zeromean model is no longer suitable for these cases So, we have the following model:

Trang 20

The model separate the time series into two parts: m t is the trend component

function which changes slowly over time and Y t is a time series with zero mean

A common assumption in many time series techniques is that the data are tionary If a time series {X t } has similar properties to those time shifted se-

sta-ries, we can loosely say that this time series is stationary To be more strict onthe properties, we focus on the ﬁrst-order and second-order moments of {X t }.

Firstly the ﬁrst-order moment of {X t } is the mean function µ x (t) = E(X t) ally we will assume {X t } be a time series with E(X2

Usu-t ) < ∞ For the

second-order moment E(X2

t), we introduce the conception of covariance The covariance

γ i = Cov(X t , X t −i) is called the lag-i autocovariance of {X t } It has two

impor-tant properties: (a) γ0 = V ar(X t ) and (b) γ −i = γ i The second property holds

because Cov(X t , X t −(−i) ) = Cov(X t −(−i) , X t ) = Cov(X t+i , X t ) = Cov(X t1, X t1−i),

where t i = t + i When normalized the autocovariance by its variance, the

auto-correlation (ACF) is obtained For a stationary process, the mean, variance andautocorrelation structure do not change over time So if we have a series of whichthe above statistical properties are constant and no periodic ﬂuctuations in seasonaltrend, we can call it stationary But stationarity have more precise mathematicaldeﬁnitions In section 2.4.1, more introduction on stationary on autoregressiveprocess will be given for our purpose

Trang 21

2.2 Time series Models

A time series model for the observed series {X t } is a speciﬁcation of the joint

distributions of the sequence of random variables {X t } Diﬀerent models for time

series data have many diﬀerent forms and represent diﬀerent stochastic processes

We have brieﬂy introduced the simplest model for a time series which are simply dependent and identically distributed (i.i.d.) random variables with zero mean andwithout trends or seasonal components Three broad classes of practical impor-tance for modeling variations of a process exist: the autoregressive (AR) models,the moving average (MA) models and the integrated (I) models Autoregressive(AR) model is a linear regression relationship of the current value of the seriesagainst one or more past values of the series We will give a detailed description

in-on autoregressive model in the following sectiin-on Moving average (MA) model is alinear regression relationship of the current value of the series against the randomshocks of one or more past values of the series The random shocks at each pointare assumed to come from the same distribution, typically a normal distributionwith zero mean and constant finite variance In the moving average model, theserandom shocks are passed to future values of the time series, which make it dis-tinct from other class of model Fitting the MA estimates is more complicatedthan fitting the AR models because the error terms in MA models are not observ-able This means that iterative non-linear fitting procedures should be used for

MA model estimation instead of linear least squares We will not go further on

Trang 22

this topic in this study.

New models, such as the autoregressive moving average (ARMA) model andautoregressive integrated moving average (ARIMA) model can be obtained if weextend the models by combining the fundamental classes together The autore-gressive moving average (ARMA) model is a combination of autoregressive (AR)model and Moving Average(MA) model The autoregressive integrated movingaverage (ARIMA) was introduced by Box and Jenkins (1976) It predicts meanvalues in a time series as a linear combination of its own past values and past errors.Autoregressive integrated moving average (ARIMA) model was advanced by Boxand Jenkins which requires long time series data Box and Jenkins introduced theconcept of seasonal non-seasonal (S-NS) ARIMA models for describing a seasonaltime series and also provided an iterative procedure for developing such models.Although seasonality violates stationarity assumption, the autoregressive fraction-ally integrated moving average (ARFIMA) model is also introduced to explicitlyincorporate the seasonality into the time series model

All these above classes represent a linearly relationship between the currentdata and previous data points In empirical situation in which more complicatedtime series are involved, linear models are not suﬃcient to cover all the information

It is also an interesting topic to consider the non-linear dependence of a series onprevious data points which generates a chaotic time series So models to repre-sent the changes of variance over time, which is also called heteroskedasticity, are

Trang 23

introduced These models are called autoregressive conditional heteroskedasticity(ARCH) and the collection of this model class has a wide variety of representations,such as GARCH, TARCH, EGARCH, FIGARCH, CGARCH, etc In the ARCHmodel class, changes in variability are related to recent past values of the observedseries Similarly the GARCH model assumes that there is correlation between atime series data and its own lagged data These ARCH model class have beenwidely used in predicting several time series data including inﬂation, stock prices,exchange rates, interest rates and for forecasting.

This study focuses on one speciﬁc type of time series model: the autoregressive(AR) model The AR(p) model was developed by Box and Jenkins in 1970 (Box,1994) As mentioned above, AR (p) model is a linear regression relationship of thecurrent value of the series against past values of the series The value of p is calledthe order of the AR model, which means that the current value is represented by

p past values in the series An autoregressive process of order p is a zero meanstationary process

To better understand the general autoregressive model, we will start from thesimplest AR(1)model:

Trang 24

For AR(1) model, conditional on the past observation, we have

From the above conditional mean and variance on the past data point X t −1, the

value of X t −1 is not correlated to the value of X t −i for i > 1 The current data point is centered around ϕ0+ ϕ1X t −1 with standard deviation σ a So, the past data

point X t −1 is not enough to determine the conditional expectation of X t, whichinspires us to take more past data points into the model to give a better indicationfor the current data point Thus a more ﬂexible and generalized model is extended

as AR(p) model satisﬁes the following equation:

where p is the order and{a t } is assumed to be a white noise series with zero mean

and constant ﬁnite variance σ a2 The representation of the AR(p) model has the

same form as the linear regression model if X t is served as the dependent variable

and lagged values X t −1 , X t −2 , , X t −p are served as the explanatory variable Thus,the autoregressive model has several properties similar to those of the simple linearregression model However there are still some diﬀerences between the two models

In this model, the past p values X t −i (i = 1, , p) jointly determine the conditional expectation of X t given the past data The coeﬃcients ϕ1, ϕ2, · · · , ϕ p are such that

Trang 25

all the roots of the polynomial equation

The foundation of time series analysis is stationarity We refer a time series

to that of (X t1+t , , X t k +t) for all t, where k is an arbitrary positive integer and

put it in a more understandable way, if the joint distribution of (X t1+t , , X t k +t)

is invariant under time shift, the time series can be recognized as strict stationary.This condition is very strong and usually used in theoretical research However,

in real world time series, it is hard to achieve Thus, we use another version ofstationarity called weak stationarity From the name we can see that it is a weaker

Trang 26

form of stationarity which stands if both the mean of X tand the covariance between

X t and X t −i are time-invariant, where i is an arbitrary integer That is to say, for

a time series {X t } to meet the requirement of weakly stationary, it should satisfy

two conditions: (a) Constant mean: E(X t ) = µ; and (b) Cov(X t , X t −i ) = γ i onlydepends on i To illustrate the weak stationarity clearly, we take a series of Tobserved data points {X t | t = 1, , T } as example If we look at the time plot of

this weak stationary series, we can find that the values of the series are fluctuatingwithin a fixed interval and with a constant variation In practical applications,weak stationarity has a wider use and enables one to make inferences concerningfuture observations If the first two moments of {X t } are finite, the time series

can be regarded as under the weak stationarity condition implicitly From thedeﬁnitions, a time series {X t } under strictly stationary condition has its ﬁrst two

moments to be ﬁnite, so we can conclude that the strong stationary implies theweak stationary However, the converse deduction does not hold In addition, ifthe time series{X t } is normally distributed, then the two stationarity is equivalent

to each other due to the special properties of the normal distribution

Methods for time series analysis may be divided into two classes: domain methods and time-domain methods Auto-correlation and cross-correlationanalysis are included in the latter class, which is to examine serial dependence

Trang 27

frequency-In linear time series analysis, correlation is of great importance to understandvarious classes of models Special attention has been paid to the correlations be-tween the variable and its past values This concept of correlation is generalized

to autocorrelation, which is the basic tool for studying a stationary time series Inother text it is also referred as serial correlations

Consider a weakly stationary time series {X t }, the linear dependence between

X t and its past values X t −iis of interest We call the correlation coeﬃcient between

X t and X t −i as the lag-i autocorrelation of X t and is commonly denoted by ρ i.Speciﬁcally, we deﬁne

Under the weak stationarity condition, V ar(X t ) = V ar(X t −i ) and ρ i is a function

of i only From the deﬁnition, we have ρ0 = 1, ρ i = ρ −i, and −1 ≤ ρ i ≤ 1 In

addition, a weakly stationary series{X t } is not autocorrelated if and only if ρ i = 0

for all i > 0.

Here, we also introduce the partial autoregressive function (PACF) for a tionary time series to understand other properties of the series PACF is a function

sta-of its ACF and is a powerful method for determining the order p sta-of an AR model

A simple, yet eﬀective way to introduce PACF is to consider the following ARmodels in consecutive orders:

Trang 28

bΦ1,1 in the ﬁrst equation is called the lag-1 sample PACF of x t; the estimate bΦ2,2

in the second equation is the lag-2 sample PACF of x t; the estimate bΦ3,3 in the

third equation is the lag-3 sample PACF of x t, and so on From the deﬁnition, thelag-2 PACF bΦ2,2 shows the added contribution of x t −2 to x tover the AR(1) model

x t = Φ0 + Φ1x t −1 + e 1t The lag-3 PACF shows the added contribution of x t −3

to x t over an AR(2) model, and so on Therefore, for an AR(p) model, the lag-psample PACF should not be zero, but bΦj,j should be close to zero for all j > p.

This means that the sample PACF cuts oﬀ at lag p and this property is often used

to determine the value of order p for the autoregressive model The following otherproperties of sample PACF can be obtained for a stationary AR(p) model:

• bΦ p,p converges to Φp as the sample size T goes to inﬁnity

Trang 29

2.5 Basic Methods for Parameter Estimation

The AR model is widely used in science, engineering, econometrics, biometrics,geophysics, etc When a series is to be modeled by the AR model, the appropriateorder p should be determined and the parameters of the model must be estimated.There are a number of methods available for estimating its parameters for thismodel and of these the following three maybe the most commonly used

Maximum Likelihood method has a wide use for estimation Time series sis also adopts it to estimate the parameters of the stationary ARMA(p,q) model

analy-To use the Maximum Likelihood method, let’s assume that time series{X t } follows

the Gaussian distribution Consider the gereral ARMA (p,q) model

where µ = E(X t ) and a t ∼ N(0, σ2

a) The joint probability density of a =

Trang 30

By maximizing ln L for the given series data, Maximum Likelihood estimator

is obtained Since the above log-likelihood function is based on the initial tion, so the estimators bϕ, bµ and bθ are called the condition Maximum Likelihood

condi-estimators

The estimator bσ2

a of σ2

a is obtained asb

a = S ∗( bϕ, bµ, bθ)

after bϕ, bµ and bθ are calculated.

Alternatively, because of the stationarity of the time series, an improvement wasproposed by Box, Jenkins, and Reinsel (1994) with the unknown future value in theforward form and unknown past backward forms The unconditional log-likelihoodfunction came out with this improvement

a= S ∗( bϕ, bµ, bθ)

Trang 31

The unconditional Maximum Likelihood method is eﬃcient in the situationsfor seasonal models, or nonstationary models or relatively short series Both theconditional and unconditional likelihood function are approximations The exactclosed form is very diﬃcult to derive Newbold (1974) illustrated an expression forthe ARMA(p,q) model.

One thing to mention here is that when X1, X2, , X n are independent andidentically distributed (i.i.d), when n is suﬃciently large, the Maximum Likelihoodestimators follow approximately normally distributions, the variances of which are

at least as small as those of other asymptotically normally distributed estimators(Lehamann, 1983) Even if {X t } is not normal distributed, Equation 2.16 still can

be used as a measure of goodness to ﬁt the model and the estimator calculated bymaximizing Equation 2.16 is still called Maximum Likelihood estimators For thescope of our study, we can obtained the ML estimator for the AR process setting

θ = 0.

Regression analysis is possibly the most widely used statistical method in dataanalysis Among the various regression methods, Least Square is well developedfor the linear regression models and been used frequently for estimation Theprincipal of Least Square approach is to minimize the standard sum of squares of

the errors term ϵ t AR model is a simple linear regression model and it utilizes

Trang 32

the least squares method to ﬁt a model by minimizing the sum of square errors for

estimating parameters Consider the following AR(p) model:

That is, ε t is a zero mean white noise series of constant variance σ t2

Let ϕ denote the vector of known parameter

Trang 33

Detailed information for the above algorithm was explained by Kay and Marple

1981 Later, we found that the LS method uses the normal equations to implementthe linear system We have two common methods for solving the normal equation.One is by Cholesky factorization and the other one is by QR factorization WhileCholesky factorization is faster in computation, QR factorization has better numer-ical properties In Least Square method, we assume that the earlier observationsreceive the same weight as recent observations It gives the linear systems equation

Ax = b from least squares normal equation as follows:

QR factorization (Golub and Van Loan, 1996) are used to solve the ﬁrst and

second linear system equations Let’s rewrite normal equations A T Ax = A T b using

Trang 34

as recent observations However, the recent observations may be more importantfor the true behavior of the process so that, so discounted least squares methodwas proposed to take into account the condition that the older observations receiveproportionally less weight than the recent ones.

Yule-Walk Method method, also called the autocorrelation method, is a merically simple approach to estimate the AR parameters of the ARMA model Inthis method, an autoregressive (AR) model is also ﬁtted by minimizing the forwardprediction error in a sense of least-squares regression The diﬀerence is that Yule-Walker method is to solves the Yule-Walker equations, which is formed from sample

nu-covariances A stationary autoregressive (AR) process {Y t } of order p can be fully

identiﬁed from the ﬁrst p + 1 autocovariances, that is cov(Y t , Y t+k ), k = 0, 1, · · · , p,

by the Yule-Walker equations Moreover, the Yule-Walker equations have beenemployed in estimating the AR parameters and the disturbance variance from the

By Multiplying both side of Equation 2.24 by Y t −j , j = 0, 1, · · · , p, then taking

expections, we could get the YW Equation

Trang 35

where Γp is the covariance matrix [γ(i −j)] p

i,j=1 and γ p = (γ(1), · · · , γ(p)) ′

Replac-ing the covariance γ(j) by the correspondReplac-ing sample covariances bγ(j), the Walker estimator of ϕ is given below by (Young and Jakeman 1979)

Here, autocovariance could be replaced with autocorrelation (ACF) when

nor-malized by the variance, then the autocovariance γ i becomes the autocorrelation

ρ i with the values varying within interval [-1,1] The terms autocovariance andautocorrelation can be used interchangeably

Various algorithms, such as the Least Square algorithm or Levinson-Durbinalgorithm, can be used here to solve the above linear Yule-Walker system TheLevinson-Durbin recursion is quite eﬃcient for computation to get the AR (p)

parameters with the ﬁrst p autocorrelations Toeplitz structure of the matrix in

Equation 2.26 provides convenience for computation and makes the Yule-Walkermethods more attractive with more computational eﬃciency than the Least Squaremethod The advantage of the computational simplicity makes Yule-Walker anattractive choice for many applications

Trang 36

2.5.4 Burg’s Estimation Method (B)

Burg’s method is another different class of estimation method It has been foundthat Burg’s method, which is to solve the lattice filter equations using the harmonicmean of forward and backward squared prediction errors, gives a quite good perfor-mance with high accuracy and is regarded to be the most preferable method whenthe signal energy is non-uniformly distributed in a frequency range This is oftenthe case with audio signals Burg’s method is quite different from the Least Squareand Yule-Walker method which estimate the autoregressive parameters directly.Different from the Least Square method which minimizing the residual, Burg’smethod deals with prediction error Different from the Yule-Walker method, inwhich the estimated coefficients bϕ p1 , · · · , bϕ pp are precisely the coefficients of the

best linear predictor of Y P +1 in terms of Y p , · · · , Y1 under the assumption that

the ACF of Y t coincides with the sample ACF at lag 1, , p, Burg’s method ﬁrst

estimates the reflection coefficients, which are defined as the last autoregressiveparameter estimate for each model order p Reflection coefficients consists of un-biased estimates of the partial autocorrelation (PACF) coefficient Under Burg’smethod, PACF Φ11, Φ22, · · · , Φ pp is estimated by minimizing the sum of squares offorward and backward one-step prediction errors with respect to the coefficients Φii.Levinson-Durbin algorithm is also used here to determine the parameter estimates

It recursively computes the successive intermediate reﬂection coeﬃcients to derivethe parameters for the AR model Given a observed stationary zero mean series

Trang 37

Y(t), we denote u i (t), t = i1, , n, 0 ≤ i < n, to be the diﬀerence between x n+1+i −t

and the best linear estimate of x n+1+i −t in terms of the preceding i observations.

Also, denote v i (t), t = i1, , n, 0 ≤ i < n, to be the diﬀerence between x n+1 −t

and the best linear estimate of x n+1 −t in terms of the subsequent i observations

u i (t) and v i (t) are so called forward and backward prediction errors and satisfy the

The values for u1(t), v1(t) and δ21 generated from Equation 2.31 can be used to

replace the value in above recursion steps with i = 2 and Burg’s estimate Φ (B)22 of

Φ22 is obtained Continuing this recursion process, we can ﬁnally get Φ(B) pp Forpure autoregressive models, Burg’s method usually performs better with a higherlikelihood than Yule-Walker method

Monte Carlo simulation is a method that takes sets of random numbers as input

to iteratively evaluate a deterministic model The aim of Monte Carlo simulation

Trang 38

is to understand the impact of uncertainty, and to develop plans to mitigate or erwise cope with risk This method is especially useful for uncertainty propagationsituations such as variation determination, sensitivity error aﬀects, performance orreliability of the system modeling without enough information For a simulationinvolving in extremely large number of evaluations of the model could only be donewith super computers Monte Carlo simulation is a sampling method which ran-domly generates the inputs from probability distributions to simulate the process ofsampling from an actual population To use this method, we ﬁrstly should choose

oth-a distribution for the inputs to moth-atch the existing doth-atoth-a, or to represent our currentstate of knowledge There are several methods to represent the data generatedfrom the simulation, such as histogram, summary statistics, error bars, reliabilitypredictions, tolerance zones, and confidence intervals Monte Carlo simulation is aall round method with a wide range of applications in various fields We can ben-efit a lot from the simulation method for analyzing the behavior of some activity,plan or process that involves uncertainty To deal with variable market demand

in economy, ﬂuctuating costs in business, variation in a manufacturing process, orunpredictable weather data in meteorology, you can always ﬁnd the important role

of Monte Carlo simulation

Thought Monte Carlo simulation has a powerful function, the steps in it are quitesimple The following steps illustrate the common simulation procedures:

Step 1: Create a parametric model, y = f (x1, x2, , x q)

Trang 39

Step 2: Generate a set of random inputs, x i 1, x i 2, , x i q.

Step 3: Evaluate the model and store the results as y i

Step 4: Repeat steps 2 and 3 for i = 1 to n

Step 5: Analyze the results using probability distribution, conﬁdence val, etc

Trang 40

In the Yule-Walker Method, AR(p) parameters depend on merely the ﬁrst p +

1 lags from γ0 to γ p This subset of the given autocorrelation lags can reﬂectonly part of the information contained in the series, which means that AR model

Định dạng
Số trang	80
Dung lượng	295,56 KB