Process or Product Monitoring and Control_9 doc

The partial autocorrelation plot should beexamined to determine the order.In summary, our intial attempt would be to fit an AR2 model with no seasonal terms and no differencing or trend

6 Process or Product Monitoring and Control

6.4 Introduction to Time Series Analysis

6.4.4 Univariate Time Series Models

6.4.4.6 Box-Jenkins Model Identification

6.4.4.6.1 Model Identification for Southern

The run sequence plot indicates stationarity

6.4.4.6.1 Model Identification for Southern Oscillations Data

Plot

The autocorrelation plot shows a mixture of exponentially decaying6.4.4.6.1 Model Identification for Southern Oscillations Data

and damped sinusoidal components This indicates that anautoregressive model, with order greater than one, may beappropriate for these data The partial autocorrelation plot should beexamined to determine the order.

In summary, our intial attempt would be to fit an AR(2) model with

no seasonal terms and no differencing or trend removal Modelvalidation should be performed before accepting this as a finalmodel

6.4.4.6.1 Model Identification for Southern Oscillations Data

6 Process or Product Monitoring and Control

6.4 Introduction to Time Series Analysis

6.4.4 Univariate Time Series Models

6.4.4.6 Box-Jenkins Model Identification

6.4.4.6.2 Model Identification for the CO 2

Concentrations Data

Example for

Monthly CO 2

Concentrations

The second example is for the monthly CO2 concentrations data set

As before, we start with the run sequence plot to check forstationarity

Run Sequence

Plot

The initial run sequence plot of the data indicates a rising trend Avisual inspection of this plot indicates that a simple linear fit should

be sufficient to remove this upward trend

6.4.4.6.2 Model Identification for the CO2 Concentrations Data

Linear Trend

Removed

This plot contains the residuals from a linear fit to the original data.After removing the linear trend, the run sequence plot indicates thatthe data have a constant location and variance, which implies

stationarity

However, the plot does show seasonality We generate anautocorrelation plot to help determine the period followed by aseasonal subseries plot

Autocorrelation

Plot

6.4.4.6.2 Model Identification for the CO2 Concentrations Data

The autocorrelation plot shows an alternating pattern of positive andnegative spikes It also shows a repeating pattern every 12 lags,which indicates a seasonality effect.

The two connected lines on the autocorrelation plot are 95% and99% confidence intervals for statistical significance of the

To help identify the non-seasonal components, we will take aseasonal difference of 12 and generate the autocorrelation plot on theseasonally differenced data

6.4.4.6.2 Model Identification for the CO2 Concentrations Data

The partial autocorrelation plot suggests that an AR(2) model might

be appropriate since the partial autocorrelation becomes zero afterthe second lag The lag 12 is also significant, indicating some6.4.4.6.2 Model Identification for the CO2 Concentrations Data

6 Process or Product Monitoring and Control

6.4 Introduction to Time Series Analysis

6.4.4 Univariate Time Series Models

6.4.4.6 Box-Jenkins Model Identification

6.4.4.6.3 Partial Autocorrelation Plot

Specifically, partial autocorrelations are useful in identifying the order

of an autoregressive model The partial autocorrelation of an AR(p) process is zero at lag p+1 and greater If the sample autocorrelation plot

indicates that an AR model may be appropriate, then the sample partialautocorrelation plot is examined to help identify the order We look forthe point on the plot where the partial autocorrelations essentiallybecome zero Placing a 95% confidence interval for statisticalsignificance is helpful for this purpose

The approximate 95% confidence interval for the partialautocorrelations are at

6.4.4.6.3 Partial Autocorrelation Plot

Sample Plot

This partial autocorrelation plot shows clear statistical significance forlags 1 and 2 (lag 0 is always 1) The next few lags are at the borderline

of statistical significance If the autocorrelation plot indicates that an

AR model is appropriate, we could start our modeling with an AR(2)model We might compare this with an AR(3) model

Definition Partial autocorrelation plots are formed by

Vertical axis: Partial autocorrelation coefficient at lag h.

Horizontal axis: Time lag h (h = 0, 1, 2, 3, ).

In addition, 95% confidence interval bands are typically included on theplot

Questions The partial autocorrelation plot can help provide answers to the

Case Study The partial autocorrelation plot is demonstrated in the Negiz data case

study.6.4.4.6.3 Partial Autocorrelation Plot

Software Partial autocorrelation plots are available in many general purpose

statistical software programs including Dataplot.6.4.4.6.3 Partial Autocorrelation Plot

6 Process or Product Monitoring and Control

6.4 Introduction to Time Series Analysis

6.4.4 Univariate Time Series Models

6.4.4.7 Box-Jenkins Model Estimation

Use Software Estimating the parameters for the Box-Jenkins models is a quite

complicated non-linear estimation problem For this reason, theparameter estimation should be left to a high quality software programthat fits Box-Jenkins models Fortunately, many commerical statisticalsoftware programs now fit Box-Jenkins models

Approaches The main approaches to fitting Box-Jenkins models are non-linear

least squares and maximum likelihood estimation

Maximum likelihood estimation is generally the preferred technique.The likelihood equations for the full Box-Jenkins model are

complicated and are not included here See (Brockwell and Davis,

1991) for the mathematical details

6 Process or Product Monitoring and Control

6.4 Introduction to Time Series Analysis

6.4.4 Univariate Time Series Models

6.4.4.8 Box-Jenkins Model Diagnostics

Box-Jenkins model is a good model for the data, the residuals shouldsatisfy these assumptions

If these assumptions are not satisfied, we need to fit a moreappropriate model That is, we go back to the model identification stepand try to develop a better model Hopefully the analysis of the

residuals can provide some clues as to a more appropriate model

6 Process or Product Monitoring and Control

6.4 Introduction to Time Series Analysis

6.4.4 Univariate Time Series Models

6.4.4.9 Example of Univariate Box-Jenkins Analysis

The graph of the data and the resulting forecasts after fitting a model are portrayed below.

Output from other software programs will be similar, but not identical.

Enter FILESPEC or EXTENSION (1-3 letters): To quit, press F10.

? bookf.bj MAX MIN MEAN VARIANCE NO DATA 80.0000 23.0000 51.7086 141.8238 70

Do you wish to make transformations? y/n n

Input order of difference or 0: 0 Input period of seasonality (2-12) or 0: 0

Time Series: bookf.bj Regular difference: 0 Seasonal Difference: 0 Autocorrelation Function for the first 35 lags

6.4.4.9 Example of Univariate Box-Jenkins Analysis

Fitting

? bookf.bj MAX MIN MEAN VARIANCE NO DATA 80.0000 23.0000 51.7086 141.8238 70

Do you wish to make transformations? y/n n

Input order of difference or 0: 0 Input NUMBER of AR terms: 2 Input NUMBER of MA terms: 0 Input period of seasonality (2-12) or 0: 0

*********** OUTPUT SECTION ***********

AR estimates with Standard Errors Phi 1 : -0.3397 0.1224 Phi 2 : 0.1904 0.1223

Original Variance : 141.8238 Residual Variance : 110.8236 Coefficient of Determination: 21.8582

***** Test on randomness of Residuals *****

The Chi-Square value = 11.7034 with degrees of freedom = 23

The 95th percentile = 35.16596

Hypothesis of randomness accepted.

Press any key to proceed to the forecasting section

6.4.4.9 Example of Univariate Box-Jenkins Analysis

Section

FORECASTING SECTION

How many periods ahead to forecast? (9999 to quit ): Enter confidence level for the forecast limits :

90 Percent Confidence limits Next Lower Forecast Upper

6.4.4.9 Example of Univariate Box-Jenkins Analysis

6.4.4.9 Example of Univariate Box-Jenkins Analysis

6 Process or Product Monitoring and Control

6.4 Introduction to Time Series Analysis

6.4.4 Univariate Time Series Models

6.4.4.10 Box-Jenkins Analysis on Seasonal

The graph of the data and the resulting forecasts after fitting a model areportrayed below

Do you wish to make transformations? y/n y

The following transformations are available:

1 Square root 2 Cube root

3 Natural log 4 Natural log log

5 Common log 6 Exponentiation

7 Reciprocal 8 Square root of Reciprocal

9 Normalizing (X-Xbar)/Standard deviation

10 Coding (X-Constant 1)/Constant 2 Enter your selection, by number: 3 Statistics of Transformed series:

Mean: 5.542 Variance 0.195 Input order of difference or 0: 1

Input period of seasonality (2-12) or 0: 12 Input order of seasonal difference or 0: 0 Statistics of Differenced series:

Mean: 0.009 Variance 0.011 Time Series: bookg.bj

Regular difference: 1 Seasonal Difference: 0 Autocorrelation Function for the first 36 lags

6.4.4.10 Box-Jenkins Analysis on Seasonal Data

Autocorrelation

Plot for

Seasonality

If you observe very large autocorrelations at lags spaced n periods apart, for

example at lags 12 and 24, then there is evidence of periodicity That effectshould be removed, since the objective of the identification stage is to reducethe autocorrelations throughout So if simple differencing was not enough,try seasonal differencing at a selected period In the above case, the period is

12 It could, of course, be any value, such as 4 or 6

The number of seasonal terms is rarely more than 1 If you know the shape ofyour forecast function, or you wish to assign a particular shape to the forecastfunction, you can select the appropriate number of terms for seasonal AR orseasonal MA models

The book by Box and Jenkins, Time Series Analysis Forecasting and Control

(the later edition is Box, Jenkins and Reinsel, 1994) has a discussion on theseforecast functions on pages 326 - 328 Again, if you have only a faint notion,but you do know that there was a trend upwards before differencing, pick aseasonal MA term and see what comes out in the diagnostics

The results after taking a seasonal difference look good!

6.4.4.10 Box-Jenkins Analysis on Seasonal Data

Model Fitting

Section

Now we can proceed to the estimation, diagnostics and forecasting routines.The following program is again executed from a menu and issues the

following flow of output:

Enter FILESPEC or EXTENSION (1-3 letters):

y (we selected a square root

transformation because a closerinspection of the plot revealedincreasing variances over time) Statistics of Transformed series:

Mean: 5.542 Variance 0.195 Input order of difference or 0: 1

Input NUMBER of AR terms: Blank defaults to 0 Input NUMBER of MA terms: 1

Input period of seasonality (2-12) or

Input order of seasonal difference or

Input NUMBER of seasonal AR

Input NUMBER of seasonal MA

Statistics of Differenced series:

6.4.4.10 Box-Jenkins Analysis on Seasonal Data

Mean: 0.000 Variance 0.002

Estimation is finished after 3 Marquardt iterations

Output Section MA estimates with Standard Errors

Theta 1 : 0.3765 0.0811 Seasonal MA estimates with Standard ErrorsTheta 1 : 0.5677 0.0775

Original Variance : 0.0021Residual Variance (MSE) : 0.0014 Coefficient of Determination : 33.9383

AIC criteria ln(SSE)+2k/n : -1.4959 BIC criteria ln(SSE)+ln(n)k/n: -1.1865

k = p + q + P + Q + d + sD = number of estimates + order of regular

difference + product of period of seasonality and seasonal difference

n is the total number of observations.

In this problem k and n are: 15 144

***** Test on randomness of Residuals *****

The Box-Ljung value = 28.4219The Box-Pierce value = 24.0967with degrees of freedom = 30 The 95th percentile = 43.76809 Hypothesis of randomness accepted

Forecasting

Section

Defaults are obtained by pressing the enter key, without input

Default for number of periods ahead from last period = 6

Default for the confidence band around the forecast = 90%

Next Period Lower Forecast Upper