Forecasting methods can be evaluated through accuracy measures calculated on the basis of errors made in the past. Such measures can be employed to select the most precise approach. Moreover, in the case of periodic predictions (like those required by inventory management), forecasting errors should be monitored in order to adjust parameters if needed. For the sake of brevity, we examine these issues for the case where a one-period-ahead forecast has to be generated.
2.9.1 Accuracy measures
To evaluate the accuracy of a forecasting method, the errors made in the past have to be computed. Then a number of indices (themean absolute deviation(MAD), the mean absolute percentage deviation(MAPD) and themean squared error(MSE)) at time periodtcan be defined:
MADt = t
k=2|ek|
t−1 , (2.20)
MAPDt =100 t
k=2|ek|/dk
t−1 , (2.21)
MSEt = t
k=2ek2
t−2 , (2.22)
where 1< t⩽T for Equations (2.20) and (2.21), and 2< t ⩽Tfor Equation (2.22).
These three accuracy measures can be used at time periodt = T to establish a comparison between different forecasting methods. In particular, MAPDT can be used to evaluate the quality of a forecasting method (see Table 2.26).
FORECASTING LOGISTICS REQUIREMENTS 65 Table 2.26 Evaluation of the forecasting accuracy through MAPDT.
MAPDT Quality of forecast
⩽10% Very good
>10%,⩽20% Good
>20%,⩽30% Moderate
>30% Poor
Table 2.27 Mean absolute deviation in the Sarath microwave ovens forecasting problem.
α MAD12
0.05 9.30 0.10 9.27 0.15 9.28 0.20 9.33 0.25 9.40 0.30 9.50 0.35 9.63 0.40 9.79 0.45 9.96
The accuracy of the exponential smoothing method will be evaluated for different values of the smoothing constantαfor the TV sets forecasting problem of Sarath company. By using MAD12 (see Table 2.27),α = 0.1 comes out to give the most precise forecast. With this value of the smoothing constant we obtain MAPD12 = 0.79%, which corresponds to a very good accuracy.
2.9.2 Forecast control
A forecasting method works correctly if the errors are random and not systematic.
Typical systematic errors occur when the demand value is constantly underestimated or overestimated, and a seasonal variation is not taken into account. Forecasting control can be done through atracking signalor acontrol chart. The tracking signal St, 1< t⩽T, is defined as the ratio between the cumulative error and the MADt,
St = Et MADt,
66 FORECASTING LOGISTICS REQUIREMENTS
Interval of acceptable values
Need of a corrective action
t SMAX
−SMAX 0 St
Figure 2.18 Use of a tracking signal for a forecasting control.
where
Et = t k=2
ek.
The tracking signal is greater than zero if the forecast systematically underestimates the demand; vice versa, a negative value ofStindicates a systematic overestimate of the demand. For this reason, a forecast is assumed to be unbiased if the tracking signal falls in the range±Smax. The value ofSmaxis established heuristically, and usually varies between 3 and 8. If the tracking signal is outside this interval, the parameters of the forecasting method should be modified or a different forecasting method should be selected (see Figure 2.18).
Unlike tracking signals, control charts are based on the plot of single errorset. Under the hypothesis that the expected value of the errors is zero, a forecast is effective if each errorek,k=2, . . . , t, is in the confidence interval±mσt, whereσt is the standard deviation of the errors. An estimate ofσt can be obtained as
σt = MSEt.
The parametermcan be related to the probability that the error be in the interval
±mσt. If the errors are normally distributed with zero mean, the error belongs to the interval±2σt with a probability around 97.7%, and to the interval ±3σt with a probability around 99.8%. Finally, it is worth observing that the interval ±3σt
corresponds approximately to the band±4 of the tracking signal.
In addition to the previous analytical check, it is useful to verify visually whether the error pattern reveals the possibility of improving the forecast by introducing suitable modifications. Here are three of the most common pathological situations.
• The errors have an expected value different from zero; this means that the forecast is biased (see Figure 2.19).
FORECASTING LOGISTICS REQUIREMENTS 67
0
t et
Figure 2.19 Nonzero mean error.
• The error pattern shows a positive or negative trend; in this case, the accuracy of the forecasting method is progressively diminishing.
• The error pattern is periodic; this can happen if an existing seasonal effect has not been identified.
By using the tracking signal (±4 band) and the control chart (±3σT), we can mon- itor the demand forecast of sports goods for Browns supermarkets. The forecasting technique is the exponential smoothing method withα =0.3. The demand history for the last few months, and the required forecasts (both in hundreds of dollars), are reported in Table 2.28. The tracking signalSt,t =2, . . . , T, is always in the inter- val±4. On the basis of this preliminary evaluation, we can state that the forecast is under control. To make a further check, the expected value and the standard deviation of the error at time periodt = T are estimated. The results are −3.02 and 45.36, respectively. We observe that, since the average error is much less than the average demand, we can consider the forecast unbiased. Furthermore, since all errors are in the interval±3σT, even this test suggests that the forecast is under control. Finally, the examination of the control chart (see Figure 2.20) does not show any systematic error.