Again suppose that we want to use simulation to estimateθ = E[X], whereX is the output of a simulation. Now suppose that for some other output variableY, the expected value ofYis known—say,E[Y]=μy. Then for any constantc, the quantity
X+c(Y −μy)
is also an unbiased estimator ofθ. To determine the best value ofc, note that Var(X+c(Y−μy))=Var(X+cY)
=Var(X)+c2Var(Y)+2cCov(X,Y) Simple calculus now shows that the above is minimized whenc=c∗, where
c∗= −Cov(X,Y)
Var(Y) (9.1)
and for this value the variance of the estimator is
Var(X+c∗(Y−μy))=Var(X)−[Cov(X,Y)]2
Var(Y) (9.2)
The quantityYis called acontrol variatefor the simulation estimatorX. To see why it works, note that c∗ is negative (positive) when X and Y are positively (negatively) correlated. So suppose that X and Y were positively correlated, meaning, roughly, that X is large when Y is large and vice versa. Hence, if a simulation run results in a large (small) value ofY—which is indicated byYbeing larger (smaller) than its known mean μy—then it is probably true thatXis also larger (smaller) than its meanθ, and so we would like to correct for this by lowering (raising) the value of the estimatorX, and this is done sincec∗is negative (positive).
A similar argument holds whenXandYare negatively correlated.
Upon dividing Equation (9.2) by Var(X), we obtain that Var(X+c∗(Y −μy))
Var(X) =1−Corr2(X,Y) where
Corr(X,Y)= √Cov(X,Y) Var(X)Var(Y)
is the correlation betweenXandY. Hence, the variance reduction obtained in using the control variateYis 100 Corr2(X,Y)percent.
The quantities Cov(X,Y)and Var(Y)are usually not known in advance and must be estimated from the simulated data. Ifnsimulation runs are performed, and the output dataXi,Yi,i =1, . . . ,n, result, then using the estimators
Cov(X,Y)= n
i=1
(Xi−X)(Yi−Y)/(n−1) and
Var(Y)= n
i=1
(Yi−Y)2/(n−1), we can approximatec∗bycˆ∗, where
ˆ c∗= −
n
i=1(Xi−X)(Yi−Y) n
i=1(Yi−Y)2 . The variance of the controlled estimator
Var(X+c∗(Y −μy))= 1 n
Var(X)−Cov2(X,Y) Var(Y)
can then be estimated by using the estimator of Cov(X,Y)along with the sample variance estimators of Var(X)and Var(Y).
164 9 Variance Reduction Techniques
Remark Another way of doing the computations is to make use of a standard computer package for simple linear regression models. For if we consider the simple linear regression model
X =a+bY +e
whereeis a random variable with mean 0 and varianceσ2, thenaˆandb, the leastˆ squares estimators ofaandbbased on the dataXi,Yi,i =1, . . . ,n, are
bˆ= n
i=1(Xi−X)(Yi−Y) n
i=1(Yi−Y)2 ˆ
a=X− ˆbY Therefore,bˆ= −ˆc∗. In addition, since
X+ ˆc∗(Y −μy)=X− ˆb(Y −μy)
= ˆa+ ˆbμy
it follows that the control variate estimate is the evaluation of the estimated regression line at the valueY =μy. Also, becauseσˆ2, the regression estimate of σ2, is the estimate of Var(X− ˆbY)=Var(X+ ˆc∗Y), it follows that the estimated variance of the control variate estimatorX+ ˆc∗(Y−μy)isσˆ2/n.
Example 9f Suppose, as in Example9b, that we wanted to use simulation to estimate the reliability function
r(p1, . . . ,pn)=E[φ(S1, . . . ,Sn)]
where
Si=
1 ifUi< pi 0 otherwise SinceE[Si]=pi, it follows that
E n
i=1
Si
= n
i=1
pi
Hence, we can use the number of working components,Y ≡
Si, as a control variate of the estimatorX ≡φ(S1, . . . ,Sn). Sincen
i=1Siandφ(S1, . . . ,Sn)are both increasing functions of theSi, they are positively correlated, and thus the sign
ofc∗is negative.
Example 9g Consider a queueing system in which customers arrive in accordance with a nonhomogeneous Poisson process with intensity function λ(s),s > 0. Suppose that the service times are independent random variables
having distributionGand are also independent of the arrival times. Suppose we were interested in estimating the total time spent in the system by all customers arriving before timet. That is, if we letWi denote the amount of time that the ith entering customer spends in the system, then we are interested inθ = E[X], where
X =
N(t)
i=1
Wi
and whereN(t)is the number of arrivals by timet. A natural quantity to use as a control in this situation is the total of the service times of all these customers. That is, letSidenote the service time of theith customer and set
Y =
N(t)
i=1
Si
Since the service times are independent ofN[t], it follows that E[Y]=E[S]E[N(t)]
whereE[S], the mean service time, andE[N(t)], the mean number of arrivals by
t, are both known quantities.
Example 9h As in Example9d, suppose we were interested in using simu- lation to compute θ = E[eU]. Here, a natural variate to use as a control is the random numberU. To see what sort of improvement over the raw estimator is possible, note that
Cov(eU,U)=E[U eU]−E[U]E[eU]
= 1
0
xexd x−(e−1) 2
=1−(e−1)
2 =0.14086 Because Var(U)= 121 it follows from (9.2) that
Var
eU+c∗
U−1 2
=Var(eU)−12(0.14086)2
=0.2420−0.2380=0.0039
where the above used, from Example9d, that Var(eU) =0.2420. Hence, in this case, the use of the control variateUcan lead to a variance reduction of up to 98.4
percent.
166 9 Variance Reduction Techniques
Example 9i A List Recording Problem Suppose we are given a set ofnelements, numbered 1 throughn, which are to be arranged in an ordered list.
At each unit of time a request is made to retrieve one of these elements, with the request being for elementiwith probabilityp(i),n
i=1p(i)=1. After being requested, the element is put back in the list but not necessarily in the same position.
For example, a common reordering rule is to interchange the requested element with the one immediately preceding it. Thus, ifn =4 and the present ordering is 1, 4, 2, 3, then under this rule a request for element 2 would result in the reorder 1, 2, 4, 3. Starting with an initial ordering that is equally likely to be any of then! orderings and using this interchange rule, suppose we are interested in determining the expected sum of the positions of the firstNelements requested. How can we efficiently accomplish this by simulation?
One effective way is as follows. The “natural” way of simulating the above is first to generate a random permutation of 1,2, . . . ,n to establish the initial ordering, and then at each of the nextNperiods determine the element requested by generating a random numberUand then letting the request be for element j ifj−1
k=1 p(k) <U ⩽j
k=1p(k). However, a better technique is to generate the element requested in such a way that small values ofUcorrespond to elements close to the front. Specifically, if the present ordering isi1,i2, . . . ,in, then generate the element requested by generating a random numberUand then letting the selection be forijifj−1
k=1p(ik) <U⩽j
k=1p(ik). For example, ifn=4 and the present ordering is 3, 1, 2, 4, then we should generateUand let the selection be for 3 if U⩽ p(3), let it be for 1 if p(3) <U ⩽p(3)+p(1), and so on. As small values ofUthus correspond to elements near the front, we can useN
r=1Uras a control variable, whereUris the random number used for therth request in a run. That is, ifPris the position of therth selected element in a run, then rather than just using the raw estimatorN
r=1Prwe should use N
r=1
Pr+c∗ N
r=1
Ur− N 2
where
c∗= −CovN
r=1Pr,N r=1Ur
N 12
and where the above covariance should be estimated using the data from all the simulated runs.
Although the variance reduction obtained will, of course, depend on the probabilities p(i),i =1, . . . ,n, and the value ofN, a small study indicates that when n = 50 and the p(i)are approximately equal, then for 15 ⩽ N ⩽ 50 the variance of the controlled estimator is less than 24001 the variance of the raw
simulation estimator.
Of course, one can use more than a single variable as a control. For example, if a simulation results in output variablesYi,i =1, . . . ,k, andE[Yi]=μi is known, then for any constantsci,i=1, . . . ,k, we may use
X+ k
i=1
ci(Yi−μi) as an unbiased estimator ofE[X].
Example 9j Blackjack The game of blackjack is often played with the dealer shuffling multiple decks of cards, putting aside used cards, and finally reshuffling when the number of remaining cards is below some limit. Let us say that a new round begins each time the dealer reshuffles, and suppose we are interested in using simulation to estimate E[X], a player’s expected winnings per round, where we assume that the player is employing some fixed strategy which might be of the type that “counts cards” that have already been played in the round and stakes different amounts depending on the “count.” We will assume that the game consists of a single player against the dealer.
The randomness in this game results from the shuffling of the cards by the dealer.
If the dealer useskdecks of 52 cards, then we can generate the shuffle by generating a random permutation of the numbers 1 through 52k; letI1, . . . ,I52k denote this permutation. If we now set
uj=Ijmod 13+1 and let
vj =min(uj,10)
thenvj,j = 1, . . . ,52k represents the successive values of the shuffled cards, with 1 standing for an ace.
Let Ndenote the number of hands played in a round, and let Bj denote the amount bet on handj. To reduce the variance, we can use a control variable that is large when the player is dealt more good hands than the dealer, and is small in the reverse case. Since being dealt 19 or better is good, let us define
Wj=1 if the player’s two dealt cards on deal jadd to atleast 19 and letWjbe 0 otherwise. Similarly, let
Zj=1 if the dealer’s two dealt cards on dealjadd to atleast 19
and letZj be 0 otherwise. SinceWj andZj clearly have the same distribution it follows thatE[Wj−Zj]=0, and it is not difficult to show that
E N
j=1
Bj(Wj−Zj)
=0
168 9 Variance Reduction Techniques
Thus, we recommend usingN
j=1Bj(Wj−Zj)as a control variable. Of course, it is not clear that 19 is the best value, and one should experiment on letting 18 or even 20 be the critical value. However, some preliminary work indicates that 19 works best, and it has resulted in variance reductions of 15 percent or more depending on the strategy employed by the player. An even greater variance reduction should result if we use two control variables. One control variable is defined as before, with the exception that theWjandZjare defined to be 1 if the hand is either 19 or 20. The second variable is again similar, but this time its indicators are 1 when
the hands consist of blackjacks.
When multiple control variates are used, the computations can be performed by using a computer program for the multiple linear regression model
X =a+ k
i=1
biYi+e
where eis a random variable with mean 0 and variance σ2. Lettingcˆ∗i be the estimate of the bestci, fori=1, . . . ,k, then
ˆ
ci∗= − ˆbi, i =1, . . . ,k
where bˆi,i = 1, . . . ,k, are the least squares regression estimates of bi,i = 1, . . . ,k. The value of the controlled estimate can be obtained from
X+ k
i=1
ˆ
ci∗(Yi−μi)= ˆa+ k
i=1
bˆiμi
That is, the controlled estimate is just the estimated multiple regression line evaluated at the point(μ1, . . . , μk).
The variance of the controlled estimate can be obtained by dividing the regression ofσ2by the number of simulation runs.
Remarks
1. Since the variance of the controlled estimator is not known in advance, one often performs the simulation in two stages. In the first stage a small number of runs are performed so as to give a rough estimate of Var(X+c∗(Y−μy)).
(This estimate can be obtained from a simple linear regression program, where Yis the independent andX is the dependent variable, by using the estimate ofσ2.) We can then fix the number of trials needed in the second run so that the variance of the final estimator is within an acceptable bound.
2. A valuable way of interpreting the control variable approach is that it combines estimators ofθ. That is, suppose the values ofXandWare both determined by the simulation, and suppose E[X] = E[W]= θ. Then we may consider any unbiased estimator of the form
αX+(1−α)W
The best such estimator, which is obtained by choosingαto minimize the variance, is given by lettingα=α∗, where
α∗= Var(W)−Cov(X,W)
Var(X)+Var(W)−2 Cov(X,W) (9.3) Now if E[Y]= μy is known, we have the two unbiased estimatorsXand X+Y −μy. The combined estimator can then be written as
(1−c)X+c(X+Y −μy)=X+c(Y −μy)
To go the other way in the equivalence between control variates and combining estimators, suppose that E[X] = E[W] = θ. Then if we use X, controlling with the variableY =X−W, which is known to have mean 0, we then obtain an estimator of the form
X+c(X−W)=(1+c)X−cW which is a combined estimator withα=1+c.
3. With the interpretation given in Remark 2, the antithetic variable approach may be regarded as a special case of control variables. That is, ifE[X]=θ, where X = h(U1, . . . ,Un), then also E[W] = θ, where W = h(1 − U1, . . . ,1−Un). Hence, we can combine to get an estimator of the form αX +(1−α)W. Since Var(X) = Var(W), as X and W have the same distribution, it follows from Equation (9.3) that the best value ofαisα=12, and this is the antithetic variable estimator.
4. Remark 3 indicates why it is not usually possible to effectively combine antithetic variables with a control variable. If a control variable Y has a large positive (negative) correlation with h(U1, . . . ,Un)then it probably has a large negative (positive) correlation with h(1 −U1, . . . ,1 −Un).
Consequently, it is unlikely to have a large correlation with the antithetic estimatorh(U1,...,Un)+h(1−U2 1,...,1−Un).