Matematik simulation and monte carlo with applications in finance and mcmc phần 3 ppsx

Then the density of Y is proportional to hy wherehy= 1 Now show that the following algorithm will sample variates x from the density fXx.. The adaptive rejection sampling method is appl

(i) the standard ratio method,(ii) the relocated method where Y= X − m = X −2

9,(iii) a numerical approximation to the optimal choice of 

11 Consider the following approach to sampling from a symmetric beta distribution withdensity

fXx=

2 x −11− x −1 0Put Y= X −1

2 Then the density of Y is proportional to hy wherehy=

1

Now show that the following algorithm will sample variates x from the density

fXx R1 and R2are two independent U0 1 random variables

1 U=

12

If U <

1

4− Y2

−1/2deliver Y+1

4 = 0731

12 The adaptive rejection sampling method is applicable when the density function islog-concave Examine whether or not the following densities are log-concave:(a) normal;

(b) fx∝ x −1e−xon support 0

Problems 57(c) Weibull: fx∝  −1exp − x on support 0

(d) lognormal: fx∝ 1/x exp−1

2ln x− /2

on support 0,  > 0,

∈ 

13 In adaptive rejection sampling from a density f∝ h, rx = lnhx must be concave

Given k ordered abscissae x0 k∈ supporth, the tangents to y = rx at x = xj

and x= xj+1 respectively, intersect at x= zj, for j 0≡ z0≡

infx  x∈ supporth and xk +1≡ zk≡ supx  x ∈ supporth Let uky, x0< y <

xk+1, be the piecewise linear hull formed from these tangents Then uk is an upper

envelope to r It is necessary to sample a prospective variate from the density

y= expuky

x k+1

x0 expukydy

(a) Show that

y=kj=1

(b) In (a), the density  is represented as a probability mixture This means that in

order to sample a variate from  a variate from the density j is sampled with

(c) Describe how you would randomly select in (b) the value of j so that a sample

may be drawn from  Give an algorithm in pseudo-code

14 In a single server queue with Poisson arrivals at rate <  and service durationsthat are independently distributed as negative exponential with mean −1, it can be

shown that the distribution of waiting time W in the queue, when it has reached a

steady state, is given by

Generation of variates from

standard distributions

4.1 Standard normal distribution

The standard normal distribution is so frequently used that it has its own notation

A random variable Z follows the standard normal distribution if its density is , where

 z=√1

2e

−z 2 /2

on support −  The cumulative distribution is  z =z

− u du It is easy toshow that the expectation and variance of Z are 0 and 1 respectively

Suppose X= + Z for any  ∈  and ≥ 0 Then X is said to be normally distributedwith mean  and variance 2 The density of X is

The Box–Müller method is simple to implement and reasonably fast in execution Westart by considering two independent standard normal random variables, X1 and X2 Thejoint density is

on support2 Now transform to polars by setting X1= R cos and X2= R sin Thenthe joint density of R and is given by

−r 2 /2r dr d

on support r

with 12R2

1and R2,

on using inversion, 12R2= − ln R1 or R=−2 ln R1, and = 2R2 Transforming back

to the original Cartesian coordinates gives

X1=−2 ln R1cos 2R2 

X2=−2 ln R1sin 2R2 

The method delivers ‘two for the price of one’ Although it is mathematically correct,

it is not generally used in that form since it can produce fewer than expected tail variates(see Neave, 1973, and Problem 1) For example, using the sine form, an extreme tailvalue of X2 is impossible unless R1 is close to zero and R2 is not However, using a

multiplicative linear congruential generator with a small multiplier will ensure that if R1

is small then so is the next random number R2 Of course, one way to avoid this problem

is to shuffle the output from the uniform generator

Usually a variant known as ‘polar’ Box–Müller is used without the problems concerningtail variates This is now described We recall that X1= R cos and X2= R sin , where1

2R2

the unit circle C≡u v  u2+ v2≤ 1 Then it is obvious that tan−1V/U 

and it is not difficult to show that U2+V2∼ U 0 1 Further, it is intuitive that these tworandom variables are independent (see Problem 2 for a derivation based on the Jacobian

of the transformation) Therefore, tan−1V/U = 2R2and U2+ V2= R1, where R1 and

R2 are random numbers Accordingly, U V  can be taken to be uniformly distributedover the square D= u v  −1 ≤ u ≤ 1 −1 ≤ v ≤ 1 Subject to U2+V2≤ 1 we returntwo independent standard normal variates as

Standard normal distribution 61

4.1.2 An improved envelope rejection method

In Example 3.3 a rejection method was developed for sampling from a folded normal

Let us now see whether the acceptance probability of that method can be improved In

the usual notation for envelope rejection we let

We require g to majorize h, so  and c must be chosen such that x2/2−  x + c ≥

0 ∀ x > c The probability of acceptance is

2 c+ 1/

Therefore, we must minimize c+ 1/ subject to x2/2−  x + c ≥ 0 ∀ x > c Imagine

that c is given Then we must maximize  subject to x2/2−  x + c ≥ 0 ∀ x > c If

2> 2c, that is if  > 2c, then x2/2− x +c < 0 for some x > c Since x2/2− x +

c≥ 0 ∀ x when  ≤ 2c, it follows that the maximizing value of , given c, is  = 2c

This means that we must minimize c+ 1/2c, that is set c =√2/2 and = 2c =√2,

giving an acceptance probability of

√2

Note that this is a modest improvement on the method due to Butcher (1961), where

c= 1 and  = 2 with an acceptance probability of √2/3= 083554, and also on that



1/√

2+ x + 1/√21/√

2+ 21/√2+ 1/√2=

√2x+ 24



x ≤

√22



4/√

2− 1/√2e− 2x +11/√





Applying inversion, given a random number R1, we have



R1≤14





√22R1− 1

1

4 < R1



Given a second random number R2, a prospective variate x is accepted if

1

R2− 1 ≥ − ln R2≥ 1 − R2Let us denote the right-hand side of the inequality in (4.1) by W Suppose that 1−R2> W Then − ln R2> W and the prospective variate is accepted Suppose − ln R2≯ W but1/R2− 1 ≤ W Then − ln R2≤ W and the prospective variate is rejected Only if boththese (inexpensive) pre-tests fail is the decision to accept or reject inconclusive On thosefew occasions we test explicitly for− ln R2> W In the terminology of Marsaglia (1977),the function− ln R2 is squeezed between 1/R2− 1 and 1 − R2

= EYe 2

− 1

Bivariate normal density 63respectively Clearly

PY < y= PX < ln y

= 



ln y− 



=√ 12 ye

2 /2

on support 0

4.3 Bivariate normal density

Suppose X∼ N1 2 and Y ∼ N2 2, and the conditional distribution of Y given

The vector X Y  is said to have a bivariate normal distribution In order to generate

such a vector two independent standard normal variates are needed, Z1 Z2∼ N0 1 Set

x= 1+ 1Z1and (from 4.2)

Later it will be seen how this lower triangular structure for the matrix in the right-hand

side of Equation (4.3) also features when generating n – variate normal vectors, where

n≥ 2

This has the property that =  − 1 − 1  > 1, and  =  − 1! when 

is an integer The notation Z∼ gamma   will be used

The density (4.4) may be reparameterized by setting X= Z Therefore,

fXx=x−1e−x

and X∼ gamma 1 Thus we concentrate on sampling from Equation (4.5) and set

Z= X/ to deliver a variate from Equation (4.4) The density (4.5) is monotonicallydecreasing when ≤ 1, and has a single mode at x =  − 1 when  > 1 The case  = 1describes a negative exponential distribution The density (4.5) therefore represents a

family of distributions and is frequently used when we wish to model a non-negative

random variable that is positively skewed

When  is integer the distribution is known as the special Erlang and there is an

important connection between X and the negative exponential density with mean one Itturns out that

where E1     E are independent random variables with density fE

ix= e−x Since themean and variance of such a negative exponential are both known to be unity, from (4.6)

EX= VarX = and therefore, in terms of the original gamma variate Z,

EZ=

and

VarZ= Var

X





= 

2

Gamma distribution 65

It can easily be shown that these are also the moments for nonintegral  The representation

(4.6) allows variates (for the  integer) to be generated from unit negative exponentials

From Equation (3.2),

X=



i=1

− ln Ri= − ln





i=1

Ri



Result (4.7) is very convenient to implement for small integer , but is clearly inefficient

for large  Also, in that case, we must test that the product of random numbers does not

underflow; that is the result must not be less than the smallest real that can be represented

on the computer However, that will not be a problem with Maple

When  is noninteger a different method is required (which will also be suitable for

integer )

4.4.1 Cheng’s log-logistic method

An algorithm will be constructed to sample from a density proportional to hx where

hx= x −1e−x, x≥ 0 The envelope rejection method of Cheng (1977) will be used,

which samples prospective variates from a distribution with density proportional to gx

where

gx= Kx−1

where = , =√2− 1 when  ≥ 1, and  =  when  < 1 Since g must envelope

h, for the maximum probability of acceptance, K is chosen as

K= maxx≥0

1

4 (at = 0+ to 2/√

= 088623 as  →  (use Stirling’s approximation,  e−1∼

√

2−1−1/2 Note that for  > 1 the efficiency is uniformly high An advantage of

the method, not exhibited by many other gamma generators, is that it can still be used

when  < 1, even though the shape of the gamma distribution is quite different in such

cases (the density is unbounded as x→ 0 and is decreasing in x)

8 6

4 2

Figure 4.1 Gamma density: acceptance probability and shape parameter value

It remains to derive a method for sampling from the prospective distribution FromEquation (4.8) the cumulative distribution function is

Gx=1/− 1/ + x

+ xand so, given a random number R1, the prospective variate is obtained by solving Gx=

1− R1(this leads to a tidier algorithm than R1) Therefore,

2

or, on using Equation (4.10),

ln4R21R2

< −  ln x

+  − x

Beta distribution 67

4.5 Beta distribution

A continuous random variable X is said to have a beta distribution on support (0,1) with

shape parameters  > 0 and  > 0 if its probability density function is

fx=+ x−11− x−1

 0≤ x ≤ 1

A shorthand is X ∼ beta  A method of generating variates x that is easy to

remember and to implement is to set

x= w

w+ ywhere w∼ gamma 1 and y ∼ gamma 1 with w and y independent To show this,

we note that the joint density of w and y is proportional to

e−w+yw−1y−1Therefore the joint density of W and X is proportional to

e−w/xw−1

w1− x

(e.g that described in Section 4.4.1), the method will be reasonably efficient

4.5.1 Beta log-logistic method

This method was derived by Cheng (1978) Let = / and let Y be a random variable

with a density proportional to hy where

hy= y−11+ y−− y≥ 0 (4.11)Then it is easy to show that

X= y

has the required beta density To sample from Equation (4.11) envelope rejection with a

majorizing function g is used where

gy= Ky−1

+ y2 y≥ 0

 =   = min   if min   ≤ 1, and  =2−  − / +  − 2otherwise Now choose the smallest K such that gy

K= maxy>0

v −1dv+ v2

  0

v−1dv+ v2

where Z1     Zn are i.i.d N 0 1 However, that is unlikely to be an efficient way

of generation unless n is small and odd-valued When n is small and even-valued

2= −2 lnR1· · · Rn/2

could be used For all other cases a gamma variate generator isrecommended

A non-central chi-squared random variable arises in the following manner Suppose

Xi∼ N i 1 for i= 1     n, and these random variables are independent Let Y =

n

i=1Xi2 By obtaining the moment generating function of Y it is found that the density

depends only upon n and /2

4.8 Generalized inverse Gaussian distribution

Atkinson (1982) has described this as an enriched family of gamma distributions

The random variable Y ∼ gig    (where gig is a generalized inverse gaussian

distribution) if it has a p.d.f proportional to

y−1exp



−12



y+y



where  > 0 ≥ 0 if  < 0  > 0  > 0 if  = 0, and  ≥ 0  > 0 if  > 0 It is

necessary to consider only the case ≥ 0, since gig −   = 1/gig   Special

cases of note are

gig   0= gamma



2

 2− 2 

, where >  ≥ 0, then the resulting mixture distribution for Wbelongs to the generalized hyperbolic family (Barndorff-Nielson, 1977) Empirical

evidence suggests that this provides a better fit than a normal distribution to the return

(see Section 6.5) on an asset (see, for example, Eberlein, 2001)

Atkinson (1982) and Dagpunar (1989b) devised sampling methods for the generalized

inverse Gaussian However, the following method is extremely simple to implement and

gives good acceptance probabilities over a wide range of parameter values We exclude

the cases = 0 or  = 0, which are gamma and reciprocal gamma variates respectively

Generation of variates from

standard distributions

4.1 Standard normal distribution

The standard normal distribution is so frequently... method is simple to implement and reasonably fast in execution Westart by considering two independent standard normal random variables, X1 and X2 Thejoint density is

14 In a single server queue with Poisson arrivals at rate <  and service durationsthat are independently distributed as negative exponential with mean