First, we have the following theorem:
Theorem 2.5. (/16]) Let € be a discrete random variable, F(z) the c.d.f of §, z€ Z+, and P the set of all PLEP’s of €. Let F(x) be a smooth function, x € Ri, such that F(z) = F(z) when2 € Z,. Then
Pe Z,={c eZ", | F(z) > ph. (2.19)
Proof. Let Zp = {y € Zs | P(€ < y) > p}, where P is the distribution function of
€. Then P © Zy. Since the values of F(z) and F(a) coincide at the lattice points,
Zy = Zp. n
14
Let € be an r-component random vector, and ? the set of all PLEP’s of €. Let
F(w;A) Ự TT , ) = ^ Tí/+1) Ty 1£ ung T, , > —Ì, 1
I 2¥(1—2)"-9-! dr
———————. -l<ÿ<n, Jọ z(1 — z)"—~v~ldz
F(y;,n,p') =
and
G@;À)=1—e %, u>0
where ['{-) is the gamma function and ứ € (0, 1). Let
2; ={u+1<Z: | lẽƑ@:^:) >ằ. ÀĂ>0, i=1,...,r}, i=1
r
Z2 ={u+1eZr | [] F@iini vi) 2p, ứ; € (0, 1), —] < tị < Thị, i=1,...,r}
i=1
and
r 1
2g ={u<Z | ]] 26:5) >p ys > 0, A= Ing ¿=1,...,r}.
Then we have the following Corollary:
Corollary 2.1. (a) If all components of € have independent Poisson distribution with parameters 1,A2,...,Ar, respectively, then P C 2z;
(b) Tƒ all components oƒ € haue independent binomial distribution with parameters (m,1), (nạ, Ð>),..., (n„, py), respectively. Then P C 29;
(c) If all components of € have independent geometric distribution with parameters D1, P2,--++,Pr, respectively. Then P © Ze.
Proof. The proof can he directly derived from Theorem 2.19. O
Let ,
5; ={y+1eR2 | |][FŒ:À) >p, Ài>0, ¿=1,...,r},
i=1 T
Z2; ={Uu+1eRT | ]]Ƒ 6m.) > pị ứ, € (0, 1), —] < ⁄ < ?Ă, ¿=1,...,r}
i=1
15
and
„
ZZ = {ye Rt? | [[ Gir) > Đ, yi > 0, =n i=1,...,r}.
t=1 1 — pj’
From Theorem 2.1, Theorem 2.3 and c.d.f. of exponential distribution is logconcave, the three above sets are all convex. From Theorem 2.5, for a multivariate random vector €, if all the components of € have independent Poisson, binomial or geometric distribution, then all the PLEP’s of € are contained in a convex set, which is obtained from incomplete gamma function, incomplete beta function or exponential distribution function, respectively.
So for problem (2.2), if all components of € have independent Poisson, or binomial or geometric distribution, we can get the corresponding relaxed convex programming problem as shown in (2.5), (2.15) and (2.18), respectively.
2.5 Local search method
From the optimal solutions of the relaxed problems, we use a direct search method to find the optimal solutions of the discrete optimization problems. The method is based on Hooke and Jeeves search method (Hook and Jeeves, 1961) and in each step, we have to check the feasibility of the new trial point. To state the method, without loss of generality, we simplify problem (2.2) in the following way:
min clr
subject to 7z >1, t¿¡ € 2
Ar>b (2.20)
[h-¡ Fis) = z€2+.
Let z be the optimal solution of problem (2.1) and z* = |z]. Let ƒ(z) = eŸz and
D={z| || (+) >p, Ar >b, xe Z.} ,
¡=1 where T; is the ?-th row vector of 7T,
16
The modified Hooke and Jeeves direct search method is as follows. In each searching step, it comprises two kinds of moves: Exploratory and Pattern. Let Aa, be the step length in each of the directions e;, i = 1,2,...,7.
The Method Exploratory move
Step 0: Set i = 1 and compute F = f(x*) where z* = |z] =
(11,23,..., #z).
Step 1: Set ứ := (Z1,Z2,...,¿ + A1,...., đr).
Step 2: If f(z) < F and z € D then set F = f(x), i:=i+1; Goto Step 1.
Step 3: If f(z) > F and x € D then set x := (21,2%9,...,2; — 2Aqg;,...,0,). If f(x) < F and x € D, the new trial point is retained.
Set F = ƒ(z), ¿:= ¿+ 1, and goto Step 1.
If f(z) > F then the move is rejected, +; remains unchanged. Set i:=%+1 and goto Step 1.
Pattern move
Step 1: Set ô = 2? + (x? — @8), where x? is the point arrived by the Exploratory moves, and #° is a point which is also arrived by exploratory move in previous step where x? is obtained from the exploratory move starting from 7°.
Step 2: Starts the Exploratory move. If for the point z obtained by the Exploratory moves f(x) < f(x?) and x € D, then the pattern move is recommended. Otherwise x? is the starting point and the process restarts from 2.
Remark When we consider the discrete random variables which have Poisson, binomial or geometric distributions, we set Az; = 1.
A simple two-dimensional Hooke and Jeeves search method is illustrated in the
17
2 3
1 2
4 7
5 — la 16 15 8 14
17 T
18 11 7 10
13
T1
Figure 2.1: An illustration of two-dimensional search of Hooke and Jeeves following figure 2.1 [15]:
In figure 2.1, the points labeled numbers according to the sequence are selected. x!
is a starting base. After x® failed, and #2 and x4 are successes, then the new base x4 is obtained by the Exploratory moves, where f(r*) < f(x'). z® is obtained from Pattern move. From 2°, 2° is a base if f(x2°) < f(x*) after the Exploratory moves. We repeat these steps, finally we reach zứ!3, If ƒ(z!3) > ƒ(zŠ), we have to return to #Š, which is a starting base and performance this procedure with reduced step lengths.
18
2.6 Probability maximization under constraints Now we consider problem (1.1) and the following problem together:
max P(Tr > &)
subject to cax<K
(2.21) Ar >b
z>0,
where € is a random vector and K is fixed number. In [27], the relations between problem (1.1) and (2.21) are discussed.
Suppose the components of random vector € are independent, then the objective
function of problem (2.21) is h(x) = []j_, Fi(yi), where Tx = y. Since F;(y;) > 0, we take natural logarithm of h(x) and problem (2.21) can be written in the following form:
max lnh(z)
subject to cla <K (2.22)
If € is a Poisson random vector, problem (2.22) can be approximated by solving the following problem:
max 3”; ¡ln (: — Ta Se the*dt)
subject to Tx =y
Az >b (2.23)
ca <K xz>0.
From Theorem 2.1, the objective function of problem (2.23) is concave. Let z be the optimal solution of problem (2.22) and x* = |x]. Then we apply the modified Hooke and Jeeves search method to search the optimal solution of problem (2.21) around 2*
as described above, and D is replaced by
D={x|c?<K, Av>b, rE Zs}
19
and” <” and” <” are replaced by ” >” and” > ”, respectively. A numerical example in Section illustrates the details of this procedure.
For the case of independent binomial and geometric random variables, it can be discussed in a similar way.
20
Chapter 3
Inequalities for the joint probability distribution of partial sums of independent random variables
In this chapter, we consider the joint probability distribution of partial sums of inde pendent random variables. We assume that the r-h.s. random variables of problem (1.1) are partial sums of independent ones where all of them are either Poisson or binomial or geometric with arbitrary parameters. The probability that a joint constraint of one of these types is satisfied is shown to be bounded from below by the product of the probabilities of the individual constraints. The probabilistic constraint is imposed on the lower bound. Then smooth logconcave c.d.f’s are fitted to the univariate discrete c.d.f’s and the continuous problem can be solved numerically.
For the proof of our main theorems in this chapter, we need the following
Lemma 3.1. (/16]/) LetO0<p, <1, g=1-p, a > a1, bạ >bị,..., zọ > 24, then we have the inequality
paobo - - - Z0 + gatbg - - -Z\ > (pao + gai)(pbo + ghi)... (p20 + gz1).
Proof. We prove the assertion by induction. For the case of ao > aj, bo > bi, the assertion is
pagby + qaib, > (pag + ga1)(pbo + gb1).
This is easily seen to be the same as
pq(ao — a1)(bạ — bị) > 0.
21
which holds true, by assumption. Looking at the general case, we can write
pao (bo - - - zo) + ga (b4 - - - Z1)
> (pag + gay) (pbo - - zo + đÙI - - - Z1)
> (pao + qga1)(pbo + qbì) (pco - - - Z0 + đề - - - 21)...
> (pag + qaz)(pbo + gbi) ... (pzo + 921).
Thus the lemma is proved. O
Let A = (a; 4) #0 be an mxr matrix with 0—1 entries and X;,...,X; independent, 0 - 1 valued not necessarily identically distributed random variables. Consider the transformed random variables
r
¥i= So ain Xk, i=1,...,m. (3.1)
=1 We prove the following
Theorem 3.1. (/16/) For any nonnegative integers y1,...,Ym we have the inequality
P( <Sựi,'--.Ym < m) > |][ PƠi < 1ì. m™m (3.2)
¿=1
Proof. Let ẽ = {¿ | œ\ = 1}, ẽ = {1,...,m}\ T, pịỡ = P(X: = 0), g =1— pị. Then
we have the relation
P(Y; Su, ?=l,--- ,m)
Tr Tr
= P So 049 X; Si ied, ằ- <yi, iel (3.3)
j=l j=2
r r
= P So a8 jp X5 Sys, ¡€1 So aig Xj < Đụ, ieI|n
j=2 j=2
Tr T
+P À 2 ajjX; <i —, ted, So aig Xj Sv, tel) aq.
j=2 j=2
We prove the assertion by induction on r. Let 1; = mìn;c; y; and look at the case r = 1.
22
We have that
PY; Sys, t= 1,---,m)
= P(XI <<, ¡€])
= P(X, S min yi) = P(X; <y;)
= P(Y; <y;)
t=1
IV
then the assertion holds for the case. Assume that it holds for r—1. Then, using (3.3) and Lemma 3.1, we can write
PCY; Sy, t=1,--- ,m)
r „
lỊP So aig X; < yi I]? So aig Xj < yi Pl 2
i€l j=2 ier =2
r Tr
+]Ị? So aigXj Si -1 [[2 So aig X; < yi 1
¡€l j=2 hủ j=2
r Tr
> ]]|P|} ;%¿X; Su |m+P |3 s¿ÄX; Su 1|
tel j=2 j=2
Ty
H P > dịjÃj S ier j=2
r Tr
= ][P|} %¿X; <u | TPP |S asx; <u
tel j=l Ăcẽ J=2
m Tr
= [[P (do asx; < yi
i=1 \j=1
= | Pữi <0. trì
¿=1
This prove the theorem. O
Theorem 3.2. (/16]) Let X1,...,X, be independent, binomially distributed random variables with parameter (n1,pi),.--, (Mr, Pr), respectively. Then for the random vari- ables (3.1), then inequality (3.2) holds true.
Proof. The assertion is an immediate consequence of Theorem 3.1. O
23
Note that in case of Theorem 3.2, the random variables ¥;, 7 = 1,...,r are not necessarily binomially distributed. They are, however, if pj =... = pr.
Theorem 3.3. (/16]) Let X1,...,X, be independent, Poisson distributed random vari- ables with parameter ằ1,..., Ar, respectively. Then for the random variables (3.1), then inequality (3.2) holds true.
Proof. If in Theorem 3.2, we let n; — oo, py; — 00 such that njpj — À¿, ý = 1,...,7,
then the assertion follows from (3.2). Oo
In case of Theorem 3.3, the random variables Y;, 1 = 1,...,m have Poisson distri- bution with parameter )7j_, ai,nA;, i = 1,...m, respectively.
Theorem 3.3 obviously remain true if X;,...,X; are independent random variables such that the c.d.f of X; can be obtained as a suitable limit of the c.d.f of binomial distri- butions, i = 1,...,r. Since m-variate normal distribution with nonnegative correlations can be obtained this way as the joint distribution of the random variables Y1,...,Ym.
In this case, inequality (3.2) is a special case of the well-known Slepian-inequality (see Slepian, 1962).
24
Chapter 4 Numerical examples
In [7], Dentcheva, Prékopa and Ruszczyiski presented an algorithm to solve a stochastic programming problem with independent random variables. In this Chapter, we com- pare the optimal values and solutions by using DPR algorithm and the approximation method to a vehicle routing problem in [7]. Also a stochastic network design problem is solved by using the new method.
The DPR algorithm is as follows:
Method
Step 0: Select a p-level efficient point v. Set Jo = {0}, k = 0.
Step 1: Solve the master problem
min cls subject to Ax >b
Tr > Dyes, 490 (4.1)
ied Aj =1 z>0,À>0.
Let u* be the vector of simplex multipliers associated with the second constraint of (4.1).
Step 2: Calculate an upper bound for the dual functional:
d(u®) = (u*) min (u iv mi kyT,,(9) ,
Step 3: Find a p-efficient solution v‘*+”) of the subproblem:
min (u*)?z
zcZp
25
and calculate
d(u*) = (+1) TuE,
Step 4: If đ(u°) = d(uŸ) then stop; otherwise set j¿¿¡ = J U {k + 1}, increase k by one and go to Step 1.
4.1 A vehicle routing example
Consider the vehicle routing problem in [7], which is a stochastic programming prob- lem with independent Poisson random variables, and the constraints have prescribed probability 0.9.
We consider a directed graph with node set NV’ and arc set €. A set of cyclic routes II, which are the sequences of nodes connected with arcs and such that the last node of the sequence is identical with the first one. For each e € €, denote R(e) the set of routes containing e, and c(7) is denoted as the unit cost on the route.
A random integer demand €(e) is associated with each arc e € €. The object is to find the nonnegative integers x(m), 7 € II, such that
P| }` a(n) > Ee), e€ E| 2p,
TER(e)
and the cost }° <7 c(7)2(m) is minimized. So the problem is following:
min Oren c(7)2(m)
subject to P (Ezô=ứ z(m) >€(e), e†Ê) >p (4.2)
#{m) > 0, integer.
Now we consider the following graph shown in Figure 4.1. Each arc in this figure represents two arcs in opposite directions.
Assume that the demands €(e) associated with arcs are independent Poisson random variables with the expected values listed in Table 4.1.
©)
Figure 4.1: The graph of the vehicle routing problem
Arc | Expected Demand
AB 2
AC AD AE BA BC CA CB CD DA DC DE EA
ED G2 892 G2 Hà B2 HF NYFF NY bò G2
Table 4.1: Expected demands
27
To approximate the optimal solution of (4.2), we formulated it as follows:
min cox subject to Tr=y
16 À¿ „y —
2, In G — TFT) J tre ‘at) > Inp
xz > 0, integer,
(4.3)
where
c = (10 15 18 15 32 32 57 57 60 60 63 63 61 61 75 75 62 62 44),
A=(2322112142432 3),
p= 0.9 and
re CC C C = CC CC CC CC oO CC CC c CC CC CC OD CC CC CC CC CC + ơ ơ CC CC oO c pe CC CC oO Oo CC c =
c
CC CC oD OF OD Oe oO FP 0 CC CC CC CC CC CC = CC CC 02 Fe CC = CC Oo a oo ao oO oc oO oO c CC œ œ
c
oO c CC
© CC CC CC CC CC CC CC
Fe Oo CC CC CC CC — So Oo oOo oOo Oo ơ oOo ơ đ CC CC CC CC CC
10 0 1 01
© CC CC CC CC CC
CC C CC CC CC CC © CC CC CC © CC CC CC CC CC 0 © CC CC CC CC CC © CC CC CC © CC CC © CC CC C © co Oo CC —
0 0 1 0 01010
The optimal solution of problem (4.2) obtained from DPR algorithm is
£=(2360000000000000447)7
and the optimal value 977 is reached at the following 0.9-level efficient point
0=(67675474868777)1.
28
By solving problem (4.3), the optimal value is 972.5315, which is reached at
a = (1.7869, 3.0314, 5.8495, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
0.0000, 0.0000, 0.0000, 0.0000, 3.9970, 4.1492, 6.7917)
Let 2* = |x], which is exactly . By using the modified Hooke and Jeeves searching method to search around +*, the optimal solution is remained at 2”, i.e.,
a* =(2360000000000000447)!.
Problem (4.3) is solved by using MatLab 6, and the time is 5 seconds in a PIII-750 CPU computer.
Now we reconsider the vehicle routing problem, suppose we have a budget $1, 000K, and we want to maximize the probability of vehicle routing. Then the problem is formulated in the following way:
max P (Eze=e z(m) > €(e), e€ £)
subject to c{z)z{(z) < 1000, x € TI (4.4) z(m) > 0, integer,
and we use the following formulation to approximate the optimal solution of (4.4):
16 À sự „—
max S), ¡In (1 — Toy fo‘ te ‘at.
subject to Tr=y
(4.5) cf+ < 1000
z >0 integer.
The optimal solution to problem (4.5) is
z = (1.8475, 3.1173, 6.0080, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
0.0000, 0.0000, 0.0000, 0.0000, 4.1117, 4.2633, 6.9857)".
The optima] probability p at this point is 0.9188 and the cost reaches the $1000K. Let
z*=(2360000000000000447)7.
29
We apply the modified Hooke and Jeeves searching method.
Exploratory move
Step 0: 2° = (2360000000000000447)!, p=0.9017 and cfz? = 977.
Step 1: z'=(3360000000000000447)”, p = 0.9049 and cfz! = 987.
Step 2: z2 = (3460000000000000447)7, p = 0.9131 and c’ x? = 1002, which is great than 1000. so z? is rejected. Then check z3 = (3260000000000000447)7,
p = 0.8821 and cfz? = 972. Since the probability at zẺ is less than the probability at x!, we do not accept 2°.
Pattern move
Step 1: Let 24 = 2z!-— 2°. Then zg = (4360000000000000447)?.
Step 2: Start the exploratory moves from ôz+. First check
z® = (5360000000000000447)", which is not in D, the set of feasible solution, because c? 2° = 1007 > 1000. Then try 2° = x4 = (4360000000000000447)’.
At 2°, p = 0.9057 and efzổ = 997 < 1000. Also the probability at ôđ is the greatest one in all the tested feasible points. Repeat the procedure from 2°, finally the procedure stops at 2°. So the optimal solution to problem (4.4) is
z=(4360000000000000447)7, and the optimal value is p = 0.9057 and the cost is $997K.