Therefore, we primarily focus on methods for computing the transient state probability vector rt for CTM- Cs as defined in Eq.. 2.53 that for the computation of transient state proba- bi
Trang 1Unlike steady-state analysis, CTMCs and DTMCs have to be treated differ- ently while performing transient analysis Surprisingly, not many algorithms exist for the transient analysis of DTMCs Therefore, we primarily focus on methods for computing the transient state probability vector r(t) for CTM-
Cs as defined in Eq (2.53) Furthermore, additional attention is given to the computation of quantities related to transient probabilities such as cumulative measures
Recall from Eq (2.53) that for the computation of transient state proba- bility vector n(t), the following linear differential equation has to be solved, given infinitesimal generator matrix Q and initial probability vector m(O):
d7r (t>
Measures that can be immediately derived from transient state probabil- ities are often referred to as instantaneous measures However, sometimes measures based on cumulative accomplishments during a given period of time
177
Queueing Networks and Markov Chains
Gunter Bolch, Stefan Greiner, Hermann de Meer, Kishor S Trivedi
Copyright 1998 John Wiley & Sons, Inc Print ISBN 0-471-19366-6 Online ISBN 0-471-20058-1
Trang 2178 TRANSIENT SOLUTION OF MARKOV CHAINS
[0, t) could be more relevant Let:
t
L(t) = I 7r(U)dU
We introduce transient analysis with a simple example of a pure birth process
5.1.1 A Pure Birth Process
As for birth-death processes in steady-state case, we derive a transient closed- form solution in this special case Consider the infinite state CTMC depicted
in Fig 5.1 representing a pure birth process with constant birth rate X The number of births, N(t), at time t is defined to be the state of the system The only transitions possible are from state Ic to state k + 1 with rate X Note that this is a non-irreducible Markov chain for any finite value of X, so the steady-state solution does not exist
From Fig 5.1, the infinitesimal generator matrix Q is given by:
Due to the special structure of the generator matrix, it is possible to obtain a closed-form transient solution of this process With generator matrix Q and
Trang 3TRANSIENT ANALYSIS USING EXACT METHODS 179
Eq (2.53), we derive a system of linear differential equations for our example:
With the initial state probabilities:
Applying again elementary differentiation and integration
ent i al equation results in another simple unique solution:
rules to this differ-
If this process is repeated,
state probability i~~k(t):
Trang 4180 TRANSIENT SOLUTION OF MARKOV CHAINS
1
0.8
fig 5.2 Selected Poisson probabilities with parameters X = 0.5 and 1.0
Trang 5TRANSIENT ANALYSIS USING EXACT METHODS
interval [0, t) This measure is computed as:
S = (0, 1) A typical application of this model is to a system subject to failure and repair
The generator matrix of the CTMC of Fig 5.3 is:
Trang 6182 TRANSIENT SOLUTION OF MARKOV CHAINS
Applying the law of total probability, we conclude:
A standard method of analysis for linear differential Eq (5.11) is to use the method of integrating factors [RaBe74] Both sides of the (rearranged) equation are multiplied by the integrating factor:
I-L -e(CL+‘Jt + c = e(P+‘b
1 = r(l) (0) = 1 5 + ce-(P+x)o
=-
Trang 7TRANSIENT ANALYSIS USING EXACT METHODS 183
Thus, unconditioning with an initial pmf 7r(l)(O) yields the corresponding integration constant:
(5.16)
(5.17)
With a closed-form expression of the transient state probabilities given, per- formance measure can be easily derived
Problem 5.3 First, derive expressions for the steady-state probabilities
of the CTMC shown in Fig 5.3 by referring to Eq (2.58) Then, take the limits as t -+ 00 of the transient pmf z(‘)(t) and compare the results to the steady-state case 7r
Problem 5.4 Derive the transient state probabilities of the CMTC shown
in Fig 5.3 assuming initial pmf rrc2)(0) = (0.5,0.5) Take the limit as t + 00
of the resulting transient pmf 7rc2)(t) and compare the results to those gained with 7r(‘)()) when taking the limit
Problem 5.5 Assume the system modeled by the CTMC in Fig 5.3 to be
“up” in state 1 and to be “down” in state 0
(a) Define the reward assignments for a computation of availability measures based on Table 2.1
(b) Derive formulas for the instantaneous availability A(t), the interval unavailability UA( t) , and the steady-state availability A for this model, based both on 7r(‘)(t) and 7rc2)(t) Refer to Section 2.2.2.3.1 for details
on how these availability measures are defined
(c) Let p = 1, X = 0.2 and compute all measures that have been specified
in this problem Evaluate the transient measures at time instants t E {1,2,3,5,10>
Trang 10186 TRANSIENT SOLUTION OF MARKOV CHAINS
computations with small numbers, a left truncation point 1 can be introduced
To this end an overall error tolerance E = ~1 +E, is partitioned to cover both left and right truncation errors Applying again a vector norm, a left truncation point 1 can be determined similarly to the right truncation point r:
(5.25)
In particular, O(a) terms are needed between the left and the right truncation points and the transient DTMC state probability vector Y(Z) must
be computed at the left truncation point 1 The latter operation according
to Eq (5.22) requires O(qt) computation ‘time Thus the overall complexity
of uniformization is 0 (qqt) [ReTr88] (Here q denotes the number of non- zero entries in Q.) To avoid underflow, the method of Fox and Glynn can
be applied in the computation of Z and r [FoG188] Matrix squaring can be exploited for a reduction of computational complexity [ReTr88, AbMa93] But because matrix fill-ins might result, this approach is often limited to models
of medium size (around 500 states)
by the stiffness index qt [ReTr88] Muppala and Trivedi observed that the most time consuming part of uniformization is the iteration to compute y(i)
in Eq (5.22) But this iteration is the main step in the power method for computing the steady-state probabilities of a DTMC as discussed in Chap- ter 2 Now if and when the values of y(i) converge to a stationary value ti, the iteration Eq (5.22) can be terminated resulting in considerable savings
in computation time [MuTr92] We have also seen in Chapter 2 that speed
of convergence to a stationary probability vector is governed by the second largest eigenvalue of the DTMC, but not by qt However, since the proba- bility vectors v(i) are computed iteratively according to the power method, convergence still remains to be effectively determined Because an a priori determination of time of convergence is not feasible, three cases have to be differentiated for a computation of the transient state probability vector z(t)
[MuTr92, MMT94] :
1 Convergence occurs beyond the right truncation point In this case, com- putation of a stationary probability vector is not effective and the tran- sient state probability vector ?r(t) is calculated according to Eq (5.25) without modification
Trang 12188 TRANSIENT SOLUTION OF MARKOV CHAINS
1 Convergence occurs beyond the truncation point, that is, c > r As a consequence, Eq (5.29) remains unaffected and must be evaluated as is
2 If convergence occurs before truncation point, Eq (5.29) is adjusted to:
&9(t) 5 I 2 2 e-G y
q i=r+1j=i+1 ’
5; ,g (i-(r+l))e-qty z=?-+1
Trang 13TRANSIENT ANALYSIS USING EXACT METHODS 189
We briefly discuss numerical methods based on ordinary differential equation approach followed by methods exploiting weak lumpability
of ordinary differential equations (ODE) can be utilized for the numerical solution of the Kolmogorov differential equations of a CTMC Such ODE solution methods discretize the solution interval into a finite number of time intervals {ti, 62, t’ *, Z?“‘? 7% t } The difference between successive time points, called the step size h, can vary from step to step There are two basic types
of ODE solution methods: explicit and implicit
In an explicit method, the solution rr(ti) is approximated based on values 7r(tj) for j < i The computational complexity of explicit methods such
as Runge-Kutta is O(qqt) [ReTr88] Although explicit ODE-based methods generally provide good results for non-stiff models, they are inadequate if stiff models need to be studied Note that stiff CTMCs are commonly encountered
in dependability modeling
In an implicit method, r(ti) is approximated based on values n;TT( tj) for
j 5 i Examples of implicit methods are TR-BDF2 [ReTr88] and implicit Runge-Kutta [MMT94] At each time step a linear system solution is required
in an implicit method The increased overhead is compensated for by bet- ter stability properties and lower computational complexity on stiff models [MMT94, ReTr88]
For non-stiff models, Uniformization is the method of choice, while for mod- erately stiff models, uniformization with steady-state detection is recommend-
ed [MMT94] For extremely stiff models, TR-BDF2 works well if the accuracy required is low (eight decimal digits) For high accuracy on extremely stiff models, implicit Runge-Kutta is recommended [MMT94]
5.1.5.2 Weak Lumpability An alternative method for transient analysis based
on weak lumpability has been introduced by Nicola [NicoSO] Nicola’s method
is the transient counterpart of Takahashi’s steady-state method, which was introduced in Section 4.2 Recall that state lumping is an exact approach for
an analysis of a CMTC with reduced state space and, therefore, with reduced computational requirements State lumping can be a very efficient method for a computation of transient state probabilities, if the lumpability condi- tions apply Note that state lumping can also be orthogonally combined with other computational methods of choice, such as stiff uniformization, to yield
an overall highly accurate and efficient method
For the sake of completeness, we present the basic definitions of weak lumpability without going into further detail here Given conditional proba-
Trang 16192 TRANSIENT SOLUTION OF MARKOV CHAINS
previously described state partition:
Let Qr = Q~r,l 5 I 5 F - 1, denote submatrices containing transition rates between grouped states within fast recurrent subset Sr only Subma- trices AIJ, I # J, contain entries of transitions rates from subset SI to SJ Finally, Ace and AFF collect intra slow states and intra fast transient states transition rates, respectively
Matrices Qr , 1 5 I 5 F - 1, contain at least one fast entry in each row Furthermore, there must be at least one fast entry in one of the matrices AFI, 0 5 I 5 F - 1 if SF # 8 Matrix AFF may contain zero or more fast entries Finally, all other matrices, i.e., Aoo,&J,-~ # J,O L I L F-LO 5
J < F, cant ain only slow entries By definition, only slow transitions are possible among these subsets SI and S J An approximation to r(t) is derived, where n(t) is the solution to the differential Eq (5.36):
The reorganized matrix Q forms the basis to create the macro-state gen- erator matrix X Three steps have to be carried out for this purpose: first, aggregation of the fast recurrent subsets into macro states and the correspond- ing adaptation of the transition rates among macro states and remaining fast transient states, resulting in intermediate generator matrix % In a second step, the fast transient states are eliminated and the transition rates between the remaining slow states are adjusted, yielding the final generator matrix
X Finally, the initial state probability vector ~(0) is condensed into a(O)
as per the aggregation pattern Transient solution of differential Eq (5.37) describing the long-term interactions between macro states is carried out:
$7(t) = tT(t)lx, (5.37)
g(o) = (~Oo(~),~~~,~O,,_,(~),~l(~),~~~,~F-l(~)) a
Once the macro state probability vector o(t) has been computed, disag- gregations can be performed to yield an approximation r”(t) of the complete probability vector n(t)
Each subset of fast recurrent states is analyzed in isolation from the rest of the system by cutting off all slow transitions leading out of the aggregate Col-
lecting all states from such a subset S’I and arranging them together with the
Trang 17AGGREGATION OF STIFF MARKOV CHAINS 193
corresponding entries of the originally given infinitesimal generator matrix Q into a submatrix &I gives rise to the possibility of computing the conditional steady-state probability vectors XT, 1 < I 5 F - 1 Since &I does not, in general, satisfy the properties of an infinitesimal generator matrix, it needs to
be modified to matrix QT:
Q;=QI+DI (5.38) Matrix DI is a diagonal matrix whose entries are the sum of all cut-off transition rates for the corresponding state Note that these rates are, by definition, orders of magnitude smaller than the entries on the diagonal of
Qr The inverse of this quantity is used as a measure of coupling between the subset Sr and the rest of the system
Since Q: is the infinitesimal generator matrix of an ergodic CMTC, the solution of:
yields the desired conditional steady-state probability vector X; for the subset
of fast states SI For each such subset SI, a macro state I is defined The set
of aggregated states {I, 1 5 I 5 F - I}, constructed in this way, together with the initially given slow states Se form the set of macro states niir = SoU{I, 1 5
I 5 F - 1) for which a transient analysis is performed in order to account for long-term effects among the set of all macro states
To create the intermediate generator matrix 2 of size (]1M( + no) x (]A!!] + no), unconditionings and aggregations of the transition rates have to be per- formed The dimensions [no x 7251 of submatrices 21~ are added to the cor- responding equations in what follows Let us define matrices of appropriate size:
E=
1
1
1
containing 1s only as entries The following cases are d
l Transitions remain unchanged among slow states
istinguished:
in Se:
zoo = Aoo, [no x no] * (5.40)
0 nanSitiOnS involving fast transient states from SF:
- Transitions between slow states and fast transient states, and vice versa, remain unchanged:
EFO = AFO, [nF x 7201 7 (5.41)