Handbook of Reliability, Availability, Maintainability and Safety in Engineering Design - Part 22 docx

c Maximum Likelihood Estimation MLE Parameter Estimation The idea behind maximum likelihood parameter estimation is to determine the pa-rameters that maximise the probability likelihood

a j and x j = regression parameters and covariates,

β andμ = the shape and scale parameters

It is often more convenient to define an additional covariate, xo= 1, in order to allow the Weibull scale parameter to be included in the vector of regression coefficients, and the proportional hazards model expressed solely by the beta (shape parameter), together with the regression parameters and covariates The PH failure rate can then

be written as

λ(t,X) =β(t)β−1exp

m

∑

j=0

a j x j

The PH reliability function is thus given by the expression

R (t,X) = exp

⎡

⎣−

t

 0

λ(u)du

⎤

⎦

R (t,X) = exp

⎡

⎣−

t

 0

λ(u,X)du

⎤

⎦

R (t,X) = exp

−tβ· exp

m

∑

j=0

a j x j

(3.150)

The probability density function (p.d.f.) can be obtained by taking the partial deriva-tive with respect to time of the reliability function given by Eq (3.150) The PH

probability density function is given by the expression f (t,X) =λ(t,X)R(t,X) The

total number of unknowns to solve in this model is m+2 (i.e.β,μ,a1,a2,a3, ,a m)

The maximum likelihood estimation method can be used to determine these

pa-rameters Solving for the parameters that maximise the maximum likelihood esti-mation will yield the parameters for the PH Weibull model Forβ= 1, the equation then becomes the likelihood function for the PH exponential model, which is similar

to the original form of the proportional hazards model proposed by Cox (1972)

c) Maximum Likelihood Estimation (MLE) Parameter Estimation

The idea behind maximum likelihood parameter estimation is to determine the pa-rameters that maximise the probability (likelihood) of the sample data From a sta-tistical point of view, the method of maximum likelihood is considered to be more robust (with some exceptions) and yields estimators with good statistical proper-ties In other words, MLE methods are versatile and apply to most models and to different types of data In addition, they provide efficient methods for quantifying uncertainty through confidence bounds Although the methodology for maximum likelihood estimation is simple, the implementation is mathematically complex By utilising computerised models, however, the mathematical complexity of MLE is not an obstacle

Asymptotic behaviour In many cases, estimation is performed using a set of

in-dependent, identically distributed measurements In such cases, it is of interest to determine the behaviour of a given estimator as the set of measurements increases

to infinity, referred to as asymptotic behaviour Under certain conditions, the MLE

exhibits several characteristics that can be interpreted to mean it is ‘asymptotically optimal’ While these asymptotic properties become strictly true only in the limit

of infinite sample size, in practice they are often assumed to be approximately true, especially with a large sample size In particular, inference about the estimated pa-rameters is often based on the asymptotic Gaussian distribution of the MLE

As MLE can generally be applied to failure-related sample data that are available

for critical components during the detail design phase of the engineering design

process, it is necessary to examine more closely the theory that underlies maximum

likelihood estimation for the quantification of complete data Alternately, when no data are available, the method of qualitative parameter estimation becomes

essen-tial, as considered in detail later in Section 3.3.3.3

Background theory If x is a continuous random variable with probability density

function:

f (x;θ1,θ2,θ3, ,θk ) ,

where:

θ1,θ2,θ3, ,θk are k unknown and constant parameters that need to be estimated

through n independent observations, x1,x2,x3, ,x n Then, the likelihood function is given by the following expression

L (x1,x2,x3, ,x n) =∏n

i=1

f (x i;θ1,θ2,θ3, ,θk ) i = 1,2,3, ,n (3.151) The logarithmic likelihood function is given by

Λ= lnL =∑n

i=1

ln f (xi;θ1,θ2,θ3, ,θk ) (3.152)

The maximum likelihood estimators (MLE) of θ1,θ2,θ3, ,θk are obtained by maximisingΛ By maximisingΛ, which is much easier to work with than L, the

maximum likelihood estimators (MLE) of the rangeθ1,θ2,θ3, ,θkare the

simul-taneous solutions of k equations where the partial derivatives ofΛare equal to zero:

∂(Λ)

∂θj = 0 j = 1,2,3, ,k Even though it is common practice to plot the MLE solutions using median ranks

(points are plotted according to median ranks and the line according to the MLE so-lutions), this method is not completely accurate As can be seen from the equations above, the MLE method is independent of any kind of ranks or plotting methods For this reason, the MLE solution appears many times not to track the data on a

prob-ability plot This is perfectly acceptable, since the two methods are independent of each other

Illustrating the MLE Method Using the Exponential Distribution:

To estimateλ, for a sample of n units (all tested to failure), the likelihood function

is obtained

L(λ|t1,t2,t3, ,t n) =∏n

i=1

f (t i)

=∏n

i=1λe−λt i

Taking the natural log of both sides

Λ= ln(L) = nln(λ ) −λ∑n

i=1

t i

∂(Λ)

∂λ =

n

λ −

n

∑

i=1

t i= 0 Solving forλ gives:

λ= n/∑n

i=1

Notes on Lambda

The value ofλ is an estimate because, if another sample from the same

popula-tion is obtained andλ re-estimated, then the new value would differ from the one previously calculated

How close is the value of the estimate to the true value? To answer this ques-tion, one must first determine the distribution of the parameterλ This methodology

introduces another term, the confidence level, which allows for the specification of

a range for the estimate with a certain confidence level The treatment of confidence

intervals is integral to reliability engineering, and to statistics in general

Illustrating the MLE Method Using the Normal Distribution

To obtain the MLE estimates for the mean, T, and standard deviation,σT, for the normal distribution, the probability density function of the normal distribution is

given by

F (T) = 1

σT

√

2πexp

−12(T − T)2

σT

where:

T = mean of the normal distribution,

σT = standard deviation of the normal distribution

If T1,T2,T3, ,T n are known times to failure (and with no suspensions), then the

likelihood function is given by

L (T1,T2,T3, ,T n |T,σT) :

L=∏n

i=1

 1

σT

√

2πexp

−

1

2(T − T)2

σT



L=  1

σT

√

2πnexp

−1

2

n

∑

i=1

(T i − T)2

σT

(3.156)

Λ = ln(L):

ln(L) = − n

2ln(2π) − nlnσT −1

2

n

∑

i=1

(T i − T)2

σT

Then, taking the partial derivatives ofΛwith respect to each one of the parameters, and setting these equal to zero yields:

∂(Λ)

∂T = 1

σ2−∑n

i=1

(T i − T) = 0

and:

∂(Λ)

∂σT = n

σT + 1

σ3

n

∑

i=1

(T i − T)2= 0.

Solving these equations simultaneously yields

T =1

n

n

∑

i=1

σ2

T =1

n

n

∑

These solutions are valid only for data with no suspensions, i.e all units are tested

to failure In cases in which suspensions are present, the methodology changes and the problem becomes much more complicated

Estimator As indicated, the parameters obtained from maximising the likelihood

function are estimators of the true value It is clear that the sample size determines the accuracy of an estimator If the sample size equals the whole population, then the

estimator is the true value Estimators have properties such as non-bias and

consis-tency (as well as properties of sufficiency and efficiency, which are not considered here)

Unbiased estimator An estimator given by the relationshipθ= d(x1,x2,x3, ,x n)

is considered to be unbiased if and only if the estimator satisfies the condition

E(θ) =θ for all θ In this case, E (x) denotes the expected value of x and is

de-fined by the following expression for continuous distributions

E (x) =

ψ



This implies that the true value is not consistently underestimated nor overestimated

Consistent estimator An unbiased estimator that converges more closely to the

true value as the sample size increases is called a consistent estimator The standard

deviation of the normal distribution was obtained using MLE However, this estima-tor of the true standard deviation is a biased one It can be shown that the consistent estimate of the variance and standard deviation for complete data (for the normal distribution) is given by

σ2

n − 1

n

∑

i=1(T i − T)2. (3.160)

Analysis of censored data So far, parameter estimation has been considered for

complete data only Further expansion on the maximum likelihood parameter esti-mation method needs to include estimating parameters with right censored data The method is based on the same principles covered previously, but modified to take into account the fact that some of the data are censored

MLE analysis of right censored data The maximum likelihood method is by far

the most appropriate analysis method for censored data When performing maxi-mum likelihood analysis, the likelihood function needs to be expanded to take into account the suspended items A great advantage of using MLE when dealing with censored data is that each suspension term is included in the likelihood function Thus, the estimates of the parameters are obtained from consideration of the entire sample population of tested components Using MLE properties, confidence bounds can be obtained that also account for all the suspension terms In the case of

sus-pensions, and where x is a continuous random variable with p.d.f and c.d.f of the

following forms

f (x;θ1,θ2,θ3, ,θk)

F (x;θ1,θ2,θ3, ,θk)

θ1,θ2,θ3, ,θk are the k unknown parameters that need to be estimated from

R failures at (T1,V T1),(T2,V T2),(T3,V T3), ,(T R ,V T R ), and from M suspensions at (S1,V S1),(S2,V S2),(S3,V S3), ,(S M ,V SM ), where V T R is the Rth stress level corre-sponding to the Rth observed failure, and V SM the Mth stress level corresponding to the Mth observed suspension.

The likelihood function is then formulated, and the parameters solved by max-imising

L ((T1,V T1), ,(T R ,V TR ),(S1,V S1), ,(S M ,V SM )|θ1,θ2,θ3, ,θk) =

R

∏

i=1

f (T i ,V Ti;θ1,θ2,θ3, ,θk)∏M

j=1



1− F(S j ,V Sj;θ1,θ2,θ3, ,θk) (3.161)

3.3.3.2 Expansion of the Exponential Failure Distribution

Estimating failure rate As indicated previously in Section 3.2.3.2, the

exponen-tial distribution is a very commonly used distribution in reliability engineering Due

to its simplicity, it has been widely employed in designing for reliability The ex-ponential distribution describes components with a single parameter, the constant failure rate The single-parameter exponential probability density function is given

by

f (T ) =λe−λT = (1/MTBF)e −T/MTBF (3.162) This distribution requires the estimation of only one parameter,λ, for its application

in designing for reliability, where:

λ = constant failure rate,

MTBF= mean time between failures, or to a failure,

MTBF> 0,

T = operating time, life or age, in hours, cycles, etc

There are several methods for estimatingλin the single-parameter exponential fail-ure distribution In designing for reliability, however, it is important to first under-stand some of its statistical properties

a) Characteristics of the One-Parameter Exponential Distribution

The statistical characteristics of the one-parameter exponential distribution are bet-ter understood by examining its paramebet-ter,λ, and the effect that this parameter has

on the exponential probability density function as well as the reliability function Effects ofλ on the probability density function:

• The scale parameter is 1/λ= m The only parameter it has is the failure rate,λ

• Asλ is decreased in value, the distribution is stretched to the right

• This distribution has no shape parameter because it has only one shape, i.e the

exponential

• The distribution starts at T = 0 where f (T = 0) =λ and decreases exponentially

as T increases (Fig 3.34), and is convex as T →∞, f (T ) → 0.

• This probability density function (p.d.f.) can be thought of as a special case of

the Weibull probability density function withβ= 1

Fig 3.34 Effects ofλ on the probability density function

Fig 3.35 Effects ofλ on the reliability function

Effects ofλ on the reliability function:

• The failure rate of the function is represented by the parameterλ

• The failure rate of the reliability function is constant (Fig 3.35).

• The one-parameter exponential reliability function starts at the value of 1 at

T= 0

• As T →∞, R (T ) → 0.

b) Estimating the Parameter of the Exponential Distribution

The parameter of the exponential distribution can be estimated graphically by

prob-ability plotting or analytically by either least squares or maximum likelihood.

Probability plotting The graphical method of estimating the parameter of the

ex-ponential distribution is by probability plotting, illustrated in the following exam-ple

Estimating the parameter of the exponential distribution with probability plot-ting Assume six identical units have pilot reliability test results at the same

ap-plication and operation stress levels All of these units appear to have failed after operating for the following testing periods, measured in hours: 96, 257, 498, 763, 1,051 and 1,744 Steps for estimating the parameter of the exponential probability density function, using probability plotting, are as follows (Table 3.22)

The times to failure are sorted from small to large values, and median rank per-centages calculated Median rank positions are used instead of other ranking meth-ods because median ranks are at a specific confidence level (50%) Exponential

probability plots use scalar data arranged in rank order for the x-axis of the prob-ability plot The y-axis plot is found from a statistical technique, Benard’s median

rank position (Abernethy 1992)

Determining the X and Y positions of the plot points The points plotted

repre-sent times-to-failure data in reliability analysis For example, the times to failure

in Table 3.22 would be used as the x values or time values Determining what the appropriate y plot position, or the unreliability values should be is a little more complex To determine the y plot positions, a value indicating the corresponding

Table 3.22 Median rank table for failure test results

Time to failure Failure order number Median rank

unreliability for that failure must first be determined In other words, the cumula-tive percent failed must be obtained for each time to failure In the example, the cumulative percent failed by 96 h is 17%, by 257 h 34% and so forth This is a sim-ple method illustrating the concept The problem with this method is that the 100% point is not defined on most probability plots Thus, an alternative and more robust

approach must be used, such as the method of obtaining the median rank for each

failure

Method of median ranks Median ranks are used to obtain an estimate of the

un-reliability, U (T j), for each failure It is the value that the true probability of failure,

Q (T j ), should have at the jth failure out of a sample of N components, at a 50% con-fidence level This essentially means that this is a best estimate for the unreliability:

half of the time the true value will be greater than the 50% confidence estimate, while the other half of the time the true value will be less than the estimate The estimate is then based on a solution of the binomial distribution

The rank can be found for any percentage point, P, greater than zero and less than one, by solving the cumulative binomial distribution for Z This represents the rank, or unreliability estimate, for the jth failure in the following equation for the

cumulative binomial distribution

P=∑N

k = j

(N k )Z k (1 − Z) N −k , (3.163) where:

N= the sample size,

j = the order number

The median rank is obtained by solving for Z at P = 0.50 in

0.50 =∑N

k = j (N k )Z k (1 − Z) N −k (3.164)

For example, if N= 6 and we have six failures, then the median rank equation would

be solved six times, once for each failure with j = 1,2,3,4,5 and 6, for the value

of Z This result can then be used as the unreliability estimate for each failure, or the

y plotting position The solution of Eq (3.164) for Z requires the use of numerical

methods A quick though less accurate approximation of the median ranks is given

by the following expression This approximation of the median ranks is known as Benard’s approximation (Abernethy 1992):

MR= j − 0.3

For the six failures in Table 3.22, the following values are equated (Table 3.23):

Table 3.23 Median rank table for Bernard’s approximation

Failure order number Bernard’s approximation (×10 ư2) Binomial equation Error margin

Kaplan–Meier estimator The Kaplan–Meier estimator is used as an alternative

to the median ranks method for calculating the estimates of the unreliability for probability plotting purposes

F (t i ) = 1 ư∏i

j=1

n j ư r j

where:

i = 1,2,3, ,m,

m= total number of data points,

n = total number of units

and:

n i=i∑ư1

j=0

S j ư i∑ư1

j=0

R j ,

where:

i = 1,2,3, ,m,

R j = number of failures in the jth data group,

S j = number of surviving units in the jth data group.

The exponential probability graph is based on a log-linear scale, as illustrated in

Fig 3.36 The best possible straight line is drawn that goes through the t= 0 and

R (t) = 100% point, and through the plotted points on the x-axis and their corre-sponding rank values on the y-axis A horizontal line is drawn at the ordinate point

Q (t) = 63.2% or at the point R(t) = 36.8%, until this line intersects the fitted straight

line A vertical line is then drawn through this intersection until it crosses the ab-scissa The value at the abscissa is the estimate of the mean

For this example, MTBF= 833 h, which means thatλ = 1/MTBF = 0.0012 This is always at 63.2%, since Q (T) = 1 ư e ư1 = 63.2%.

The reliability value for any mission or operational time t can be obtained For

example, the reliability for an operational duration of 1,200 h can now be obtained.

To obtain the value from the plot, a vertical line is drawn from the abscissa, at

t = 1,200 h, to the fitted line A horizontal line from this intersection to the ordinate

is drawn and R (t) obtained This value can also be obtained analytically from the exponential reliability function In this case, R (t) = 98.15% where R(t) = 1ưU and

U = 1.85% at t = 1,200.

in- dependent, identically distributed measurements In such cases, it is of interest to determine the behaviour of a given estimator as the set of measurements increases... with a certain confidence level The treatment of confidence

intervals is integral to reliability engineering, and to statistics in general

Illustrating the MLE Method Using the...

T = operating time, life or age, in hours, cycles, etc

There are several methods for estimating? ?in the single-parameter exponential fail-ure distribution In designing for reliability,