Volume 17 - Nondestructive Evaluation and Quality Control Part 19 ppt

A mathematically equivalent form of the log odds PODa model is given by Ref 8: Eq 4 In this form, μ= ln a0.5, where a0.5 is the flaw size that is detected 50% of the time, that is, the

Fig 2 Schematic of distribution of detection probabilities for cracks of fixed

length

Equation 1 implies that the POD(a) function is the curve through the averages of the individual density functions of the

detection probabilities This curve is the regression equation and provides the basis for testing assumptions about the

applicability of various POD(a) models In Ref 4, seven different functional forms were tested for applicability to

available POD data, and it was concluded that the log-logistics (log odds) function best modeled the data and provided an acceptable model for the data sets of the study Note that the log odds model is commonly used in the analysis of binary (hit/miss) data because of its analytical tractability and its close agreement with the cumulative log normal distribution (Ref 8)

Two mathematically equivalent forms of the log odds model have subsequently been used The earliest form is given by:

(Eq 2)

This parametrization can also be expressed as:

(Eq 3)

In the Eq 3 form, the log of the odds of the probability of detection (the left-hand side of Eq 3) is expressed as a linear

function of ln (a) and is the source of the name of the log odds model Note that given the results of a large number of

independent inspections of a large number of cracks, the parameters of the model can be fit with a regression analysis As

an example, Fig 3 shows Eq 3 fit to the data of Fig 1 This regression approach will not be discussed further, because the maximum likelihood estimates (see the section "Analysis of Hit/Miss Data" in this article) can be applied to much smaller samples of inspection results and can give equivalent answers for large sample sizes

Fig 3 Example linear relation between log odds of crack detection and log crack size

Although the parametrizations of Eq 2 and 3 are sensible in terms of estimation through regression analyses, and are

not easily interpretable in physical terms A mathematically equivalent form of the log odds POD(a) model is given by

(Ref 8):

(Eq 4)

In this form, μ= ln a0.5, where a0.5 is the flaw size that is detected 50% of the time, that is, the median detectable crack

size The steepness of the POD(a) function is inversely proportional to ; that is, the smaller the value of σ, the steeper the POD(a) function The parameters of Eq 2 and 4 are related by:

(Eq 5)

(Eq 6)

The log odds POD(a) function is practically equivalent to a cumulative log normal distribution with the same parameters,

μ and σ of Eq 4 Figure 4 compares the log odds and cumulative log normal distribution functions for μ= 0 and σ= 1 Equation 4 is the form of the log odds model that will be used in the section "Analysis of Hit/Miss Data" in this article

Fig 4 Comparison of log odds and cumulative log normal models

POD(a) From Signal Response Data. The NDE flaw indications are based on interpreting the response to a stimulus In eddy current or ultrasonic systems, the response might be a peak voltage referenced to a calibration In fluorescent penetrant inspections, the response would be a combination of brightness and size of the indication Assume the response can be quantified and recorded in terms of a parameter, , that is correlated with flaw size Then summarizes the information for determining if a positive flaw indication will be given Only if exceeds a defined decision threshold, dec, will a positive indication be given

As an example of the concept, Table 2 summarizes the results of highly automated eddy current inspections of 28 cracks

in flat plate specimens The three data sets resulted from the use of three probes, with all other factors held constant The values in Table 2 are the depth of each crack and the peak voltage in counts recorded by the system Figure 5 shows a plot

of the versus a data for probe A No signal was recorded for 2 of the cracks, because their values were below the

recording signal threshold, th These points are indicated by a down arrow at th, indicating that the response was at an indeterminable value below the recording signal threshold Similarly, for 5 of the cracks exceeded the saturation limit,

sat, of the recording system These points are indicated by an up arrow at sat, indicating that the response was at an indeterminable value above the recording saturation limit In Fig 5, the decision threshold is set at 250 counts Only those cracks whose value is above 250 would have been flagged (detected)

Table 2 Example of a summary data sheet of versus a data

The example is based on eddy current inspections of flat plates

Crack depth Peak voltage in counts Crack identification

mm in Probe A Probe B Probe C

Fig 5 Example inspection signal response as a function of crack depth

The POD(a) function can be obtained from the relation between and a If ga( ) represents the probability density of the values for fixed crack size a, then:

(Eq 7)

This calculation is illustrated in Fig 6, in which the shaded area under the density functions represents the probability of detection

Fig 6 Schematic of POD(a) calculation from versus a relation

In general, the correlating function between and a defines the mean of ga( ), that is:

where (a) is the mean of ga( ) and is a random error term accounting for the differences between and (a) The

distributional properties of δ determine the probability density ga( ) about μ (a), as will be shown

In the data analyzed to date, a linear relation between ln ( ) and ln (a) with normally distributed deviations has proved

satisfactory (for example, Fig 5) This model is expressed by:

where δ is normally distributed with zero mean and constant standard deviation, Data have been observed that flatten

at the large crack sizes However, because the decision threshold was far below the non-linear range, restricting the range

of cracks to smaller sizes permitted the application of Eq 9 The normality of has proved to be an acceptable assumption

Assuming that the versus a relation is modeled by Eq 9 and that is normally distributed with zero mean and standard

deviation of , the POD(a) function is calculated as:

In the section "Signal Response Analysis" in this article, maximum likelihood methods for estimating β0, β1, and σ from

versus a data will be presented Note that the values below the recording threshold and above the saturation limit must

be properly accounted for in these analyses Note also that data from multiple inspections of the same cracks require analysis methods that are dependent on the design of the reliability experiment Methods for placing lower confidence

bounds on the estimated POD(a) function using the sampling distributions of the maximum likelihood estimates of β0, β1, and are also included in the section "Signal Response Analysis."

References cited in this section

3 W.H Lewis, W.H Sproat, B.D Dodd, and J.M Hamilton, "Reliability of Nondestructive Inspections Final Report," SA-ALC/MME 76-6-38-1, San Antonio Air Logistics Center, Kelly Air Force Base, Dec 1978

4 A.P Berens and P.W Hovey, "Evaluation of NDE Reliability Characterization," AFWAL-TR-81-4160, Vol

1, Air Force Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, Dec 1981

5 A.P Berens and P.W Hovey, Statistical Methods for Estimating Crack Detection Probabilities, in

Probabilistic Fracture Mechanics and Fatigue Methods: Applications for Structural Design and Maintenance, STP 798, J.M Bloom and J.C Ekvall, Ed., American Society for Testing and Materials, 1983,

p 79-94

6 D.E Allison et al., "Cost/Risk Analysis for Disk Retirement Volume I," AFWAL-TR-83-4089, Air Force

Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, Feb 1984

7 A.P Berens and P.W Hovey, "Flaw Detection Reliability Criteria, Volume I Methods and Results," AFWAL-TR-84-4022, Air Force Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, April

1984

8 D.R Cox, The Analysis of Binary Data, Methuen and Co., 1970

NDE Reliability Data Analysis

Alan P Berens, University of Dayton Research Institute

Design of NDE Reliability Experiments

An NDE reliability experiment comprises a test matrix of inspections on a set of specimens with known flaw locations and sizes The specimens are inspected under conditions that simulate as closely as practical the actual application conditions The experimental design determines the test matrix, and there are four major analysis concerns to be addressed in the experimental design These are:

• The method of controlling the factors to be evaluated in the experiment

• The method of accounting for the uncontrolled factors in the experiment

• The number of flawed and unflawed inspection sites

• The sizes of the flaws in the specimens

These topics are addressed in the following sections

Controlled and Uncontrolled Factors

The primary objective of NDE reliability experiments has been to demonstrate efficacy for a particular application by

estimating the POD(a) function and its lower 95% confidence bound (Although NDE reliability experiments can also be

conducted to optimize a system, analyses to meet this objective are beyond the scope of this article.) To demonstrate capability, it is assumed that the protocol for conducting the inspections is well defined for the application, that the inspection process is under control (hit/miss decisions are stable over time), and that all other factors introducing variability into the inspection decision will be representative of the application The representativeness of these other factors can be ensured either by controlling the factors during the inspection or randomly sampling the factors to be used

in the experiment The methods of accounting for these factors are important aspects of the statistical design of the

experiment and significantly influence the statistical properties of the estimates of the POD(a) function parameters Of particular note in this regard is that k inspections on n flaws is not equivalent to inspections on n · k different flaws, even

if the inspections are totally independent

The most important of the factors introducing variation are:

• Differences in physical properties of cracks of nominally identical sizes

• The basic repeatability of the magnitude of the NDE signal response when a specific crack is independently inspected by a single inspector using the same equipment

• The summation of all the human factors associated with the particular inspectors in the population of interest

• Differences introduced by changes in inspection hardware

These factors must be addressed explicitly or implicitly in every NDE reliability experiment

In general, the specimens used in NDE reliability experiments are very expensive to obtain and characterize in terms of the sizes of the flaws in the specimens Therefore, each experiment is based on one set of specimens containing flawed and unflawed inspection sites Because the results are significantly influenced by the specimens, it must be assumed that the flaws are representative of those that will be present in the structural application If other factors are to be included in the experiment, they will be based on repeated inspections of the same flaws From a statistical viewpoint, this restriction

on the experimental design limits the sample size to the number of flaws in the specimen set Because different cracks of the same size can have significantly different crack detection probabilities, multiple inspections of the same crack provide information about the detection probability of only that crack

The generality of the capability characterization is limited to the application for which the experiment is representative Either important factors must be typical of the application or random samples must be chosen from the population of interest and repeat inspections performed for these factors For example, if a single inspector is used to characterize a fluorescent penetrant inspection, it must be assumed that this inspector is typical of all the inspectors in the shop An alternative might be to choose a random sample of inspectors from the total pool and have each of the selected inspectors perform the experiment

Depending on the application of the results of the experiment, stratified sampling may be required to obtain a representative sample For instance, if the capability will apply to two facilities and one of them inspects twice as many components as the other, then that facility should have twice as many inspectors in the experiment An alternative method

is to characterize each facility independently Care is then required in combining the results for the joint characterization

Factorial Experiments for Hit/Miss Data. The analysis for the hit/miss data requires that all factors be balanced in any one analysis When practical, this can be most easily achieved by performing complete factorial experiments For example, Table 1 contains the results of a two-factor experiment, with the factors being cracks and inspectors These data

can be analyzed as one data set with three inspections per crack The resulting POD(a) function and its confidence bound

would be representative of the population of inspectors from which the sample was drawn

If the effect of a third factor, for example, different lots of penetrants, were to be included, the entire experiment would be repeated for each of the lots chosen at random from the population of all lots If three lots were sampled, a total of nine

inspections would be performed on each of the flawed specimens, and the resulting POD(a) would apply to the entire

inspection process Suppose, however, that the second and third samples of penetrant were used only by Inspector A In this case, the two additional sets of inspection data cannot be combined with the other three in a single analysis, because

the triple representation of Inspector A would bias the resulting POD(a) function toward his specific capability The three

sets of inspection results for Inspector A can be combined, but the range of applicability of the answer is limited to Inspector A (unless it can be shown or assumed that Inspector A is representative of the entire population)

When many factors must be considered, the number of possible combinations in a factorial experiment can easily become prohibitive More sophisticated experimental designs (fractional replications, for example) may then be required In such cases, the assistance of a professional statistician is recommended

Experimental Design for Data. Inspection-result data in the form contain considerably more information than hit/miss data and, as a consequence, permit more flexibility in the design of the experiment In analysis, the parameters

of the POD(a) function are estimated from the slope, intercept, and standard deviation of residuals of the ln ( ) versus ln (a) relation, as given by Eq 9, 12, and 13 In Eq 9, can be considered to be the sum of random effects, and experiments

can be designed to estimate the components of the total variation in For example, operators, probes, and repeatability

can be jointly evaluated in a factorial experiment and their effects accounted for in the estimate of POD(a) The statistical

model for this experiment would be:

ln ( ) = β0 + β1 ln (a) + Ci + Oj + Pk + Rl

where Ci, Oj, Pk, and Rl are the random effects due to cracks, operators, probes, and repeats, respectively The random

term, δ of Eq 9, is the sum of all random effects It can be assumed that the mean and variance of random effect X are zero

and , respectively Then:

assumed that only the variation due to cracks and one other factor is being investigated It is recommended that the assistance of a qualified statistician be obtained for more sophisticated experimental designs

Sample Sizes and Flaw Sizes

Sample sizes in NDE reliability experiments are driven more by the economics of specimen fabrication and

characterization than by the desired degree of precision in the estimate of the POD(a) function Although apparently reasonable POD(a) functions can often be obtained from applying the maximum likelihood analysis to relatively few test

results, the confidence bound calculation is based on asymptotic (large sample) properties of the estimates It should be emphasized that the calculations can also produce totally unacceptable results from the relatively few test results or from data that are not reasonably represented by the assumptions of the models Therefore, there are minimal sample size requirements that must be met to provide a degree of reasonable assurance in the characterization of the capability of the system

Larger sample sizes in NDE reliability experiments will, in general, provide greater precision in the estimate of the

POD(a) function However, the sample size is determined from the number of cracks in the experiment, and there is a

coupling with the flaw sizes that must also be considered The effect of this coupling manifests itself differently for the hit/miss and analyses

Sample Size Requirements for Hit/Miss Analysis. Data from hit/miss experiments are generally not amenable to

testing assumptions regarding the form of the POD(a) model These tests require either large numbers of independent

inspections on each flaw of a specimen set or inspection results from an extremely large number of compatible specimens (Ref 3, 4) Number and size considerations in hit/miss experiments are directed at their effect on the sampling properties

of the parameters of the POD(a) function (Ref 9)

In the hit/miss analysis, the output of an inspection states only whether or not a crack of known length was found in the inspection (Table 1) There are probabilities associated with the outcomes, and the analysis assumes that this probability increases with flaw size Because it has been assumed that the inspection process is in a state of control, there is a range of

flaw sizes over which the POD(a) function is rising In this flaw size range of uncertainty, the inspection system has

limited discriminating power in the sense that detecting or failing to detect would not be unusual Such a range might be

defined by the interval (a0.10, a0.90), where ap denotes the flaw size that has probability of detection equal to p; that is:

Flaws smaller than a0.10 would then be expected to be missed, and flaws greater than a0.90 would be expected to be detected

In a hit/miss reliability experiment, flaws outside the range of uncertainty do not provide as much information concerning

the POD(a) function as cracks within this range Cracks in the almost-certain detection range and almost-certain miss

range provide very little information concerning probability of detection Therefore, in the hit/miss experiment, not all flaws convey the same amount of information, and the effective sample size is not necessarily the total number of flaws in the experiment Adding a large number of very large flaws does not increase the precision in the estimate of the

parameters of the POD(a) function

In a reliability experiment, the location of the increasing range of the POD(a) function is not known Further, the same

sets of specimens are often used in many different experiments Therefore, it is not possible to fabricate a set of specimens with optimal flaw sizes for a particular experiment To minimize the chances of completely missing the crack size range of maximum information and to accommodate the multiple uses of specimens, flaw sizes should be uniformly distributed between the minimum and maximum of the sizes of potential interest A minimum of 60 flaws should be distributed in this range, but as many as economically possible should be used

Sample Size Requirements for Analysis. The recorded signal response, , provides significantly more

information for analysis In particular, the POD(a) model is derived from the correlation of the versus a data, and the assumptions concerning the POD(a) model can be tested using the signal response data Further, the pattern of

responses can indicate an acceptable range of extrapolation Therefore, the range of crack sizes in the experiment is not as critical in an analysis as in a hit/miss analysis For example, if the decision threshold in Fig 5 were set at 250 counts, all but four of the cracks would have been detected The larger cracks would have provided little information about the

POD(a) function in a hit/miss analysis In an analysis, however, all of the recorded values provided full information

concerning the relation between signal response and crack size, and the values at the signal threshold and saturation limit provided partial information The linearity of the fit, the normality of the deviations, and the constancy of the residual

variation can all be easily evaluated from the versus a plot

Because of the added information in the data, it is recommended that at least 30 flaws be present in experiments whose results can be recorded in this form Increasing the number of flaws increases the precision of the estimates, so the test set should contain as many flawed specimens as economically feasible

Unflawed Inspection Sites. In the context of the analyses presented in this section, sample size refers to the number

of known flaws in the specimens to be inspected The total specimen set should also contain at least twice this number of unflawed inspection sites The unflawed sites are necessary to ensure that the NDE procedure is discriminating between flawed and unflawed sites and to provide an estimate of the false call rate

Although the false call rate can have important economic consequences, the NDE reliability analyses in this section were dictated by the requirements of damage tolerance analyses The primary objective was to estimate the chances of missing flaws that might lead to structural failures The concepts of these NDE reliability analyses can be generalized to include a

non-zero probability of a flaw indication when no flaw is present at an inspection site, that is, POD(a = 0) > 0

References cited in this section

3 W.H Lewis, W.H Sproat, B.D Dodd, and J.M Hamilton, "Reliability of Nondestructive Inspections Final Report," SA-ALC/MME 76-6-38-1, San Antonio Air Logistics Center, Kelly Air Force Base, Dec 1978

4 A.P Berens and P.W Hovey, "Evaluation of NDE Reliability Characterization," AFWAL-TR-81-4160, Vol

1, Air Force Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, Dec 1981

9 A.P Berens and P.W Hovey, The Sample Size and Flaw Size Effects in NDI Reliability Experiments, in

Review of Progress in Quantitative Nondestructive Evaluation 4B, D.O Thompson and D.E Chimenti, Ed.,

Plenum Press, 1985

NDE Reliability Data Analysis

Alan P Berens, University of Dayton Research Institute

Maximum Likelihood Analysis

Parameter estimation based on the principle of maximizing the likelihood of an observed sample of data is a standard statistical technique and is amply described in the literature (Ref 10, 11) The purpose of this section is to summarize the method and its asymptotic sampling distribution properties in the context of analyzing NDE reliability data Further, a

method for using this information to calculate lower confidence bounds on the POD(a) function is also presented

Parameter Estimation. Let Xi represent the outcome of the ith inspection and f(Xi;θ) represent the probability of obtaining Xi, where θ= (θ1, θ2, .θk)' is the vector of the k parameters in the probability model For example, in a hit/miss

experiment, Xi would be 0 or 1 with probability defined by Eq 4, where a is the size of flaw i and θ= (μ,σ)' In an versus

a experiment, X i is the log of the signal response, and f(Xi;θ) is a normal density function with mean and standard

deviation given by θ= (β0 + β1 ln a, σ )', as defined in Eq 9 Let X1, ., Xn represent the results of independent inspections

of n flaws The likelihood, L, of a specific result is given by the likelihood function:

(Eq 17)

For a given outcome of the experiment, Xi is known and Eq 17 is a function of θ The maximum likelihood estimate is the

value, , which maximizes L(θ) For the models considered here, it is more convenient to work with the log L(θ):

which is also maximized at The maximum likelihood estimates are given by the solution of the k simultaneous

equations:

(Eq 19)

Asymptotic Sampling Distribution Properties. For the models being used in NDE reliability studies, the maximum likelihood estimates are invariant, consistent, and efficient Further, they are asymptotically joint normally distributed with means given by the true parameter values, θi, and the variance-covariance matrix defined by:

where I is the information matrix whose elements Iij are the expected (E) values:

(Eq 21)

In application, the maximum likelihood estimate, , is substituted for θ in Eq 21 Therefore, given the results of

inspecting a large number of flaws and a specific function for the POD(a) model, the parameters of the model can be

estimated, and the sampling distribution of the parameters will be joint normal with the known variance-covariance matrix Examples of these equations for the hit/miss and response signal models are given in the sections "Analysis of Hit/Miss Data" and "Signal Response Analysis," respectively, in this article In these applications, the assumed models will be the log odds and cumulative log normal distribution functions However, other models can also be used if evidence is available to support their selection

Confidence Bounds on the POD(a) Function. Because the POD(a) function is equivalent to a cumulative

distribution function and the parameters are being estimated by maximum likelihood, a procedure developed by Cheng

and Iles (Ref 12 and 13) can be used to place lower confidence bounds on the POD(a) function Such bounds are

calculated from the variance-covariance matrix of the estimates and reflect the sensitivity of the experiment to both the number and sizes of flaws in the specimens of the experiment

The assumed POD(a) model is a cumulative log normal distribution function with parameters θ= (μ,σ)' For distribution

functions defined by location and scale parameters (as is the case of the log normal distribution), the information matrix can be written in the form:

(Eq 22)

where n is the number of cracks in the experiment The lower one-sided confidence bound of the POD(a) function is

given by:

where (z) is the standard cumulative normal distribution, and:

(Eq 24)

(Eq 25)

where is obtained from Table 3 for the number of cracks in the experiment and the desired confidence level

Table 3 Values of for lower confidence bounds on the POD(a) function

References cited in this section

10 H Cramer, Mathematical Methods of Statistics, Princeton University Press, 1946

11 J.F Lawless, Statistical Models and Methods for Lifetime Data, John Wiley & Sons, 1982

12 R.C.H Cheng and T.C Iles, Confidence Bands for Cumulative Distribution Functions of Continuous

Random Variables, Technometrics, Vol 25 (No 1), Feb 1983, p 77-86

13 R.C.H Cheng and T.C Iles, One Sided Confidence Bands for Cumulative Distribution Functions,

Technometrics, Vol 32 (No 2), May 1988, p 155-159

NDE Reliability Data Analysis

Alan P Berens, University of Dayton Research Institute

Analysis of Hit/Miss Data

Estimation of the parameters of the log odds model for hit/miss data is based directly on the probability of each 0 or 1

result of an inspection Assume that a balanced experiment has produced k inspections on each of n cracks For this

application, the likelihood function is given by:

Because iterative techniques converge to local maxima, the solution to Eq 32 and 33 may be sensitive to the initial values

A set of initial values based on the method of moments has been found to be useful (Ref 7) These are given by:

(Eq 34)

(Eq 35)

where X1, .,Xn are the ordered values of the natural logs of the flaw sizes and pi is the observed percentage of detections

of the ith ordered flaw size If convergence is not obtained, increasing the initial estimate of a has often provided

convergence However, Eq 32 and 33 are not always solvable This will be discussed further in the section "Comments on Hit/Miss Analysis" in this article

Confidence Bound Calculation in Hit/Miss Analysis. The information matrix is estimated from Eq 21, using

and for and For this POD(a) model, the elements of the information matrix are given by:

(Eq 36)

(Eq 37)

(Eq 38)

Note that k0, k1, and k2, the parameters required in the calculation of the lower confidence bound on the POD(a) function,

are also defined by Eq 36, 37, and 38 All of the parameters required by Eq 23, 24, and 25 to calculate the lower

confidence bound on the POD(a) function are available

Hit/Miss Analysis Examples. As examples of the application of the method to real data, the parameters of the log

odds POD(a) function were obtained for the data in Table 1 Table 4 presents a summary of the parameters of the POD(a)

function for each data set of Table 1 and the combination of the three data sets in a single analysis Figure 7 shows the

POD(a) function and the lower 95% confidence bound for Inspector A and the same information when the data from the

three inspectors are combined Adding inspections of the same cracks did not increase the precision of the estimate of the

POD(a) function Figure 8 compares the POD(a) functions for the three inspectors and the composite

Table 4 POD(a) parameters for the hit/miss data in Table 1

Inspector Parameter

Composite

0.96 1.11 0.82 0.96

0.59 1.04 0.87 0.88

Fig 7 POD(a) function and lower 95% confidence bound from hit/miss analysis of the data in Table 1 for one

inspection per crack (from Inspector A) and for three inspections per crack (from the composite result of Inspectors A, B, and C)

Fig 8 POD(a) functions from hit/miss analysis of the data in Table 1

Comments on Hit/Miss Analysis. In a well-designed experiment of sufficient sample size for which the log odds

model is a reasonable representation of the POD(a) function, the maximum likelihood hit/miss analysis will provide a

valid solution Conversely, lacking any of these elements, it is possible that either no solution or an unacceptable solution can result If there is no overlap in the flaw size ranges of the detections and misses, Eq 32 and 33 will not yield a

solution More flaws are needed in the region of increase of the POD(a) function It is also possible to obtain an estimate

of a POD(a) function that decreases with flaw size if the inspection system is poorly designed or not in control and if

large flaws tend to be missed more often than small flaws Both of these types of results are readily apparent, albeit disconcerting

A third type of unacceptable result is an apparently acceptable POD(a) function but a confidence bound that eventually

decreases with flaw size This situation is most easily understood in terms of the log odds versus log flaw size plot If the

slope is positive, the POD(a) function will appear reasonable, but if it is not significantly greater than zero, the lower

confidence bound will eventually decrease with flaw size Therefore, a decreasing confidence bound is evidence of lack

of fit of the log odds model

Finally, lack of fit of the model is often manifest in large values of coupled with small values of or extremely wide confidence intervals Although there are, in general, insufficient data in hit/miss experiments to test hypotheses about the

POD(a) model, as a minimum each fit should be subjectively judged For example, in Fig 9, the observed detection proportions of each crack in the data of Table 1 are superimposed on the composite POD(a) function and confidence limit from Fig 7 The uncertainty in the POD(a) function as indicated by the width of the confidence bound seems justified by

the plot of the raw data In this example, if greater precision (narrower confidence bounds) were desired, more cracks in the 2 to 8 mm (0.08 to 0.3 in.) range would be needed in the experiment Such plots provide an indication of the fit of the

model to the data as well as the range of flaw sizes that are contributing to the information from which the POD(a)

function is being estimated This is true even for experiments in which there is only one inspection per crack and all detection probabilities are plotted at 0 or 1

Fig 9 Example fit of hit/miss POD(a) function and lower 95% confidence bound to observed detection

probabilities (three inspections per crack)

References cited in this section

7 A.P Berens and P.W Hovey, "Flaw Detection Reliability Criteria, Volume I Methods and Results," AFWAL-TR-84-4022, Air Force Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, April

1984

14 A Ralston, A First Course in Numerical Analysis, McGraw-Hill, 1965

NDE Reliability Data Analysis

Alan P Berens, University of Dayton Research Institute

Signal Response Analysis

In signal response data analysis, the parameters of the POD(a) function are calculated from parameters of the versus a

relation If all the values are between the signal recording threshold and the saturation limit, a simple regression analysis

of ln ( ) versus ln (a) will yield the necessary information to estimate the POD(a) function and its lower confidence

bound In fact, the least squares estimates from the regression analysis also happen to be the maximum likelihood estimates The analysis presented in this section is applicable to the more general case in which some of the values are censored at the recording threshold or the saturation limit This more general analysis will give answers identical to those

of the regression analysis if all values are available for all of the flaws (Ref 15)

In the response signal analysis, it is assumed that the values for a flaw of size a have a normal distribution, with mean

and standard deviation given by:

The likelihood function is partitioned into three regions:

• Region R, for which values were recorded

• Region T, for which only a maximum value is known (the values fall below the recording signal threshold and cannot be recorded)

• Region S, for which only a minimum value is known (the values fall above the saturation limit and cannot be recorded)

These regions are identified by the open circles, the down arrows, and the up arrows, respectively, in Fig 5 The likelihood function for the entire sample is the product of the likelihood functions for the three regions:

But (suppressing the dependency of L on 0, 1, and ):

(Eq 45)

(Eq 46)

(Eq 47)

because 1/ (Zi) dz is the probability of observing i for the ith flaw in R, i(ath) is the probability of obtaining an ln

i value below the recording threshold for the ith flaw in S, and 1 - i (asat) is the probability of obtaining an value

above the saturation limit for the ith flaw in T The log of the likelihood function is:

(Eq 48)

where r is the number of cracks in R, that is, the number of cracks for which values were recorded

Parameter Estimation in Analysis. The maximum likelihood estimates are given by the solutions to:

Standard numerical methods, such as the Newton-Rhapson iterative procedure (Ref 14), can be used to find the solutions

to Eq 49, 50, and 51 Excellent choices for the initial estimates of iterative procedures are the intercept, slope, and standard deviation of residuals obtained from a standard regression analysis of only those values for which a valid response was recorded

Confidence Bound Calculation in Analysis. Because the POD(a) parameters are calculated from the estimates of the versus a data, the calculation of the lower confidence bound is a five-step process:

• The information matrix for the estimates of 0, 1, and is obtained using Eq 21

• The variance-covariance matrix of 0, 1, and is obtained by inverting the information matrix (Eq 20)

• The variance-covariance matrix of the estimates of and are calculated based on a first-order Taylor series expansion of the equations relating 0, 1, and to and (Eq 12 and 13)

• The information matrix for and is obtained by inverting the variance-covariance matrix to obtain Eq

22

• The calculated values are substituted into Eq 23, 24, and 25 to obtain the lower confidence bound

The elements of the information matrix for = ( 0, 1, ) are given by (dropping the subscripts):

(Eq 59)

Let V ( 0, 1, ) represent the variance-covariance matrix of the maximum likelihood estimates of the ln ( ) versus ln

(a) analysis The value V ( 0, 1, ) is obtained from the inverse of the information matrix Let the elements of V ( 0,

Inverting this variance-covariance yields the values of k0, k1, and k2 required in Eq 25 to calculate the lower confidence

bound on the POD(a) function

Multiple Inspections Per Flaw. Repeat values for the same flaw can be analyzed to estimate the magnitude of the total variability being introduced by factors other than the flaws in the experiment In essence, the random term, , of Eq

9, can be partitioned into components that can be estimated if the experiment is properly designed The relative magnitude

of the components of variance indicates potential areas for improving the system However, the methods for using the

values to generate POD(a) functions and confidence bounds from complex experiments with censored values are still

Because it is reasonable to assume that the variability introduced by flaws is independent of that introduced by other factors:

where can be estimated as the pooled-within variance of i values for each flaw using those flaws for which values

were recorded (Ref 17):

(Eq 71)

where n* is the number of flaws with uncensored values and ki is the number of uncensored values for flaw i

Between-crack variability, , cannot be estimated directly, but can be estimated indirectly from a censored regression analysis First, the mean log response for each flaw is calculated This mean response may be a simple average if all values are available, but a mean based on an analysis of censored data (as previously discussed) will be required for the flaws for which values were censored at the decision threshold or saturation limit The analysis in the section

"Parameter Estimation in Analysis" in this article is then used on the model:

to obtain estimates 0, 1, and * However:

To date, the lower confidence bound has been placed on the POD(a) function using the information matrix derived from

the censored regression through the average of the ln ( ) values for each crack This procedure does not account for the added uncertainty resulting from the estimate of When is significantly greater than , the error introduced by neglecting this variation is judged to be negligible (Most of the applications of the analysis have been on highly automated eddy current systems In these applications, the variability of ln ( ) values within cracks has been significantly less than that between cracks.) The method of constructing exact confidence bounds is under development This process can be extended to account for more than one component of the variability of ln ( ) values

Examples of Analysis. The data in Table 2 resulted from three eddy current inspections of 28 fatigue cracks in flat plates simulating the web/bore of an aircraft engine disk The three inspections are from the use of three different probes,

with all other factors being held fixed Table 5 presents a summary of the parameters of the POD(a) function for each data set in Table 2 and the combination of the three data sets in a single analysis The ln ( ) versus ln (a) data from probe A

are presented in Fig 5 This data set had two inspection results below the recording threshold and five results above the

saturation limit The POD(a) function for probe A and its lower 95% confidence bound are shown in Fig 10 The combined results from the three probes yield the composite POD(a) function and its lower 95% confidence bound, which

are also shown in Fig 10 If the probes were in some sense selected at random from the population of all probes applicable to this equipment and if other factors were assumed to be representative of the application, this composite

POD(a) function would be representative of the inspection system Figure 11 shows the individual POD(a) functions for

all three probes and their composite

Table 5 Example of POD(a) parameters determined from the data in Table 2

The decision threshold is 250 counts

Probe Parameter

(a) a

50 = exp( ) = estimate of crack size at 50% POD

(b) a

90 = exp( + 1.282 ) = estimate of crack size at 90% POD

(c) a90/95 = upper 95% confidence bound on the estimate of a90

Fig 10 POD(a) function and lower 95% confidence bound from signal response analysis of the data in Table 2

with one inspection per crack (with probe A) and with three inspections per crack (from the composite result of probes A, B, and C)

Fig 11 POD(a) functions from signal response analysis of the data in Table 2

Comments on Analysis. There are several advantages of the analysis over the hit/miss analysis in estimating the

POD(a) function These accrue primarily because of the added information contained in the values

It is not as critical in the experiment to have flaws with sizes in the range of increase of the POD(a) function Because the

analysis does not depend on whether or not flaws were detected, the decision threshold can be arbitrarily set after the experiment Even if the recording threshold is close to the decision threshold, the method permits extrapolation of the results from larger flaw sizes to the smaller sizes of interest The extrapolation is reasonable over the flaw size range for

which the ln ( ) versus ln (a) relation is linear, and deviations from the fit are normally distributed with constant

variance

A principal advantage of the analysis is that statistical tests of the underlying assumptions are readily available (Ref 17)

In most of the experiments that have been analyzed by this method, the assumptions could not be rejected However, for

very large flaws, a tendency for the ln ( ) versus ln (a) relation to bend down has been observed on occasion In this

region, the values are very large, and all of the flaws are easily detected One method of linearizing the ln ( ) response

is to restrict the analysis to a range of smaller flaw sizes by censoring values at a lower saturation limit This is equivalent to deleting data points and may reduce the sample size to an unacceptable number In this case, the flaw sizes

in the experiment were not appropriate for the application It should be noted that ignoring this type of nonlinearity tends

to produce a nonconservative POD(a) function, because the effect is to produce a smaller value of the median detectable

crack size,

When the data do not fit the model, it is possible to obtain results that have no relation to reality This can occur if ln ( )

is not an increasing linear function of ln (a) or if there are outliers in the data set that have a significant influence on the

analysis The effects of these anomalies are sometimes manifested in unreasonable values of or a or by a lower

confidence bound that eventually decreases for large crack sizes As a minimum, it is recommended that a plot of ln ( )

versus ln (a) be obtained for all experiments This will permit at least a subjective judgment concerning the assumptions

of the analysis

If the data do not fit the model, it is necessary to ensure that the process is in control and that the experiment was properly

designed and executed It may also be possible to use other relations between and a as the basis of analysis, but these

have not yet been explored

References cited in this section

14 A Ralston, A First Course in Numerical Analysis, McGraw-Hill, 1965

15 M Glaser, Regression Analysis With Dependent Variable Censored, Biometrics, Vol 21, June 1965, p

300-307

16 S.R Searle, Linear Models, John Wiley & Sons, 1971

17 W.J Dixon and F.J Massey, Jr., Introduction to Statistical Analysis, McGraw-Hill, 1957

NDE Reliability Data Analysis

Alan P Berens, University of Dayton Research Institute

References

1 B.G.W Yee, F.H Chang, J.C Couchman, G.H Lemon, and P.F Packman, "Assessment of NDE Reliability Data," NASA CR-134991, National Aeronautics and Space Administration, Oct 1976

2 W.D Rummel, Recommended Practice for Demonstration of Nondestructive Evaluation (NDE)

Reliability on Aircraft Production Parts, Mater Eng., Vol 40, Aug 1982, p 922-932

3 W.H Lewis, W.H Sproat, B.D Dodd, and J.M Hamilton, "Reliability of Nondestructive Final Report," SA-ALC/MME 76-6-38-1, San Antonio Air Logistics Center, Kelly Air Force Base, Dec

Inspections 1978

4 A.P Berens and P.W Hovey, "Evaluation of NDE Reliability Characterization," AFWAL-TR-81-4160, Vol 1, Air Force Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, Dec 1981

5 A.P Berens and P.W Hovey, Statistical Methods for Estimating Crack Detection Probabilities, in

Probabilistic Fracture Mechanics and Fatigue Methods: Applications for Structural Design and Maintenance, STP 798, J.M Bloom and J.C Ekvall, Ed., American Society for Testing and Materials,

1983, p 79-94

6 D.E Allison et al., "Cost/Risk Analysis for Disk Retirement Volume I," AFWAL-TR-83-4089, Air Force

Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, Feb 1984

7 A.P Berens and P.W Hovey, "Flaw Detection Reliability Criteria, Volume I Methods and Results," AFWAL-TR-84-4022, Air Force Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, April 1984

8 D.R Cox, The Analysis of Binary Data, Methuen and Co., 1970

9 A.P Berens and P.W Hovey, The Sample Size and Flaw Size Effects in NDI Reliability Experiments, in

Review of Progress in Quantitative Nondestructive Evaluation 4B, D.O Thompson and D.E Chimenti,

Ed., Plenum Press, 1985

10 H Cramer, Mathematical Methods of Statistics, Princeton University Press, 1946

11 J.F Lawless, Statistical Models and Methods for Lifetime Data, John Wiley & Sons, 1982

12 R.C.H Cheng and T.C Iles, Confidence Bands for Cumulative Distribution Functions of Continuous

Random Variables, Technometrics, Vol 25 (No 1), Feb 1983, p 77-86

13 R.C.H Cheng and T.C Iles, One Sided Confidence Bands for Cumulative Distribution Functions,

Technometrics, Vol 32 (No 2), May 1988, p 155-159

14 A Ralston, A First Course in Numerical Analysis, McGraw-Hill, 1965

15 M Glaser, Regression Analysis With Dependent Variable Censored, Biometrics, Vol 21, June 1965, p

300-307

16 S.R Searle, Linear Models, John Wiley & Sons, 1971

17 W.J Dixon and F.J Massey, Jr., Introduction to Statistical Analysis, McGraw-Hill, 1957

Models for Predicting NDE Reliability

J.N Gray, T.A Gray, N Nakagawa, and R.B Thompson, Center for NDE, Iowa State University

The implementation of such an approach requires the knowledge and integration of stress, flaw size, and failure mechanisms As shown in Fig 1 for the case in which fatigue can be modeled by linear-elastic fracture mechanics, cyclic

stress excursions, Δσ, and flaw size, a, are required as inputs to fracture mechanics to predict a stress intensity range, ΔK

If the stress intensity exceeds a critical value known as the plane-strain fracture toughness, KIc, catastrophic failure is

imminent Otherwise, crack growth laws of the form da/dN = A(ΔK)m (where N is the number of stress excursions and A

and m are constants) can be used to estimate the safe life available before catastrophic failure

Fig 1 Methodology of lifetime prediction for metal parts undergoing cyclic fatigue Source: Ref 1

Implementation of this damage-tolerant design approach rests on three methodologies: stress analysis, nondestructive evaluation (NDE), and failure modeling Incorporation of these methodologies in the design of damage-tolerant

components would ideally take advantage of analytical or numerical computations to model the expected lifetime performance of a component At present, extensive capabilities are in place for modeling stresses and failures, and these are widely used in the design process However, the modeling of nondestructive evaluation is not nearly as widely accepted Instead, frequent use is made of empirical rules based on extensive demonstration programs For both economic and time reasons, there is a significant need to develop a model base for estimating NDE reliability (which is often measured in terms of the probability of flaw detection at given confidence level) It is the purpose of this article to present the current status and future directions of efforts to develop such a capability This is not intended as a review of international efforts in NDE reliability modeling but rather as a summary of the authors' experience in modeling the inspectability of aerospace components, with emphasis on engine components Therefore, attention is given to ultrasonic, eddy current, and radiographic inspection Broader sets of references for the case of ultrasonics can be found in recent review articles (Ref 2, 3) Of particular note is the work performed by the Central Electricity Generating Board in modeling the inspectability of nuclear power generating components (Ref 4)

The details of models for NDE reliability are partially dictated by their envisioned uses (Ref 5, 6), which are conceptually illustrated in Fig 2 One would like to have a model that predicts the probability of detecting flaws of various sizes This would clearly require as inputs the design of the component, its history of processing and service, and a specification of the inspecting methodology to be used Given the specifications of these input parameters, it could first be asked whether the predicted probability of flaw detection of the NDE system is adequate to meet the demands imposed by the required performance of the component Should the expected probability of detection (POD) be inadequate, the model could be exercised to modify the specifications of the inspection, the design of the component, or the processing or service profiles The NDE models thus become an integral part of a broader concept known as unified life cycle engineering or simultaneous engineering (Ref 7) The essential feature is that one should consider all aspects of the life of a component

in the design process, including the ability to inspect and maintain the component, rather than just the initial costs This will ultimately lead to more sophisticated networking of models, as shown in Fig 3 Figure 3 illustrates a number of factors that must be added to the traditional computer-aided design and manufacturing (CAD/CAM) methodologies to produce a design optimized for life cycle performance Multiple interactions among the various factors must be considered to allow the design to address simultaneously all the issues associated with damage tolerance (Ref 8)

Fig 2 Diagram of probability of detection model and its application to NDE system qualification and

optimization and to computer-aided design for inspectability POD, probability of detection

Fig 3 Schematic of possible linkages needed for unified life cycle engineering

Given these objectives, it is obvious that an NDE model must exhibit certain characteristics First, it must predict the response of a real measurement system, as influenced by the specific characteristics of commercially available probes and instruments, rather than an idealized response based on assumptions such as plane wave illumination Second, the models should give as outputs the information obtained by real inspection protocols For example, if a signal strength is compared

to a threshold as a criterion for detection, this operation should be simulated by the model If separate protocols are followed in detection and sizing, these should be described by separate models Third, the models should be used to develop more reliable standardization approaches This is necessary to ensure that inspections specified in the design process are uniformly implemented by NDE units at the various manufacturing and maintenance departments encountered

by the component in its lifetime Fourth, it is desirable in certain industries for the models to be configured such that they can be integrated with standard CAD packages This article discusses some ultrasonic, eddy current, and x-ray radiography models that have been developed to exhibit the characteristics mentioned above As noted previously, primary emphasis is placed on formulations that have been developed in response to the particular needs of the aerospace industry This article also presents a broader discussion of possible future applications of a reliability modeling capability

References

1 S.T Rolfe and J.M Barson, Fracture and Fatigue Control in Structures: Application of Fracture Mechanics,

Prentice-Hall, 1977

2 R.B Thompson and T.A Gray, Use of Ultrasonic Models in the Design and Validation of New NDE

Techniques, Philos Trans R Soc (London) A, Vol 320, 1986, p 329-340

3 R.B Thompson and H.N.G Wadley, The Use of Elastic Wave-Material Structure Interaction Theories in

NDE Modeling, CRC Crit Rev Solid State Mater Sci., in press

4 J.M Coffey and R.K Chapman, Application of Elastic Scattering Theory for Smooth Flat Cracks to the

Quantitative Prediction of Ultrasonic Defect Detection and Sizing, Nucl Energy, Vol 22, 1983, p 319-333

5 R.B Thompson, D.O Thompson, H.M Burte, and D.E Chimenti, Use of Field-Flaw Interaction Theories to

Quantify and Improve Inspection Reliability, in Review of Progress in Quantitative Nondestructive

Evaluation, Vol 3A, D.O Thompson and D.E Chimenti, Ed., Plenum Press, 1984, p 13-29

6 T.A Gray and R.B Thompson, Use of Models to Predict Ultrasonic NDE Reliability, in Review of Progress

in Quantitative Nondestructive Evaluation, Vol 5, D.O Thompson and D.E Chimenti, Ed., Plenum Press,

1986, p 911

7 H.M Burte and D.E Chimenti, Unified Life Cycle Engineering: An Emerging Design Concept, in Review of

Progress in Quantitative Nondestructive Evaluation, Vol 6B, D.O Thompson and D.E Chimenti, Ed.,

Plenum Press, 1987, p 1797-1809

8 D.O Thompson and T.A Gray, The Role of NDE in Global Strategies for Materials Synthesis and

Manufacturing, Proceedings of the 1988 Fall Meeting, Materials Research Society, in press

Models for Predicting NDE Reliability

J.N Gray, T.A Gray, N Nakagawa, and R.B Thompson, Center for NDE, Iowa State University

Ultrasonic Inspection Model

Empirical determinations of ultrasonic inspectability based on demonstration experiments are of limited utility because their predictions cannot, in general, be extrapolated to new situations beyond the bounds of the data set upon which they are based Additional data are needed in order to apply them to other cases, and the costs and time required to develop such results for all (or many) possible component designs and scan plans are prohibitive However, the physical principles upon which many different ultrasonic inspection techniques are based, as applied to a variety of components, are quite similar Therefore, a mathematical model, whose validity has been proved against a relatively small amount of empirical data, can be used to accurately predict inspectability beyond the bounds of the experimental evidence

The foundation of a physically based mathematical or computer model of ultrasonic inspectability is an analytical formalism, a numerical algorithm, or a combination of the two incorporating the physical principles of the measurement Such a model consists of descriptions of the waves radiated by the probes, their modification by the geometry of the testpiece, the wave propagation and scattering from defects, and the effects of signal processing and display The first four sections below review the technical details of each of these elements as developed to describe measurements made in aircraft engine components

In the fifth section, the use of the models to predict the POD of flaws is discussed The prediction of the POD for flaws is emphasized because of the importance of that parameter in damage-tolerant design Other uses of models, such as assisting in the interpretation of data during flaw characterization and sizing, are also important but are not explicitly discussed Some early applications of POD models are summarized in the sixth section

Reciprocity Relation. The ultrasonic NDE simulation models described in this section are based on the formalism of the electromechanical reciprocity relationship of Auld (Ref 9) This relationship, when specialized to the case of elastic wave scattering, can be expressed as follows Assume that two identical ultrasonic transducers, a and b, are placed in a fluid to be used in an immersion, pitch-catch measurement (a single probe, or pulse-echo, configuration is a special case)

of a component containing a flaw, F Let be the ratio of the electrical signal radiated into coaxial line b by the receiving transducer to the electrical signal incident on the transmitting probe from coaxial line a Then the change, F, in this signal induced by the presence of the flaw in the insonified region of the component is given by:

(Eq 1)

where a and a are the displacement vector field and the stress tensor, respectively, that would be produced in the

presence of the flaw when probe a is excited by an electrical power P, b and b are the fields that would have been established in the absence of the flaw if probe b had been excited, the overhead dot denotes time differentiation, SF is an

arbitrary closed surface enclosing the flaw, and n is the inward normal to that surface Equation 1 is an exact result known

as the electromechanical reciprocity relationship Its simplicity, however, belies its intractability, except in a very few special cases of limited practical use

Measurement Model. A number of approximations and simplifications are necessary in deriving an accurate yet computationally efficient inspection measurement model from Eq 1 One situation for which a useful and accurate model can be extracted from Eq 1 through rather remarkable simplifications is the pulse-echo inspection of isotropic, homogeneous elastic materials containing small flaws of fairly simple shape This is applicable, for example, to typical ultrasonic inspections of gas turbine aircraft engine components, in which fracture-critical flaws are quite small because

of the high stresses created during engine operation

The assumption of small in this case means that the dimensions of the defect are small relative to the variations in the transverse profile of the ultrasonic beam The ultrasonic fields can be approximated locally as plane waves whose displacement and stress fields are the same as those of the true fields (Ref 10) Scattered fields can also be simply modeled, provided their variation is not significant over the face of the receiving transducer These fields can be represented by the product of a spherically spreading wave times the far-field, unbounded medium scattering amplitude of

the flaw (Ref 10) This scattering amplitude A is formally defined as:

(Eq 2)

where us is the displacement amplitude of the fields at a distance r from the center of the flaw, u0 is the amplitude of an

illuminating plane wave, and ks is the wavenumber of the scattered wave mode A time-harmonic term of the form exp

(i t) is implicitly assumed in this discussion However, the model has the capacity to deal with pulses, as described later

Applying these approximations to Eq 1, the a fields (those produced when probe a is fired and the flaw is present) are expressed as the sum of a plane wave illuminating field and a scattered field due to that plane wave, using the scattering amplitude as in Eq 2 The b field is simply a plane wave term representing illumination by probe b in the absence of the flaw After some manipulation of the resulting integral and neglecting higher-order terms, a measurement model is derived that represents the measured signal as a product of factors describing the effects of transducer efficiency, transmission through interfaces, attenuation and beam spread, and scattering (Ref 10) Specifically, one finds:

(Eq 3)

where β is an efficiency factor; TaCa and TbCb represent local plane wave amplitudes at the flaw depth for the a and b

fields (these are the product of interface transmission, T, and beam diffraction, C, factors); A is the scattering amplitude of

the flaw; ρ1Vb and ρ0V0 are the acoustic impedances of the solid and fluid media, respectively; kb is the wavenumber for

the received wave mode; aT is the transducer radius (which is assumed to be the same for both a and b); and the two exponential terms represent the ultrasonic phase change and attenuation, respectively Equation 3 represents only one frequency A frequency spectrum can be obtained via superposition of individual frequency terms Thus, time-domain waveforms, such as would be observed on an oscilloscope screen, can be obtained by an inverse Fourier transform of this measurement model spectrum (Further details of this derivation and the determination of the efficiency factor in Eq 3 can be found in Ref 10.) Work at a number of institutions has led to formulas, algorithms, and data bases for the ultrasonic beam and scattering amplitude factors in Eq 3, so that a variety of useful simulations can be made, including

the use of planar or focused probes (Ref 11, 12, 13, and 14), inspection through planar or curved liquid/solid interfaces (Ref 11, 12, 14, and 15), and scattering from both volumetric (Ref 16, 17, 18) and cracklike (Ref 17, 19) defects These aspects of the measurement model will be described in subsequent sections in this article An alternative formulation, appropriate when the flaw is not small, is found in Ref 4

As an example, Fig 4 shows a comparison of an experimental pulse-echo radio-frequency (RF) waveform obtained from

a circular flat crack versus two model-based simulations The magnitude of the Fourier spectrum of each RF signal is also shown The simulated crack was a circular, disk-shaped cavity 0.8 mm (0.032 in.) in diameter and 0.08 mm (0.003 in.) thick located in a diffusion bond plane of a specimen of a nickel-base powder metal alloy (IN 100) The specimen was a

25 mm (1.0 in.) thick plate The face of the crack was parallel to the surface of the sample, and a 10-MHz, 6.35 mm (0.25 in.) diam unfocused transducer was tilted approximately 7° from normal to the sample surface to generate a 30° refracted longitudinal wave in the sample

Fig 4 Comparisons between experimental and model-predicted RF waveforms (top) and their Fourier spectra

(bottom) for 30° longitudinal wave backscatter from a 0.8 mm (0.03 in.) diam circular crack in IN100 (a) Experimental measurements of scattering amplitudes (top) and their Fourier spectra (bottom) (b) Model of scattering with method of optimal truncation (c) Model of scattering with the Kirchhoff approximation

The scattering amplitudes used in Fig 4 were either results of the method of optimal truncation (MOOT) (Ref 17), which

is a computationally intensive algorithm, or the elastodynamic Kirchhoff approximation (Ref 19) The experimental and MOOT results are very nearly identical in both the time and frequency domains Because the MOOT results are in quasi-exact agreement with the measured scattering amplitudes in this case, this comparison highlights the accuracy of the measurement model The Kirchhoff model result fails to reproduce some of the detailed wiggles of the experimental and MOOT results, but quite accurately reproduces the overall signal amplitude (voltage) This amplitude is the measured quantity that is routinely used in ultrasonic flaw detectors Therefore, the approximation, which in this case yields a simple and computationally efficient model, can simulate a practical inspection problem Note, however, that the error of the Kirchhoff model will depend on the flaw size and the orientation and polarization of the ultrasonic wave Care must

be taken to ensure that sufficient accuracy is obtained in particular applications by validating the model through comparison with controlled experiments and/or more exact theories for special cases

Beam Models. To perform the preceding comparison, one essential element in the simulation was the representation of the ultrasonic fields in the vicinity of the flaw Because of the finite size of any realistic transducer, these fields will be quite complex, exhibiting peaks and valleys along the axis of the beam and side-lobes away from the axis The simulations shown in Fig 4 represent the case of a scatterer on the axis of the beam, for which a number of approximate beam models have been generated (Ref 11, 13, 14) In a typical automated ultrasonic scan of a component, however, a defect in that component will not generally lie along the axis of the ultrasonic beam The degree of misalignment will depend, for example, on the coarseness of the scan mesh used to inspect the part To simulate such an inspection situation,

it is necessary to incorporate the full fields of an ultrasonic transducer into the model This is a formidable task because of the elastic (tensor) nature of wave propagation in a solid and because of the need to consider the interaction of the probing fields with a possibly curved liquid/solid interface at the component surface However, two approximations have emerged

as useful models of transducer radiation patterns: the Gaussian model (Ref 12) and the Gaussian-Hermite model (Ref 13,

14, 20)

The former, and simpler, of these models assumes that the transverse profile of the radiation profile is Gaussian in shape

at all distances from the probe This Gaussian beam model provides a set of simple algebraic formulas that predict diffraction effects (beam spread only), effects of lenses, and refraction/focusing due to transmission through curved liquid/solid interfaces (Ref 12) However, typical ultrasonic transducers do not generate Gaussian radiation patterns For example, typical piston-type radiators exhibit side-lobes and peaks and nulls along the axis of the probe in the near field However, in the far field (that is, several times the near-field distance), the Gaussian model, if suitably normalized, does accurately predict the amplitude and width of the main lobe in the radiation pattern of a typical piston-type transducer (Ref 12) One application of this approximation, therefore, is the simulation of the fields near a focal region, which can occur either as a result of an acoustic lens on the probe or the focusing effect of a curved component surface

The Gaussian-Hermite model is based on a series expansion of the radiated fields of a transducer in terms of a complete set of orthogonal solutions to a reduced wave equation (Ref 13, 14, 20) These functions are products of a Gaussian factor and a Hermite polynomial The coefficients in the series expansion are obtained by integrating the product of the Gaussian-Hermite functions and the velocity distribution on the face of a probe over its area This distribution and the shape of the probe face are arbitrary, so that virtually any probe shape, lens type, and so on, can be modeled The laws for transmission through curved liquid/solid interfaces and propagation in elastic isotropic media are implemented as simple algebraic operations The primary disadvantage of the Gaussian-Hermite model is that it is a series solution and therefore can require significantly longer computation times than the Gaussian approximation because of the need for a large number of terms, especially in the near field However, this becomes less of a disadvantage as computational speeds continue to increase as a result of advances in computer hardware

Scattering Approximations. Another key element in the simulation of ultrasonic inspection of structures is the

model, or models, for representing the interaction of the probing ultrasonic fields with defects In the most general case, this is represented by a complicated and computationally intractable integral, such as Eq 1 In some cases, however, the effects of the probing ultrasonic fields can be separated from the scattering effects Specifically, under the assumptions that led to the measurement model (Eq 3), the ultrasonic beam can be described by one of the models just mentioned For example, elastic wave scattering can be modeled through the use of a far field, unbounded medium scattering amplitude, whose definition was given in Eq 2 Fortunately, considerable research effort has been directed over the past several years toward the development of various models, approximations, and solutions for scattering amplitudes of both volumetric and cracklike defects (Ref 3)

For application to the ultrasonic inspection of jet aircraft engine components, a reasonable inventory of scattering models includes formalisms for both volumetric and cracklike flaws For volumetric flaws, of ellipsoidal shape and arbitrary orientation, both voids and inclusions can be represented by an elastodynamic Kirchhoff approximation (Ref 18) This approximation is exact in its treatment of the strength of the front surface reflection ( function) It is valid for both longitudinal and shear wave backscatter, such as would be used to simulate pulse-echo inspections The limitation of this model, however, is that it is accurate only for early-time events in the scattering Therefore, it does not predict the amplitude of scattered fields that have reverberated within an inclusion; in some cases, these scattered fields can be of higher amplitude than the initial front surface reflection The Kirchhoff approximation is therefore a conservative model for scattering from volumetric flaws It does have the benefit of simplicity and computational efficiency

For cracklike flaws, an elastodynamic Kirchhoff approximation to scattering from internal flat cracks of elliptical shape has been implemented for both pulse-echo and pitch-catch techniques and for longitudinal and shear wave modes (Ref 19) This Kirchhoff approximation accurately predicts the specular content (mirror reflection) of scattering, but does not properly include edge diffraction contributions or surface wave modes It also does not contain any provision for surface roughness or partial closure of the crack faces It does, however, yield reasonably accurate predictions of signal

amplitudes in the near-specular regime, as can be inferred from Fig 4 One fairly significant limitation of the model is its inaccuracy in the nonspecular regime It predicts, for example, that the scattering amplitude is identically zero for edge-on incidence from a crack, which is inconsistent with established crack scattering results (Ref 17, 21) However, the model is

of significant utility because an inspection system for detecting cracks would be set up to take advantage of specular orientations, if possible Furthermore, in this case, the Kirchhoff approximation is a simple and computationally fast algorithm

Probability of Detection Models. One important application of ultrasonic measurement modeling is the simulation

of probability of detection Detection is appropriately described in terms of a probability for several reasons A given size and type of defect may occur at random positions and with a range of orientations within each of a set of nominally identical components The detailed shape of the defect may vary in a way that influences its ultrasonic response differently from its fracture response Variations in grains, porosity, surface roughness, and so on, as well as the electronic equipment in a detection system, will cause noise, which will interfere with the signals from a flaw Therefore, a given size and type of defect will exhibit a distribution of signal amplitudes if measured in a population of components containing such defects Because these signal amplitudes are compared to a preset amplitude threshold in typical flaw detectors, some of the flaws will be missed, while others will be detected The POD is the ratio of the number of flaws that are detected to the total number of flaws in the inspected components

A formalism to predict the probability, p(S, N, T), that a given signal, S (video envelope), in the presence of noise with total power N2 will be detected by exceeding a threshold, T, is given by:

(Eq 4)

where io(z) = exp (-z)I0(z), with I0(z) being the modified Bessel function of the first kind and order zero This approach,

based on work performed by Rice (Ref 22), was developed for a narrow-band signal, as is typical in radar analysis

Equation 4 represents only the probability of detecting a single signal level, S, and does not take into account the

distribution of signal amplitudes for variations in the size, shape, and type of flaw

A POD model for ultrasonics has been developed by using the measurement model to calculate the variability of signal

levels as influenced by the position and orientation of the flaws Then, for a given root-mean-square noise level, N, Eq 4

can be used to represent the probability that the video signal from a specific defect (a given size, shape, type, location, orientation, and so on) in a given component (material, geometry, and so on) and using a specific inspection system will

exceed a threshold amplitude As an example, let px and p represent the probability distribution functions for the

location, x, and orientation, θ, of a given size, shape, and type of defect, and let S(x, θ) represent its signal amplitude, as

simulated by the measurement model An expression for POD, assuming a noise level N and a detection threshold amplitude T, can then be written formally as:

(Eq 5)

where the integrals are taken over the range of orientations and positions of possible flaws (Ref 15) Obvious generalizations of Eq 5 can treat the effects of flaw shape, type, and so on

Equation 5 does not contain an explicit factor to represent the probability distribution of the presence of flaws; therefore,

it predicts the probability that an ultrasonic indication (signal plus noise) will be detected assuming that a flaw is, in fact, present The use of the POD model to analyze reliability issues, such as the probabilities of falsely accepting flawed components or of falsely rejecting good ones, would require the incorporation into the model of the probability distribution function for flaws as a function of defect size, location within the part, and so on The result of Eq 5 would then need to be further integrated with respect to that probability distribution

Applications. Figure 5 shows the results of simulating the detectability (POD) of circular cracks at three different depths below a cylindrical component surface and for two different scan plans (Ref 6) Figure 5(a) illustrates the use of

the POD model to quantify the detection capability of an NDE system For the specific parameters in that simulation, cracks that are otherwise identical have significantly different detectability levels, depending on their depth below the surface of the part In this example, the curved surface of the component behaves like an acoustic lens that happens to focus the beam at the middle depth (25 mm, or 1 in.) Because of the relatively coarse scan mesh and the reduced beam width in this focal region, there is a significant likelihood that a flaw will be located in a low-amplitude portion of the beam profile and, therefore not be detected For the other two depths, the beam width is greater than the scan mesh distance The plot in Fig 5(b) shows the result of sufficiently refining the scan mesh so that the beam width at the focal region is greater than the distance between scan points In this case, the POD curves are nearly the same for the three different depths This example illustrates the use of the POD model for quantifying the capability of a flaw detection system and for suggesting improvements in this area in the system or its operation

Fig 5 Predicted influence of scan plan on POD (a) POD at three depths for axial and circumferential scan

increments of 2.5 mm (0.1 in) (b) POD at three depths for axial and circumferential scan increments of 5 and 1.3 mm (0.2 and 0.05 in.), respectively

Another example of the use of the POD model is shown in Fig 6 (Ref 6) The POD curve is expressed in a rather nonstandard manner, because it does not represent the typical POD versus flaw size plot Instead, Fig 6 shows the variation in POD due to modification of a geometrical parameter of a component Specifically, the component is a simulated turbine disk assumed to contain radially oriented circular cracks below a bicylindrical fillet The abscissa of Fig 6 is the in-plane radius of curvature of that fillet The flaw size is assumed to be constant, representing, for example, the critical flaw size as predicted by fracture mechanics Most important, Fig 6 shows that flaw detectability can be improved by modifying the geometry of the component This information is important for the definition of sonic near-net shapes of components in production, for example, and ensuring in-service inspectability during maintenance Moreover, because the characteristics of the inspection system are contained explicitly in the POD model, the scan plan required to achieve the necessary detectability levels are easily determined This concept of predicting component inspectability at the design stage, of determining the component design parameters that favorably influence flaw detectability, and of incorporating the requisite scan procedure into the design data base is perhaps the ultimate application of inspectability modeling

Fig 6 Simulated POD for a 0.8 mm (0.03 in.) diam circular crack below a bicylindrical fillet as a function of fillet

radius of curvature

References cited in this section

3 R.B Thompson and H.N.G Wadley, The Use of Elastic Wave-Material Structure Interaction Theories in

NDE Modeling, CRC Crit Rev Solid State Mater Sci., in press

4 J.M Coffey and R.K Chapman, Application of Elastic Scattering Theory for Smooth Flat Cracks to the

Quantitative Prediction of Ultrasonic Defect Detection and Sizing, Nucl Energy, Vol 22, 1983, p 319-333

6 T.A Gray and R.B Thompson, Use of Models to Predict Ultrasonic NDE Reliability, in Review of Progress

in Quantitative Nondestructive Evaluation, Vol 5, D.O Thompson and D.E Chimenti, Ed., Plenum Press,

1986, p 911

9 B.A Auld, General Electromechanical Reciprocity Relations Applied to the Calculation of Elastic Wave

Scattering Coefficients, Wave Motion, Vol 1, 1979, p 3

10 R.B Thompson and T.A Gray, A Model Relating Ultrasonic Scattering Measurements Through

Liquid-Solid Interfaces to Unbounded Medium Scattering Amplitudes, J Acoust Soc Am., Vol 74, 1983, p 1279

11 R.B Thompson and T.A Gray, Analytical Diffraction Corrections to Ultrasonic Scattering Measurements,

in Review of Progress in Quantitative Nondestructive Evaluation, Vol 2, D.O Thompson and D.E

Chimenti, Ed., Plenum Press, 1983, p 567

12 R.B Thompson and E.F Lopes, The Effects of Focusing and Refraction on Gaussian Ultrasonic Beams, J

Nondestr Eval., Vol 4, 1984, p 107

13 R.B Thompson, T.A Gray, J.H Rose, V.G Kogan, and E.F Lopes, The Radiation of Elliptical and

Bicylindrically Focused Piston Transducers, J Acoust Soc Am., Vol 82, 1987, p 1818

14 B.P Newberry and R.B Thompson, A Paraxial Theory for the Propagation of Ultrasonic Beams in

Anisotropic Solids, J Acoust Soc Am., to be published

15 B.P Newberry, R.B Thompson and E F Lopes, Development and Comparison of Beam Models for

Two-Media Ultrasonic Inspection, in Review of Progress in Quantitative Nondestructive Evaluation, Vol 6, D.O

Thompson and D.E Chimenti, Ed., Plenum Press, 1987, p 639

16 C.F Ying and R Truell, Scattering of a Plane Longitudinal Wave by a Spherical Obstacle in an

Isotropically Elastic Solid, J Appl Phys., Vol 27, 1956, p 1086

17 J.L Opsal and W.M Visscher, Theory of Elastic Wave Scattering: Applications of the Method of Optimal

Truncation, J Appl Phys., Vol 58, 1985, p 1102

18 J.-S Chen and L.W Schmerr, Jr., The Scattering Response of Voids A Second Order Asymptotic Theory,

in Review of Progress in Quantitative NDE, Vol 7, D.O Thompson and D.E Chimenti, Ed., Plenum Press,

1988, p 139

19 L Adler and J.D Achenbach, Elastic Wave Diffraction by Elliptical Cracks: Theory and Experiment, J

Nondestr Eval., Vol 1, 1980, p 87

20 B.D Cook and W.J Arnoult III, Gaussian-Laguerre/Hermite Formulation for the Nearfield of an Ultrasonic

Transducer, J Acoust Soc Am., Vol 59, 1976, p 9

21 J.D Achenbach, A.K Gautesen, and H McMaken, Ray Methods for Waves in Elastic Solids, Pittman

Publishing, 1982

22 S.O Rice, Mathematical Analysis of Random Noise, Bell Syst Tech J., Vol 23, 1944, p 282; Vol 24, 1945,

p 96

Models for Predicting NDE Reliability

J.N Gray, T.A Gray, N Nakagawa, and R.B Thompson, Center for NDE, Iowa State University

Eddy Current Inspection Model

The eddy current NDE method has a long history of use (Ref 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, and 34) Because

of its conceptually straightforward design principle, the method allows measurements to be made in a noncontacting, single-sided inspection geometry Eddy current is therefore a cost-effective and truly nondestructive method of inspection However, as a fully quantitative method, eddy current inspection has not achieved the level of sophistication found in other methods, such as ultrasonics and x-ray radiography The reason for the delay in its development is associated with the fundamental nature of the measurement In essence, the measurement response is a complex function

of the probe fields and their interaction with the flaw, and it is difficult to isolate these two contributions Therefore, one cannot develop simple models in which various effects can be described by separate factors, as in the case of ultrasonics

A significant problem has also been obtaining material property data (such as permeability and conductivity) for complex engineering materials Welds are especially difficult

From a modeling perspective, it is necessary to obtain electromagnetic field solutions for a given probe/flaw system by solving Maxwell's equations When written in the form suitable to eddy current problems, the equations show that the basic dynamics are of a dissipative, nonscattering nature One well-known consequence is that only near-surface flaws, confined within a region of a finite skin depth, are detectable Putting this obvious limitation aside, there is a subtler problem posed by the basic principle Namely, electromagnetic fields spread over a wide region outside the specimen, permitting no simple way of focusing the probe sensitivity to flaw regions (In the other inspection methods, the beam focusing, for example, can be used for this purpose) The impedance signal, being an integrated quantity, contains not only flaw information but also redundant environment information (such as probe lift-off), which is uninteresting in terms

of flaw detection and characterization and may contribute significant noise Extracting flaw information from impedance signals therefore becomes highly dependent on component geometry and is a computationally intensive task

Many efforts have been devoted to overcoming this difficulty, and the results have been promising (Ref 25, 26, 27, 28,

29, 30, 31, and 32) Rapid progress has occurred recently in computational methods with regard to both hardware and software It appears that the new-generation desktop workstations are sufficiently capable of handling the computational requirements of eddy current data analyses One may find an even better situation when using eddy current models to