Nondestructive Evaluation and Quality Control (1998) Part 15 doc

It must be emphasized that quantitative data are not necessary for the multitude of NDE applications used to add confidence to routine process control and other applications in which the

Fig 8 Residual life analysis and test procedure

A typical test configuration for a cyclic spin test is shown in Fig 9, and typical empirical data are shown in Fig 10, 11,

12 Verification requirements may also include engine test with components that are preflawed or cracked in critical locations to determine the effects of the real environment (temperature and gradient, vibration, and so on) Such tests must

be closely controlled and monitored using the inspection requirements planned for service engines to ensure safety of the test engine

Fig 9 Typical setup for a cyclic spin test (heated spin pit) of turbine rotor Arrows indicate critical locations that

were preflawed to simulate the worst expected damage

Fig 10 Crack progagation as influenced by increases in three parameters (a) Frequency, f (b) Stress ratio, R

(c) Temperature, T The variable K is the calculated stress at the crack tip

Fig 11 Actual and predicted flaw growth for an engine test on a second-stage high-pressure turbine disk

forward cooling air hole

Fig 12 Methodology correlation (specimens and components)

Nondestructive evaluation requirements are implemented on fracture-critical parts during manufacture and during field/depot inspection to ensure safety Specific inspection requirements are derived through design analysis trade-offs among initial flaw size assumption, stress level, and material properties for a given usage (stress environment spectrum)

As discussed in the section "Initial Flaw Size" in this article, a flaw size assumption (for surface flaws) of less than 0.75

mm (0.030 in.) requires implementation of enhanced NDE (that is, eddy current inspection) The ability of eddy current inspection to reliably detect surface flaws having depths of 0.13 mm (0.005 in.) has been demonstrated in several applications Primary emphasis on the use of eddy current inspection is for stress concentration areas in which a small flaw size assumption is required to achieve the necessary residual life without excessive weight penalty Typical probability of detection data for eddy current inspection in Fig 13

Fig 13 POD curves with eddy current inspection All curves are for lower 95% confidence limit

In general, fluorescent penetrant inspection is specified for areas in which the detection of a flaw with a surface length of 0.75 mm (0.030 in.) or larger is required to achieve the specified residual life However, the ability of current fluorescent penetrant inspection processes to reliably detect 0.75 mm (0.030 in.) flaws is not clear Therefore, in some cases, eddy current inspection may be specified for large areas if susceptibility data (that is, probabilistic data on the capability of fluorescent penetrant inspection) indicate the need Data generated on numerous demonstration programs clearly indicate that the fluorescent penetrant inspection process can be significantly improved through upgraded training, equipment, and procedures (proper cleaning, including etch, hydrophilic emulsifier, and wet developer) These demonstration programs have been conducted on several engine development programs (F100, F101, F110, F110-GE-129, F100-PW-229) and laboratory technology programs (Air Force Wright Research and Development Center) Some typical detection improvements that have been demonstrated for upgraded fluorescent penetrant inspection processes are shown in Fig 14 The critical need is to implement the best fluorescent penetrant inspection process within industry and within the Air Force logistics centers because this method will likely remain the most widely used for inspecting large areas for cracklike damage

Fig 14 Probability of detection with fluorescent penetrant inspection LCL, lower confidence limit

Another critical NDE need is to quantify the POD of ultrasonics to detect imbedded defects in bulk volumes and to develop inspection methods for finished shapes Very limited data indicate that reliable detection limits may be as large as 1.3 mm2 (0.002 in.2) (approximately equal to a planar disk of 1.2 mm, or in., diameter) The goal is to develop and implement ultrasonic inspection methods such that a residual life equal to two times the required life or the inspection interval can be achieved, assuming the largest undetectable flaw size without excessive impact on weight

Retirement-for-Cause (RFC). Traditionally, components whose dominant failure mode is low-cycle fatigue have been designed to a crack initiation criterion With this approach, only 1 component in a population of 1000 would have actually initiated a crack, and the remaining 999 components would be discarded with substantial undefined useful life to crack initiation remaining Figure 15 shows that the difference between the number of cycles to reach the design-allowable curve and the population average curve for an average component would have consumed only 10% or less of its potential useful life-to-crack initiation Under the initiation criterion, there is no way to utilize this potential life without accepting a higher probability of failure of the remaining components

Fig 15 Stress versus loading cycles to crack initiation for Inconel 718 Temperature is 540 °C (1000 °F), and

the ratio of alternating stress to mean stress (A ratio) = 1

Under the RFC concept, this additional useful life can be utilized by adopting a rejection criterion that uses each component in a population until it specifically initiates a crack, rather than rejecting the entire population on the behavior

of the statistical minimum The development of fracture mechanics concepts over the last several years has permitted the degree of predictability for crack progagation rates necessary to implement such an approach on a safe basis

The RFC concept would apply the fracture control philosophy (Fig 4) to life management In using RFC as an operating system, all components would be inspected first at the end of a safety limit period divided by an appropriate safety

margin, and only those components containing detectable cracks equal to or greater than ai would be retired or repaired

All others would be returned for additional service (with the assumption that if a flaw existed, it would be smaller than aifor another inspection interval) In this way, the crack progagation residual life is continually reset to a safe value By following this approach, components are rejected only for cause (cracks), and each component is allowed to operate for its own specific crack initiation life It should be noted that if a crack is missed at the first inspection interval, another chance should exist to find a larger crack

It is clear that not all fatigue-limited components can be handled in this way and that each component must be evaluated individually to determine the technical feasibility of RFC Low-cycle fatigue is a real physical phenomenon that is not directly associated with the presence of defects Any criterion, initiation or otherwise, that allows components to run beyond a design-allowable life (-3 , for example) or beyond the average life will inherently result in a significant increase in the probability of a large number of cracks and possible failures Such an increase in risk may be acceptable, but must be understood and evaluated Any inspection process has associated with it a finite probability of detection and therefore a finite probability of missing real cracks (not just defects assumed to be there and assumed to be in the form of sharp cracks) Missing real cracks and presuming that crack growth knowledge is sufficient to detect cracks at the next inspection has significantly more risk associated with it than concern over possible defects

The economic feasibility of RFC must also be evaluated The inspection interval must be such that it does not place undue constraints on the operation of the component or that the cost of the necessary tear-down and inspection does not negate the advantage of the life extension It seems unlikely that RFC can be applied to components limited by high-cycle fatigue considerations, but for many high-cost components limited by low-cycle fatigue, such as engine disks, this approach does offer significant economic advantages

It is also clear that in applying RFC, nondestructive evaluation becomes a critical factor The crack length determines the residual life of the component, and its detection is limited by the resolution and reliability of the inspection system employed In many cases, the decision as to whether or not RFC can be applied to a component will be predicated upon

the ability of available NDE approaches to detect the initial flaw, ai, with sufficient sensitivity and reliability Because RFC procedure is based on fracture control concepts, the NDE techniques can be selected, refined, and focused on a particular local area, rather than attempting to critically inspect large areas

Fracture Control Philosophy

William D Cowie, United States Air Force, Aeronautical System Division, Propulsion Directorate

Selected References

• W.D Cowie and T.A Stein, "Damage Tolerant Design and Test Considerations in the Engine Structural Integrity Program," Paper presented at the propulsion session of the 21st Structures, Structural Dynamics, and Materials Conference, Seattle, WA, American Institute of Aeronautics and Astronautics, May 1980

• T.T King (ASD/EN), W.D Cowie (ASD/YZEE), and W.H Reimann (AFWL/MLLN), "Damage Tolerance Design Concepts for Military Engines," Paper presented at AGARD Conference 393, San Antonio, TX, Advisory Group for Aerospace Research and Development, April 1985

• C.F Tiffany and W.D Cowie, "Progress on the ENSIP Approach to Improved Structural Integrity in Gas Turbine Engine/An Overview," Paper 78-WA/GT-13, presented at the Winter Annual Meeting (San Francisco), American Society of Mechanical Engineers, Dec 1978

Applications of NDE Reliability to Systems

Ward D Rummel, Martin Marietta Astronautics Group; Grover L Hardy and Thomas D Cooper, Wright Research & Development Center, Wright-Patterson Air Force Base

Introduction

THE SUCCESS of a reliable NDE application depends greatly on the expertise and thoroughness of the NDE engineering that is performed This involves comprehensive analyses to define the relationships between the NDE measurements to be made and the impact on the system functions being assessed and on the capability to implement the NDE measurements

to discriminate to the expected level of acceptance applied Most failures in NDE systems applications and in the automation of an NDE system can be attributed to failures in NDE engineering and to unrealistic NDE performance expectations All modern engineering is based on comprehensive applications of principles that can be implemented by qualitative measurements and predictive modeling and on comprehensive procedural applications of principles based on prior art and systems test data Modern engineering methods are equally applicable to the use of NDE on a material, component, structure, or system The principles and data available are, however, not as well defined, generally recognized, or understood as those of other engineering disciplines

Applications of NDE Reliability to Systems

Ward D Rummel, Martin Marietta Astronautics Group; Grover L Hardy and Thomas D Cooper, Wright Research & Development Center, Wright-Patterson Air Force Base

General Considerations

Prior Art. Particular concern must be given to NDE assessment and analyses based on prior art Although NDE methods and principles have been applied since the beginning of time and have been applied specifically to quantitative materials evaluation during the last decade, the perceived performance level is often considerably different from the actual performance level Differences can be attributed to the primitive level of understanding of materials, engineering, and NDE engineering sciences and principles; excessive optimism in effecting early application of an NDE procedure; economic and social pressure to solve problems with troublesome engineering systems; and the attitudes and practices of our legal system

Progressive developments in the evaluation of NDE reliability have established a new dimension for the assessment of NDE performance Applications of prior NDE art must be judiciously examined to determine suitability for new applications Quantification of the performance level, calibration (or process control) methods, and acceptance criteria are

particularly important in extrapolating the applicability of prior NDE data to a new NDE engineering problem Conversely, the lessons learned in applications of prior art can be very useful in design and calibration and process control procedures, in establishing the characteristics and performance of materials and equipment used, in anticipating the problems and controls necessary to effect application in a production environment, and in assessing human factors relevant to the application

Incorporation and integration of the qualitative factors and considerations in the application of prior art are essential for making the transition from laboratory test data to production line use However, careful analysis and criteria must be applied to quantitative data from application of prior art Quantitative assessment of ongoing NDE process applications has shown that performance levels may vary considerably in NDE applications to established specification/process requirements Performance variations are rarely integrated into overall system reliability estimates and management is rarely accurate in recognizing and identifying superior performance by human operators Quantitative data must be supported by actual NDE system measurements and accurate descriptions of NDE materials, equipment, procedures, and human operator qualifications to be seriously considered in NDE system design or qualification by similarity It must be emphasized that quantitative data are not necessary for the multitude of NDE applications used to add confidence to routine process control and other applications in which the NDE procedure does not constitute final acceptance of performance characteristics Quantitative data are required when the NDE measurement/acceptance is integral to design acceptance and/or performance acceptance

NDE Response. The response from an NDE system or process may take the form of a signal output (or outputs) or a direct or indirect image Acceptable conditions can be differentiated from unacceptable conditions by threshold discrimination from the electronic output or by pattern recognition and threshold discrimination by image analyses Discrimination can be automated or performed by a human operator Discrimination of threshold electronic signals can be automated or gated to alert the human operator The consistency and reliability of electronic signal discrimination can often be improved by automating the discrimination process Superior consistency and reliability of pattern recognition and discrimination level for images are usually achieved by the human operator The feasibility of application of NDE to

a system is dependent on the establishment and characterization of a relationship between the response from an NDE output and a desired engineering system performance parameter

A direct or indirect relationship between an NDE response and a system performance characteristic may be functional under laboratory conditions, but may be impractical in applications under production or service conditions Factors such

as calibration, acceptance criteria, component accessibility, surface condition, inspection material compatibility, and inspection environment must be assessed to determine that a positive relationship between NDE response and system performance can be maintained

NDE System Management and Schedule. The implementation of a reliable NDE procedure is dependent on

allowing time to collect data, perform the critical analyses, apply required resources, and develop operator (personnel) skills Many critical NDE procedures have been implemented as a result of unanticipated engineering system failures The economic and social pressures resulting from an engineering system failure must be judiciously balanced against the required time and resources necessary to perform disciplined and thorough NDE engineering analyses, procedure development, and procedure validations After the required procedures have been implemented, NDE system/process control must be maintained to ensure a consistent level of discrimination Shortcuts in NDE engineering, NDE procedure development, and NDE system/process control increase risks in system performance, may not reduce the risk of engineering system failure, and may contribute to a false confidence level in system performance

Applications of NDE Reliability to Systems

Ward D Rummel, Martin Marietta Astronautics Group; Grover L Hardy and Thomas D Cooper, Wright Research & Development Center, Wright-Patterson Air Force Base

NDE Engineering

The difference between NDE engineering and the classical engineering disciplines result from the variety of problems and the indirect nature of NDE measurements on engineering system performance The functional performance of most NDE methods can be measured, controlling parameters can be documented, and performance output can be modeled; however, the interaction of the NDE method with the test object necessitates the generation of new response parameters and

characteristics for many new applications In addition, variations in material properties, geometry, surface condition, access, or environmental conditions may modify the NDE responses Therefore, NDE engineering is an essential element

of critical engineering system design, qualification, acceptance, and life cycle management A critical design is not complete until the NDE engineering has been performed and NDE system/process performance validated to functional design requirements and acceptance criteria levels

Procedure Selection/Development. Trade studies to identify and select candidate NDE procedures are needed for establishing the most economical and reliable procedure that meets acceptance requirements The process may be satisfied by the assessment of prior art and applications to similar problems or may require research and development of totally new procedures For demanding applications, a combination of complementary NDE procedures may be required

to meet the acceptance criteria objectives One or more methods can be further characterized and assessed to ensure that the performance objectives, NDE performance margins, NDE costs, engineering system performance risks, and the risks

of NDE system/process false alarms can be balanced in overall engineering system management

System/Process Performance Characteristics. Although care, discipline, and control measures are applied to ensure a consistent output from an NDE system or process, the output will vary within the established control parameters and as a result of slight variations in engineering hardware materials properties, geometry, surface condition, and so on If repetitive applications are made, a probability density distribution of signal/image output will be generated This distribution is similar to that obtained by repetitive measurements of a dimension such as a hole diameter or the length of

a bolt

Nondestructive measurements are usually indirect, and positive signals may be generated from nonrelevant sources, such

as surface roughness, grain structure, and geometry variations Such signals constitute the application noise inherent in a specific NDE process or procedure Discrimination of NDE signal/image outputs must be derived from those signal levels/amplitudes that exceed the level of the application noise (Fig 1) Analysis of signal and signal plus noise are common in electronic devices, optics, and other discrimination processes Similarly, the signal-to-noise margin (ratio) is a primary factor in establishing the level of discrimination of an NDE procedure Signal/noise reduction procedures can be used to enhance the overall performance of an NDE procedure However, it is important to recognize that the dominant noise source in an NDE process is not electronic noise that may be reduced by filtering, multiple sampling, and averaging techniques, but is instead the noise due to nonrelevant signals generated in applying the NDE procedure to a specific hardware element

Fig 1 Signal/noise density distribution for a large flaw (a), a medium flaw (b), and a small flaw (c)

Conditional Probability in NDE Discrimination. Nondestructive evaluation involves the measurement of complex parameters with inherent variations in both the measurement process and the test object The output from such a measurement/decision process can be analyzed as a problem in conditional probability When an NDE assessment is performed for the purpose of crack detection, the outcome is not a simple accept/reject (binary) process, as is frequently envisioned It is actually the product of conditional acceptance due to the interdependence of the measurement and decision responses Figure 2 shows the four possible outcomes that result from the application of NDE procedure for crack detection As shown in Fig 2, the possible outcomes from an inspection process are:

• True positive (TP): A crack exists and is detected, where M(A,a) is the total number of true positives and P(A,a) is the probability of a true positive

• False positive (FP): No crack exists but one is identified, where M(A,n) is the total number of false positives and P(A,n) is the probability of a false positive

• False negative (FN): A crack exists but is not detected, where M(N,a) is the total number of false negatives and P(N,a) is the probability of a false negative

• True negative (TN): No crack exists and none is detected, where M(N,n) is the total number of true negatives and P(N,n) is the probability of a true negative

The interdependence of these matrix quantities can be expressed as:

M(A,a) + M(N,a) = (TP) and (FN) outcomes giving the total opportunities for positive calls (Total number of

defects)

and

M(A,n) + M(N,n) = Total opportunities for false alarms from the possible (FP) and (TN) outcomes

Fig 2 Matrix of four possible outcomes from an NDE procedure for flaw detection

Because of the interdependent relationship, only two independent probabilities need be considered to quantify the

inspection/decision task The probability of detection (POD) or probability for a true positive P(A,a) can be expressed as:

Similarly, the probability of false alarms (POFA) or the probability for a false positive P(A,n) can be expressed as:

Signal/Noise Relationships. The desired results of the application of NDE procedure are crack detection (signal present) or crack nondetection (signal absent) The basis for detection is that of sensing a signal response and determining that the signal response is above a predetermined threshold Both sensing and interpretation are dependent on the signal (plus noise) and the noise (application background or response to nonrelevant parameters) that are subjected to the discrimination media (programmed machine discriminator or human operator)

If an NDE procedure is repetitively applied to a single flaw of a given size (in a part with a fixed geometry, surface condition, and so on), the output responses can be used to plot probability density distributions of both flaw signal and application noise responses Under ideal conditions, such as the response from a large flaw, the signal and noise distributions will be well separated, as shown in Fig 1(a)

The discrimination of flaw responses from application noise responses is a simple process; POD will be high, and the POFA will be low In practical engineering applications, the flaw size is not fixed (and is rarely large), and the discrimination process is more complex Indeed, the discrimination process is applied to a continuous range of flaw sizes, where the capability for discrimination is dependent on the inherent performance characteristic of the NDE procedure and

on the separation of the signal (plus noise) from the noise response of the process

If the NDE procedure is applied to a single flaw of intermediate size (in a part with the same fixed surface finish, geometry, and so on), the output responses can be used to generate probability density distributions for signal and noise,

as shown in Fig 1(b) For this flaw size, the distributions overlap (in part), and the capability for discrimination is dependent on the response from a single set of output signals within these distributions If the single set has output signals that are well separated (that is, signals at the outer extremes of the distributions), the output response will be interpreted as acceptable (no flaw condition) for those cases where the threshold response acceptance level is located between the signal and noise signals If the single set of outputs lies at the inner extremes of the distributions, the output response may be interpreted as acceptable (no flaw or undetected flaw condition) or may be interpreted as unacceptable (false alarm condition) for the same threshold response acceptance level For this condition, the POD will be lower and the POFA will

be higher than for the case of discrimination with positive signal/noise separation margins If the process is repeated for a small flaw (under the same operating conditions), the signal and noise response distribution will approach coincidence, as shown in Fig 1(c) The POD will be low, and the POFA will be high

It is clear that the performance capability of a given NDE procedure is dependent on the nature and distribution of the signal outputs generated under the conditions of application It is also clear that the threshold acceptance criterion applied

in the discrimination process is an important factor in the successful application of a procedure Consider the application

of an NDE procedure to a large flaw under conditions that produce a significant separation of probability density distributions of signal and noise, as shown in Fig 3 If the threshold acceptance criterion (represented by the vertical arrow) is placed at too high a level (Fig 3a), some of the flaws will be missed (reduce POD) If the acceptance criterion is placed at a proper level (Fig 3b), clear discrimination will result (high POD) If the acceptance criterion is placed too low (Fig 3c), all of the flaws will be rejected; but some false alarms will result, and good parts will be rejected (high POFA)

Fig 3 Influence of acceptance criterion (vertical arrow) on process discrimination (a) Acceptance criterion too

high (b) Acceptance criterion at proper level (c) Acceptance criterion too low

The NDE procedure performance characteristics of primary importance are the signal-to-noise ratio (separation margin) and the threshold acceptance criteria applied in the discrimination process Optimum NDE procedure performance can be obtained by characterizing an NDE procedure and by matching the threshold acceptance criteria to the performance capabilities of the NDE procedure Such characterization also enables the assessment and quantification of risks that result from changes in acceptance criteria

Reference Standards. Historically, reference standards for NDE methods of defect detection have been used to ensure the reproducibility of the application of the method(s) and to establish an acceptable quality of the process rather than to establish the dimensions or other applicable parameter of the defect or anomaly Some methods (such as those used for thickness gaging or electrical conductivity determination) were able to provide an extremely accurate measurement of the appropriate parameter, usually by extrapolation between two known and closely spaced reference standards representative

of the condition to be determined The remaining methods, as typified by radiography, offer at best only a crude estimate

of the dimensions, orientation, shape, or other characteristics of the detected defects

The primary reason for not using reference standards for the quantitative evaluation of defects was, and still is, that the NDE methods respond to most of the parameters of a defect simultaneously; in most cases, there is no way to separate the response from a single parameter, or there is not an accurate response to a single parameter in other cases Consider, for example, the case of penetrant inspection The indications formed are usually greater in length, width, or area than the discontinuity present because of the flow of the penetrant material out of the discontinuity during development When this excess material is removed and the indication is viewed as it starts to appear again, the full length of the discontinuity, if it

is linear, will initially not be revealed, because the ends provide little or no penetrant for formation of the indication As the indication continues to form, it will eventually reach the same length as the discontinuity and will then continue to grow as additional penetrant flows to the surface This lack of response to the extremities of a discontinuity is common to all NDE methods and illustrates the difficulties associated with using NDE methods for sizing defects

Quantitative NDE, however, requires that a good estimate be made of the defect size that is detected or, more important, the size of the largest defect that might be left in the part Because the NDE methods, by the laws of physics, are

inherently inaccurate in sizing, the only available approach is to make a conservative estimate of the size of the defects that can remain The approach requires a second type of standard used along with the conventional reference standards This second type, called a qualification standard, contains defects that represent the worst case for both flaw detection and crack growth (generally a surface fatigue crack) Fatigue cracks have the advantages that they can be grown in the laboratory and, when produced under well-controlled conditions, have predictable geometries The qualification standards are then used in sets to define the lower limit of the flaw size that a given NDE process can reliably detect The conventional reference standards are used to control the NDE process; therefore, once the qualification standards establish the process sensitivity, the reference standards can be used to ensure that the sensitivity is maintained Consequently, there are certain requirements that reference standards must meet

First, reference standards must produce a response comparable to that produced by the smallest qualification standard flaw that is considered reliably detectable This comparability includes not only response to the flaw itself but also the geometry in which the flaw is contained In the ultrasonic method, for example, the response to a fatigue crack located in the center of a flat plate can be far different from the response to the same size crack that is located in the bore of a large-diameter hole Consequently, application of the ultrasonic method for inspection of the two geometries requires qualification standards as well as reference standards for both geometries

Second, reference standards for a specific inspection must be relatable to other reference standards used for the same inspection That is, when several reference standards are available, the responses for each one must be known, and more important, the differences in responses for the standards must be known so that adjustments can be made in the inspection

to ensure that a uniform process sensitivity can be maintained Ideally, the responses of all reference standards for a given inspection should be identical; however, from a practical standpoint this is impossible to achieve As stated previously, the response of the NDE methods is from a multitude of parameters associated with a given discontinuity

For example, if an electrically discharge machined slot is selected as a defect for a reference standard for an ultrasonic inspection, exact control over the size of the slot in each standard is not sufficient to guarantee identical responses Slight variations in the orientation of the slot with respect to the surface of the standard and in the surface finish of the slot itself can cause noticeable differences in the ultrasonic response This is the worst case; other NDE methods vary in their response to subtle geometric parameters, with the magnetic particle and penetrant methods probably being the most tolerant However, even these methods are highly sensitive to the width of the flaw used in a reference standard

Another important property of reference standards is durability Both the material and the type of flaw in a reference standard must be selected so that the standard will not readily deteriorate or change in the environment in which it will be used These selections are affected by the NDE method for which the standard is intended Both ultrasonic and penetrant methods are very sensitive to the presence of foreign material inside the flaw For ultrasonics, this can affect the amount

of energy that is reflected from the flaw For penetrant inspection, the quantity of penetrant that can enter the defect, and consequently the brightness of the indication, will be reduced The foreign material may be fluids, soils, or corrosion products For these reasons, magnesium, ferritic steels, and aluminum are particularly poor choices for reference standards for penetrant inspection and require some type of protection if used as ultrasonic standards

Eddy current methods are not affected by foreign material in the flaw, but are very sensitive to such surface conditions as scratches, pitting, and corrosion Magnetic particle methods are sensitive to the width of the defect and to anything, such

as cold working, that may change the magnetic permeability of the standard Radiographic methods are sensitive to the thickness of the standard and to changes in the shape of the flaw In general, a good choice of material for any method is one that is reasonably hard and forms an adherent, tough, and stable surface oxide layer (such as a titanium or nickel-base alloy), thus providing protection against mechanical damage and the gradual buildup of corrosion products

Personnel. Unless the inspection process is fully automated, the proficiency of the inspection personnel is the largest variable affecting inspection reliability This proficiency varies widely not only from inspector to inspector but also with the same inspector, depending on his working environment and his mental condition For fully automated inspections, the proficiency of the inspector in operating the equipment is important but has little or no impact on inspection reliability

The first task in addressing the contribution of the inspector to inspection reliability is to ensure that he is knowledgeable

of the specific techniques to be used and has the basic proficiency to perform the inspection to the required reliability Experience has demonstrated that the previous qualifications of the inspector for example, certification to MIL-STD-410D (Ref 1) are not sufficient to ensure the desired performance with a new inspection that must be performed with high reliability Therefore, the most straightforward way to assess proficiency is to require inspector participation in the demonstration of inspection reliability All inspectors that will be required to perform the inspection should also participate This not only establishes the reliability of the proposed inspection but also identifies those personnel requiring

additional training or experience before they can be expected to perform adequately Careful observation of the inspector during the demonstration and of the results obtained is necessary to identify the additional training or experience needed After training and/or additional experience is acquired, the demonstration effort can be repeated to indicate if the inspector has become sufficiently proficient in the inspection technique

After basic inspection proficiency has been demonstrated, it becomes a supervisory task to ensure that this proficiency is maintained Control of the work environment of the inspector is important Distractions such as noise, extremes in temperature, and other irritants should be eliminated to the extent possible Break periods should be frequent enough to reduce fatigue Personnel who are ill or otherwise physically impaired should be temporarily assigned other tasks to the extent possible Other efforts that improve or maintain a good mental attitude are excellent ways to ensure sustained inspection reliability These include providing acceptable materials and equipment with which to conduct the inspection Finally, when it is not possible to provide a consistently conducive environment for a highly reliable inspection, two inspectors can perform the same inspection independently to achieve higher reliability than can be obtained with a single inspector Two inspectors generally will not make identical mistakes

Reference cited in this section

1 "Nondestructive Testing Personnel Qualification and Certification," MIL-STD-410D, 25 June 1974

Applications of NDE Reliability to Systems

Ward D Rummel, Martin Marietta Astronautics Group; Grover L Hardy and Thomas D Cooper, Wright Research & Development Center, Wright-Patterson Air Force Base

NDE Engineering

The difference between NDE engineering and the classical engineering disciplines result from the variety of problems and the indirect nature of NDE measurements on engineering system performance The functional performance of most NDE methods can be measured, controlling parameters can be documented, and performance output can be modeled; however, the interaction of the NDE method with the test object necessitates the generation of new response parameters and characteristics for many new applications In addition, variations in material properties, geometry, surface condition, access, or environmental conditions may modify the NDE responses Therefore, NDE engineering is an essential element

of critical engineering system design, qualification, acceptance, and life cycle management A critical design is not complete until the NDE engineering has been performed and NDE system/process performance validated to functional design requirements and acceptance criteria levels

Procedure Selection/Development. Trade studies to identify and select candidate NDE procedures are needed for establishing the most economical and reliable procedure that meets acceptance requirements The process may be satisfied by the assessment of prior art and applications to similar problems or may require research and development of totally new procedures For demanding applications, a combination of complementary NDE procedures may be required

to meet the acceptance criteria objectives One or more methods can be further characterized and assessed to ensure that the performance objectives, NDE performance margins, NDE costs, engineering system performance risks, and the risks

of NDE system/process false alarms can be balanced in overall engineering system management

System/Process Performance Characteristics. Although care, discipline, and control measures are applied to ensure a consistent output from an NDE system or process, the output will vary within the established control parameters and as a result of slight variations in engineering hardware materials properties, geometry, surface condition, and so on If repetitive applications are made, a probability density distribution of signal/image output will be generated This distribution is similar to that obtained by repetitive measurements of a dimension such as a hole diameter or the length of

a bolt

Nondestructive measurements are usually indirect, and positive signals may be generated from nonrelevant sources, such

as surface roughness, grain structure, and geometry variations Such signals constitute the application noise inherent in a specific NDE process or procedure Discrimination of NDE signal/image outputs must be derived from those signal

levels/amplitudes that exceed the level of the application noise (Fig 1) Analysis of signal and signal plus noise are common in electronic devices, optics, and other discrimination processes Similarly, the signal-to-noise margin (ratio) is a primary factor in establishing the level of discrimination of an NDE procedure Signal/noise reduction procedures can be used to enhance the overall performance of an NDE procedure However, it is important to recognize that the dominant noise source in an NDE process is not electronic noise that may be reduced by filtering, multiple sampling, and averaging techniques, but is instead the noise due to nonrelevant signals generated in applying the NDE procedure to a specific hardware element

Fig 1 Signal/noise density distribution for a large flaw (a), a medium flaw (b), and a small flaw (c)

Conditional Probability in NDE Discrimination. Nondestructive evaluation involves the measurement of complex parameters with inherent variations in both the measurement process and the test object The output from such a measurement/decision process can be analyzed as a problem in conditional probability When an NDE assessment is performed for the purpose of crack detection, the outcome is not a simple accept/reject (binary) process, as is frequently envisioned It is actually the product of conditional acceptance due to the interdependence of the measurement and decision responses Figure 2 shows the four possible outcomes that result from the application of NDE procedure for crack detection As shown in Fig 2, the possible outcomes from an inspection process are:

• True positive (TP): A crack exists and is detected, where M(A,a) is the total number of true positives and P(A,a) is the probability of a true positive

• False positive (FP): No crack exists but one is identified, where M(A,n) is the total number of false positives and P(A,n) is the probability of a false positive

• False negative (FN): A crack exists but is not detected, where M(N,a) is the total number of false negatives and P(N,a) is the probability of a false negative

• True negative (TN): No crack exists and none is detected, where M(N,n) is the total number of true negatives and P(N,n) is the probability of a true negative

The interdependence of these matrix quantities can be expressed as:

M(A,a) + M(N,a) = (TP) and (FN) outcomes giving the total opportunities for positive calls (Total number of

defects)

and

M(A,n) + M(N,n) = Total opportunities for false alarms from the possible (FP) and (TN) outcomes

Fig 2 Matrix of four possible outcomes from an NDE procedure for flaw detection

Because of the interdependent relationship, only two independent probabilities need be considered to quantify the

inspection/decision task The probability of detection (POD) or probability for a true positive P(A,a) can be expressed as:

Similarly, the probability of false alarms (POFA) or the probability for a false positive P(A,n) can be expressed as:

Signal/Noise Relationships. The desired results of the application of NDE procedure are crack detection (signal present) or crack nondetection (signal absent) The basis for detection is that of sensing a signal response and determining that the signal response is above a predetermined threshold Both sensing and interpretation are dependent on the signal (plus noise) and the noise (application background or response to nonrelevant parameters) that are subjected to the discrimination media (programmed machine discriminator or human operator)

If an NDE procedure is repetitively applied to a single flaw of a given size (in a part with a fixed geometry, surface condition, and so on), the output responses can be used to plot probability density distributions of both flaw signal and application noise responses Under ideal conditions, such as the response from a large flaw, the signal and noise distributions will be well separated, as shown in Fig 1(a)

The discrimination of flaw responses from application noise responses is a simple process; POD will be high, and the POFA will be low In practical engineering applications, the flaw size is not fixed (and is rarely large), and the discrimination process is more complex Indeed, the discrimination process is applied to a continuous range of flaw sizes, where the capability for discrimination is dependent on the inherent performance characteristic of the NDE procedure and

on the separation of the signal (plus noise) from the noise response of the process

If the NDE procedure is applied to a single flaw of intermediate size (in a part with the same fixed surface finish, geometry, and so on), the output responses can be used to generate probability density distributions for signal and noise,

as shown in Fig 1(b) For this flaw size, the distributions overlap (in part), and the capability for discrimination is dependent on the response from a single set of output signals within these distributions If the single set has output signals that are well separated (that is, signals at the outer extremes of the distributions), the output response will be interpreted as acceptable (no flaw condition) for those cases where the threshold response acceptance level is located between the signal and noise signals If the single set of outputs lies at the inner extremes of the distributions, the output response may be interpreted as acceptable (no flaw or undetected flaw condition) or may be interpreted as unacceptable (false alarm condition) for the same threshold response acceptance level For this condition, the POD will be lower and the POFA will

be higher than for the case of discrimination with positive signal/noise separation margins If the process is repeated for a small flaw (under the same operating conditions), the signal and noise response distribution will approach coincidence, as shown in Fig 1(c) The POD will be low, and the POFA will be high

It is clear that the performance capability of a given NDE procedure is dependent on the nature and distribution of the signal outputs generated under the conditions of application It is also clear that the threshold acceptance criterion applied

in the discrimination process is an important factor in the successful application of a procedure Consider the application

of an NDE procedure to a large flaw under conditions that produce a significant separation of probability density distributions of signal and noise, as shown in Fig 3 If the threshold acceptance criterion (represented by the vertical arrow) is placed at too high a level (Fig 3a), some of the flaws will be missed (reduce POD) If the acceptance criterion is placed at a proper level (Fig 3b), clear discrimination will result (high POD) If the acceptance criterion is placed too low (Fig 3c), all of the flaws will be rejected; but some false alarms will result, and good parts will be rejected (high POFA)

Fig 3 Influence of acceptance criterion (vertical arrow) on process discrimination (a) Acceptance criterion too

high (b) Acceptance criterion at proper level (c) Acceptance criterion too low

The NDE procedure performance characteristics of primary importance are the signal-to-noise ratio (separation margin) and the threshold acceptance criteria applied in the discrimination process Optimum NDE procedure performance can be obtained by characterizing an NDE procedure and by matching the threshold acceptance criteria to the performance capabilities of the NDE procedure Such characterization also enables the assessment and quantification of risks that result from changes in acceptance criteria

Reference Standards. Historically, reference standards for NDE methods of defect detection have been used to ensure the reproducibility of the application of the method(s) and to establish an acceptable quality of the process rather than to establish the dimensions or other applicable parameter of the defect or anomaly Some methods (such as those used for thickness gaging or electrical conductivity determination) were able to provide an extremely accurate measurement of the appropriate parameter, usually by extrapolation between two known and closely spaced reference standards representative

of the condition to be determined The remaining methods, as typified by radiography, offer at best only a crude estimate

of the dimensions, orientation, shape, or other characteristics of the detected defects

The primary reason for not using reference standards for the quantitative evaluation of defects was, and still is, that the NDE methods respond to most of the parameters of a defect simultaneously; in most cases, there is no way to separate the response from a single parameter, or there is not an accurate response to a single parameter in other cases Consider, for example, the case of penetrant inspection The indications formed are usually greater in length, width, or area than the discontinuity present because of the flow of the penetrant material out of the discontinuity during development When this excess material is removed and the indication is viewed as it starts to appear again, the full length of the discontinuity, if it

is linear, will initially not be revealed, because the ends provide little or no penetrant for formation of the indication As the indication continues to form, it will eventually reach the same length as the discontinuity and will then continue to grow as additional penetrant flows to the surface This lack of response to the extremities of a discontinuity is common to all NDE methods and illustrates the difficulties associated with using NDE methods for sizing defects

Quantitative NDE, however, requires that a good estimate be made of the defect size that is detected or, more important, the size of the largest defect that might be left in the part Because the NDE methods, by the laws of physics, are inherently inaccurate in sizing, the only available approach is to make a conservative estimate of the size of the defects that can remain The approach requires a second type of standard used along with the conventional reference standards This second type, called a qualification standard, contains defects that represent the worst case for both flaw detection and crack growth (generally a surface fatigue crack) Fatigue cracks have the advantages that they can be grown in the laboratory and, when produced under well-controlled conditions, have predictable geometries The qualification standards are then used in sets to define the lower limit of the flaw size that a given NDE process can reliably detect The conventional reference standards are used to control the NDE process; therefore, once the qualification standards establish the process sensitivity, the reference standards can be used to ensure that the sensitivity is maintained Consequently, there are certain requirements that reference standards must meet

First, reference standards must produce a response comparable to that produced by the smallest qualification standard flaw that is considered reliably detectable This comparability includes not only response to the flaw itself but also the geometry in which the flaw is contained In the ultrasonic method, for example, the response to a fatigue crack located in the center of a flat plate can be far different from the response to the same size crack that is located in the bore of a large-diameter hole Consequently, application of the ultrasonic method for inspection of the two geometries requires qualification standards as well as reference standards for both geometries

Second, reference standards for a specific inspection must be relatable to other reference standards used for the same inspection That is, when several reference standards are available, the responses for each one must be known, and more important, the differences in responses for the standards must be known so that adjustments can be made in the inspection

to ensure that a uniform process sensitivity can be maintained Ideally, the responses of all reference standards for a given inspection should be identical; however, from a practical standpoint this is impossible to achieve As stated previously, the response of the NDE methods is from a multitude of parameters associated with a given discontinuity

For example, if an electrically discharge machined slot is selected as a defect for a reference standard for an ultrasonic inspection, exact control over the size of the slot in each standard is not sufficient to guarantee identical responses Slight variations in the orientation of the slot with respect to the surface of the standard and in the surface finish of the slot itself can cause noticeable differences in the ultrasonic response This is the worst case; other NDE methods vary in their response to subtle geometric parameters, with the magnetic particle and penetrant methods probably being the most tolerant However, even these methods are highly sensitive to the width of the flaw used in a reference standard

Another important property of reference standards is durability Both the material and the type of flaw in a reference standard must be selected so that the standard will not readily deteriorate or change in the environment in which it will be used These selections are affected by the NDE method for which the standard is intended Both ultrasonic and penetrant methods are very sensitive to the presence of foreign material inside the flaw For ultrasonics, this can affect the amount

of energy that is reflected from the flaw For penetrant inspection, the quantity of penetrant that can enter the defect, and consequently the brightness of the indication, will be reduced The foreign material may be fluids, soils, or corrosion

products For these reasons, magnesium, ferritic steels, and aluminum are particularly poor choices for reference standards for penetrant inspection and require some type of protection if used as ultrasonic standards

Eddy current methods are not affected by foreign material in the flaw, but are very sensitive to such surface conditions as scratches, pitting, and corrosion Magnetic particle methods are sensitive to the width of the defect and to anything, such

as cold working, that may change the magnetic permeability of the standard Radiographic methods are sensitive to the thickness of the standard and to changes in the shape of the flaw In general, a good choice of material for any method is one that is reasonably hard and forms an adherent, tough, and stable surface oxide layer (such as a titanium or nickel-base alloy), thus providing protection against mechanical damage and the gradual buildup of corrosion products

Personnel. Unless the inspection process is fully automated, the proficiency of the inspection personnel is the largest variable affecting inspection reliability This proficiency varies widely not only from inspector to inspector but also with the same inspector, depending on his working environment and his mental condition For fully automated inspections, the proficiency of the inspector in operating the equipment is important but has little or no impact on inspection reliability

The first task in addressing the contribution of the inspector to inspection reliability is to ensure that he is knowledgeable

of the specific techniques to be used and has the basic proficiency to perform the inspection to the required reliability Experience has demonstrated that the previous qualifications of the inspector for example, certification to MIL-STD-410D (Ref 1) are not sufficient to ensure the desired performance with a new inspection that must be performed with high reliability Therefore, the most straightforward way to assess proficiency is to require inspector participation in the demonstration of inspection reliability All inspectors that will be required to perform the inspection should also participate This not only establishes the reliability of the proposed inspection but also identifies those personnel requiring additional training or experience before they can be expected to perform adequately Careful observation of the inspector during the demonstration and of the results obtained is necessary to identify the additional training or experience needed After training and/or additional experience is acquired, the demonstration effort can be repeated to indicate if the inspector has become sufficiently proficient in the inspection technique

After basic inspection proficiency has been demonstrated, it becomes a supervisory task to ensure that this proficiency is maintained Control of the work environment of the inspector is important Distractions such as noise, extremes in temperature, and other irritants should be eliminated to the extent possible Break periods should be frequent enough to reduce fatigue Personnel who are ill or otherwise physically impaired should be temporarily assigned other tasks to the extent possible Other efforts that improve or maintain a good mental attitude are excellent ways to ensure sustained inspection reliability These include providing acceptable materials and equipment with which to conduct the inspection Finally, when it is not possible to provide a consistently conducive environment for a highly reliable inspection, two inspectors can perform the same inspection independently to achieve higher reliability than can be obtained with a single inspector Two inspectors generally will not make identical mistakes

Reference cited in this section

1 "Nondestructive Testing Personnel Qualification and Certification," MIL-STD-410D, 25 June 1974

Applications of NDE Reliability to Systems

Ward D Rummel, Martin Marietta Astronautics Group; Grover L Hardy and Thomas D Cooper, Wright Research & Development Center, Wright-Patterson Air Force Base

Applications (Case Studies)

Airframes. In the aerospace industry, the first application of quantitative NDE in a production facility was on airframe structures Loss of an F-111 aircraft by the propagation of an undetected manufacturing defect led to the incorporation of fracture control in the design and qualification of aircraft structure by the United States Air Force This was first applied rigorously to the B-1 aircraft and consisted of:

• The identification of critical structural components whose failure would cause the loss of aircraft

• The identification of those areas in the critical structural components experiencing the highest stresses defined

• The estimation of the maximum size of a rogue defect in these areas that could exist without growing to failure in twice the estimated design life

The maximum acceptable size of a rogue defect was established as 6.4 mm (0.250 in.) long by 3.2 mm (0.125 in.) deep for a surface flaw, or 1.3 × 1.3 mm (0.050 × 0.050 in.) for a corner crack in a bolt hole or a structural edge For an embedded flaw, the size was a 6.4 mm (0.250 in.) diam circle These sizes were selected because experience had shown that they were large enough to be readily detected during production inspection If the crack sizes that the designer selected for his structure were to be smaller, the capability of the production NDE facility to find the smaller cracks reliably would have to be demonstrated For the B-1, the crack sizes for many of the critical structural members was smaller than those suggested Redesign of the components to tolerate the larger, suggested flaws would have resulted in unacceptable weight penalties; consequently, the B-1 production NDE facility was subjected to the first quantitative NDE capability evaluation

A conservative approach was taken in the design of the NDE reliability demonstration program Fatigue cracks were selected to represent manufacturing defects for surface and corner flaws, and voids created by diffusion bonding were to serve as embedded defects Flat panels were the geometries selected, either single panels for surface defect areas or panel stackups for fastener hole areas, and rectangular blocks housed the embedded defects The materials were aluminum, titanium, and steel The surface finish for the specimens was the same as for production parts The methods to be demonstrated were penetrant inspection for all materials with surface flaws, magnetic particle for steel panels with surface flaws, eddy current for all materials with bolt holes, and ultrasonic for titanium and steel embedded flaws The surface and corner flaws were divided into four groups based on size (Table 1) Because of the difficulty of manufacturing embedded flaws, only two sizes were selected: 1.3 mm (0.050 in.) diam circle and a 1.3 × 2.5 mm (0.050 × 0.100 in.) ellipse

Table 1 Groups of manufactured flaws used to demonstrate the reliability of NDE methods in the B-1 program

Length range Depth range

Number of observations Misses

Table 2 Results of production NDE facilities in the B-1 program

Method Material Flaw type Flaw depth

Ultrasonic, longitudinal Titanium Embedded 1.17 0.046

Eddy current All Corner 0.75 0.030

Fracture control, with the resultant quantitative NDE requirement, has been incorporated into the design of every Air Force aircraft since the B-1 In addition, designs already in existence at the time have been analyzed to determine which inspections would be required in service to ensure attainment of the design life of a particular system Generally, where the suggested flaw sizes mentioned above could not be tolerated, the assumed design flaw for new designs has been a 0.75 mm (0.030 in.) corner crack and a surface flaw with a depth of 1.25 mm (0.050 in.) and a length of 2.5 mm (0.100 in.) The analysis of existing designs, however, has required the assumption of even smaller design flaws These instances

of having to detect smaller flaws in service have been few, and fortunately, identification of the requirement has occurred

in sufficient time to allow adequate NDE engineering to address the problems and explore various noninspection options

Gas Turbine Engines. Following the successful application of damage tolerance concepts and designs to Air Force aircraft structures, attention began to focus on aircraft gas turbine engines The reason is that the failure of a high-energy (rotating) component in an aircraft gas turbine engine usually results in catastrophic consequences for that engine Although most engines are designed to contain failure of the blades, the fracture of a disk or spacer will result in destruction of the engine and can cause significant damage to adjacent structures, such as fuel tanks, major structural members, or other engines Even the contained failure of a blade can cause immediate engine shutdown, which can also have catastrophic consequences for a high-performance, single-engine fighter Therefore, the integrity of many high-performance components in gas turbine engines is critical to aircraft safety

As part of an effort to increase the reliability and reduce the costs of operating and maintaining gas turbine engines for the U.S Air Force, a program known as the Engine Structural Integrity Program (ENSIP) has evolved over the last few years Its objective is to establish an approach to defining the structural performance, design, development, verification, and life management requirements for new engines for Air Force aircraft Military standard MIL-STD-1783 defines ENSIP and is currently being applied to the development of all new engines for the Air Force (Ref 2) The document is written in a generic format so that it can be tailored for use by specific System Program Offices to define an engine that will satisfy their own needs An accompanying handbook is attached as an appendix to provide specific guidance on the rationale, background criteria, lessons learned, and instructions necessary to tailor specific sections of the standard for application The technical approach is similar to the Aircraft Structural Integrity Program (ASIP), which is defined in MIL-STD-1530 and which has been successfully used for several years in the design of airframes for Air Force systems (Ref 3)

One of the most significant differences between ENSIP and the traditional approaches used in designing engine structures

is the requirement to apply damage tolerance and durability criteria to critical components This requires the designer to assume that flaws exist in the engine structure as manufactured and then to design the critical parts so that the flaws

cannot grow to the size that will cause failure in the lifetime of the part or at least within some predetermined inspection interval It also establishes life management requirements and procedures to ensure that the necessary inspections capable

of finding flaws in the size range used in design are conducted and that the engine parts are sufficiently durable so that the economic life of the engine is acceptable

The impact of this approach on the inspection community is very significant Based on the estimated capability of of-the-art inspection methods and procedures, many improvements have had to be made both in manufacturing and depot practice to satisfy the intent of ENSIP Furthermore, to allow the full implementation of this approach, the problem of defining available inspection capability in quantitative terms must continue to receive attention Acceptable procedures are being established so that it is possible to define exactly how sensitive the inspection methods are on a statistical basis

state-The first application of this technology by the Air Force was made to an already designed and operational aircraft gas turbine engine The F-100 engine, designed and built by Pratt & Whitney, was already widely used in the dual-engine F-

15 and the single-engine F-16 aircraft when, in 1978, a durability and damage tolerance assessment effort was initiated This engine was selected because it was to be purchased in large numbers for Air Force applications for many years to come and because it was a high-performance machine that was very demanding of materials and designs The costs of owning and operating this system could quickly become untenable if problems developed that significantly limited the life of critical parts or caused significant down-time for repairs Following the pattern that had been established for applying ASIP to airframes, a joint Air Force/Pratt & Whitney team was formed to work on-site at the facilities of the contractor to complete the analysis and to provide a viable Force Structural Maintenance Plan that could be implemented

at the San Antonio Air Logistics Center, where maintenance responsibility for the engines resided

After the team was in place and the necessary analytical studies were started, it was recognized early in the program that a quantitative understanding of the inspection procedures used in the manufacture of the engine was lacking and that a reliable definition of the largest flaw that could escape detection during manufacture had never been determined Qualitative statements were made expressing confidence in the inspection methods used, based on the good performance

of the engines to that time It was acknowledged, however, that the fleet was still young and that a quantitative definition

of the inspection process was urgently needed

To fill that need, a joint Air Force Materials Laboratory/Pratt & Whitney effort was established to prepare specimens with known flaws in selected size ranges that would allow quantitative determination of the capability of the inspection methods to be established The specimens were designed to contain flawed areas in geometrical features that simulated the real areas in the actual hardware These included holes, the edges of holes, radii, and flat surfaces Small flaws were generated by damaging the surface, initiating and growing a fatigue crack, and then removing the damaged area until only the desired depth of the flaw that remained was used to produce the specimens During 1979, some 39 sets of specimens with the desired geometries were fabricated from nickel- and titanium-base alloys for the program Target crack depths of 0.13, 0.25, and 0.50 mm (0.005, 0.010, and 0.020 in.) with a nominal 3:1 aspect ratio were prepared Similar specimens with no flaws were also included in each set The specimens were to be used not only to determine the capability of the manufacturing inspection methods but also to provide guidance concerning the establishment of inspection methods to be used in the depot during maintenance Fluorescent penetrant inspection and eddy current procedures were evaluated using these specimens The specimen sets were evaluated by both laboratory and production inspectors

Because this was the first documented attempt to fabricate flawed specimens of this complexity, a great deal of experience was gained, not only in specimen preparation but also in evaluating the effectiveness of the inspection methods Fluorescent penetrant inspection was included because, at the time, it was the most extensively applied inspection method used in both manufacturing and depot maintenance Eddy current methods were included because they had the best potential for finding the very small surface-connected flaws of concern Based on the work done in this program, the following conclusions were reached:

• The capability of fluorescent penetrant inspection to find small flaws with confidence was affected by many variables, including surface condition, nature of the flaw, the process used and the extent to which

it was controlled, and the skills and abilities of the inspectors The process did not have the necessary reliability to detect very small flaws in many of the critical areas that had been defined

• Eddy current methods appeared to have the best potential for detecting small flaws with the confidence level required

• Eddy current technology and procedures developed in the program could also be adequately automated

to make this method viable for use in the depot inspection environment They could also be adapted for

manufacturing

As a result of this study, the decision was made to implement semiautomated eddy current inspection methods at the San Antonio Air Logistics Center to provide reliable inspection of defined areas in selected critical parts A facility was established at the depot to allow the inspection of critical components as they were cycled through the depot The eddy current equipment designed by Pratt & Whitney and incorporated into the facility has been demonstrated to have the capability of finding 0.13 mm (0.005 in.) deep flaws with a reliability of 90% POD at 95% CL

The second application of damage tolerance analysis involved the TF-34 engine, which powers the A-10 ground-support attack aircraft and the S-3A antisubmarine aircraft In this case, an Air Force/General Electric team was formed at the General Electric Aircraft Engine Business Group facility in May 1981 Once again, a critical part of the assessment activity was to establish the level of quality built into the parts during the years the engines were manufactured This was needed to provide an indication of the largest flaw that could have been missed by the inspection methods being used at the time Because production of the engine was essentially completed by the time the assessment started, the only impact that establishing improved NDE procedure could have was on the methods being used at the maintenance and overhaul depot

In the case of the TF-34 engine, depot responsibility had been assigned to the Navy and was conducted at the Naval Air Newark Facility in Alameda, CA Once again, the assessment determined that to achieve acceptable inspection intervals for certain critical rotating components, inspection methods more sensitive and reliable than fluorescent penetrant inspection would be required A special clean room containing eddy current equipment was established at Alameda to allow inspection of the engine disks and spacers that were considered to be the most critical The equipment selected for this room was the Eddy Current II system developed under an Air Force Materials Laboratory manufacturing technology contract by the General Electric Company In addition, the fluorescent penetrant inspection facility at Alameda was upgraded to make it more reproducible and reliable Inspections have been conducted with these systems since 1984

Other applications of this technology by the U.S Air Force have been made to all new gas turbine engines now being used This includes the F-101 engine in the B-1B and the F-110 engine in the F-16 New engines now being designed for Air Force applications will also incorporate ENSIP technology and will therefore have quantitative inspection capabilities

as an integral part of their development Additional information on ENSIP is available in the article "Fracture Control Philosophy" in this Volume

Space Shuttle Program. Design requirements for the space shuttle program of the National Aeronautics and Space Administration included the use of fatigue and fracture mechanics principles in all systems designs The implementation

of fatigue and fracture mechanics requires knowledge of the quantitative performance capabilities of the materials,

components, and systems acceptance methods For this application, Rummel et al introduced the concepts of statistical

assessments of NDE process performance capabilities (Ref 4) Fatigue cracks in 2219-T87 aluminum alloy were selected

as the test specimens for the assessment of various NDE methods Test specimens were prepared by inducing fatigue cracks of varying size in aluminum alloy sheet specimens (fatigue crack growth from electrodischarge machined starter notches), by machining the specimen surfaces to produce a surface that was representative of production conditions, and

by passing the specimens through various NDE procedures and measuring the success in crack detection for the various procedures and operating conditions The concept of probability of detection as a function of flaw size was introduced, and the performance level of the various NDE procedures was quantified as summarized below

The experimental test sequence for data gathering, including nondestructive testing (NDT), is shown in Fig 5 The test specimens were given a precrack (starter notches) that enabled the growth of fatigue cracks of varying size and aspect ratio A total of 328 cracks were grown in 118 specimens, with flaw length ranging from 0.3 to 18 mm (0.012 to 0.700 in.) The specimens were subjected to x-ray radiographic, liquid penetrant, ultrasonic, and eddy current procedures in the as-machined, after-etch, and after-proof-test conditions The resulting POD curves for machined and etched surfaces are also shown in Fig 6

Fig 5 Experimental test analysis sequence

Fig 6 POD plots for four different NDE methods on the same set of specimens (a) Penetrant inspection (b)

Ultrasonic inspection (c) Eddy current inspection (d) X-ray inspection

The precision in crack sizing was also measured, and the results are shown in Fig 7 The composite threshold detection results were reduced to a plot, as shown in Fig 8 These data were further simplified to produce the design limits, as shown in Fig 9 Figure 9 contains design limits for standard NDE and special NDE The use of special NDE to meet difficult design constraints requires actual demonstration of the performance capabilities of the proposed NDE procedures and a system of controls to ensure that the performance conditions are maintained These design limits, set by actual NDE performance demonstration, were used as the basis for design and risk management for all space shuttle system components Assessments and NDE performance capability demonstrations have been continued with space shuttle contractors for all systems production and revalidation

Fig 7 Actual versus NDE estimated crack length for etched specimens (a) Penetrant inspection (b) X-ray

inspection

Fig 8 Combined threshold detection results

Fig 9 Plot of design limits

Special Inspection Systems. As a result of the evolution of damage tolerance requirements, special equipment has been developed in the last few years for increasing the reliability of inspection operations by automating the process and

by incorporating computerized data generation and control This has significantly reduced the dependence on human operators, thus eliminating one of the major sources of error in the process Most of these automation efforts have been directed toward the inspection of critical aircraft gas turbine engine hardware, primarily in the maintenance depot environment The systems could be used equally effectively in manufacturing operations The following are examples of the systems that have been developed over recent years

Structural Assessment Testing Applications. The NDE performance requirements for overhaul and revalidation

of the structural integrity of aircraft engine components (United States Air Force facilities) required the use of special, controlled methods to meet meantime between overhaul and life cycle performance requirements Special equipment, fixturing, procedures, and personnel training are implemented at the San Antonio Air Logistics Center at Kelly Air Force Base to approach the imposed design constraints Assessment, demonstration, and validation of the system performance capabilities and reliabilities were necessary to ensure that design requirements were being met

The special NDE methods were successful in meeting design requirements, and new levels of understanding and performance were gained by the implementation of advanced NDE methods and controls in this special facility The most significant output from the assessment of these processes was the demonstration of the need for excellence in NDE engineering in validating NDE procedures and in setting acceptance limits The performance capability of an eddy current method with an acceptance threshold limit set at 3.0 mV is shown in Fig 10(a) The performance level of the same procedure (same data) with an acceptance limit set at 0.5 mV is shown in Fig 10(b) This example clearly illustrates the importance of fully characterizing the NDE procedure and managing the NDE procedure within achievable acceptance limits

Fig 10 POD plotted (a) at a threshold of 3.0 mV and (b) at a threshold of 0.5 mV

Integrated Blade Inspection System. The quantitative assessment of process performance capabilities and process characterization is absolutely necessary in implementing automated NDE systems At a gas turbine overhaul facility, for example, quantitative assessment methods were applied to the implementation of an integrated blade inspection system with an automated fluorescent penetrant inspection module (Fig 11) The processed blades are introduced into a robotic

handling system that manipulates the blade in a high-gain optical-laser scan readout system to produce a digitized image

of the fluorescent penetrant indications A computerized data processing and image analysis system provides the readout and decision processing to accept or reject the blades

Fig 11 Flow diagram of an automated fluorescent penetrant inspection system

The optical performance capabilities of the readout system enable the detection of very small indications of varying brightness level The performance of the system is therefore limited by the fluorescent process capabilities and by the pattern recognition capabilities of the system Signal/noise distributions must be addressed by the decision discrimination process, as shown in Fig 12 Actual signal/noise distributions measured for the system are shown in Fig 13 If the discrimination threshold is set too low, a high false call rate will reduce the effectiveness of the automated process A balance of discrimination at an acceptable false call rate is used to achieve the figure of merit of performance level (Fig 14) for the system Signal/noise distribution data can be used to manage the overall process at any desired discrimination/false call level The process can be modeled to establish performance at varying discrimination levels, as shown in Fig 15 Quantification and characterization of process parameters are essential for the design, implementation, and management of automated NDE processes

Fig 12 Probability distribution of a noise signal and flaw indication

Fig 13 A sample of the signal/noise distribution for an integrated blade inspection system

Fig 14 POD curve for an automated fluorescent penetrant inspection system

Fig 15 Performance curves for various discrimination levels

Retirement-For-Cause (RFC) Inspection Equipment. As described previously, the concept of removing critical rotating components from gas turbine engines because of the initiation of an actual flaw rather than on the basis of an arbitrarily selected time period has already saved the Air Force significant sums of money One of the key technologies

that has allowed the RFC concept to be accepted has been the development of an accurate, repeatable, reliable inspection system capable of finding very small flaws in the parts The RFC system, developed under an Air Force Manufacturing Technology contract, has been in operation at the San Antonio Air Logistics Center at Kelly Air Force Base since late

1986 It was developed under a contract with Systems Research Laboratories, and although applied specifically to parts for the F-100 engine (used in the F-15 and F-16 aircraft), it was designed to be sufficiently flexible to inspect parts from any gas turbine engine This capability was achieved through the use of a team of subcontractors on the program that included Pratt & Whitney, General Electric, Garrett, and Allison

Eddy current methods for surface flaw detection and ultrasonic methods for detecting embedded flaws were the two inspection techniques selected for the RFC system Significant advances had been made in automating eddy current inspection through the development of the Eddy Current II system by the General Electric Company on earlier Air Force Manufacturing Technology contracts These stand-alone automated eddy current devices are in use at the Alameda Naval Air Rework Facility, as previously discussed; the Oklahoma City Air Logistics Center; the General Electric Manufacturing facility in Evandale, OH; and other commercial overhaul facilities The technology developed and proved

in these systems provided an excellent base for the evolution of the more complex RFC system

The RFC inspection system, which is housed in the special facility shown in Fig 16, consists of an operator console, a system computer, and eddy current and ultrasonic inspection stations The operator console is used to monitor the operational status of the system, to track inspection status at each NDE station, and to generate inspection data reports The system computer performs advanced data processing, system-wide communication, and sophisticated high-speed mathematical and scientific data analyses critical to the inspection process The NDE inspection stations perform the automated part inspections, flaw detection, and signal-preprocessing activities The system functions essentially independent of any human operator input, with the exception of loading and unloading the parts to be inspected onto the stations

Fig 16 Retirement-for-cause inspection facility

When the system was installed, a series of critical tests was conducted to establish its flaw size detection capability and reliability Tests included automatic scans of engine disks and a statistically significant number of representative fatigue-cracked test specimens Rivet hole inspection data showed a 90% POD with a 95% CL at the 100 m (4 mil) crack depth range Bolt hole and flat surface data indicated reliable detection in the desired 125 to 250 m (5 to 10 mil) depth range

A strong correlation between apparent versus actual flaw depth data was seen in all test data The ultrasonic inspection data were similarly encouraging Examples of POD at 95% CL generated for various geometrical configurations are shown in Fig 17 A similar facility is to be installed at the Air Force engine maintenance depot at the Oklahoma City Air Logistics Center at Tinker Air Force Base

Fig 17 RFC system POD curves for various geometrical features

References cited in this section

2 "Engine Structural Integrity Program (ENSIP)," MIL-STD-1783 (USAF), 30 Nov 1984

3 "Aircraft Structural Integrity Program, Airplane Requirements," MIL-STD-1530A (USAF), 11 Dec 1975

4 W.D Rummel, P.H Todd, Jr., S.A Frecska, and R.A Rathke, "The Detection of Fatigue Cracks by Nondestructive Testing Methods," CR-2369, National Aeronautics and Space Administration, Feb 1974

Applications of NDE Reliability to Systems

Ward D Rummel, Martin Marietta Astronautics Group; Grover L Hardy and Thomas D Cooper, Wright Research & Development Center, Wright-Patterson Air Force Base

References

1 "Nondestructive Testing Personnel Qualification and Certification," MIL-STD-410D, 25 June 1974

2 "Engine Structural Integrity Program (ENSIP)," MIL-STD-1783 (USAF), 30 Nov 1984

3 "Aircraft Structural Integrity Program, Airplane Requirements," MIL-STD-1530A (USAF), 11 Dec 1975

4 W.D Rummel, P.H Todd, Jr., S.A Frecska, and R.A Rathke, "The Detection of Fatigue Cracks by Nondestructive Testing Methods," CR-2369, National Aeronautics and Space Administration, Feb 1974

NDE Reliability Data Analysis

Alan P Berens, University of Dayton Research Institute

Introduction

INSPECTION SYSTEMS are inevitably driven to their extreme capability for finding small flaws When applied at this extreme, not all flaws of the same size will be detected In fact, repeat inspections of the same flaw will not necessarily produce consistent hit or miss indications, and different flaws of the same size may have different detection probabilities Because of this uncertainty in the inspection process, capability is characterized in terms of the probability of detection

(POD) as a function of flaw size, a At present, the function POD(a) can be estimated only through inspection reliability

experiments on specimens containing flaws of known size Further, statistical methods must be used to estimate the

parameters of the POD(a) function and to quantify the experimental error in the estimated capability

The methods for analyzing NDE reliability data have undergone a considerable evolution since the middle of the 1970s Formerly, a constant probability of detection of all flaws of a given size was postulated, and binomial distribution methods were used to estimate this probability and its lower confidence bound (Ref 1, 2) This nonparametric method of analysis produced valid statistical estimates for a single flaw size, but required very large sample sizes to obtain reasonable lower confidence bounds on the probability of detection In the absence of large numbers of representative specimens with equal flaw sizes, various methods were devised for analyzing data based on grouping schemes Although the resulting POD appeared more acceptable using these schemes, the lower confidence bounds were no longer valid

In recent years, an approach based on the assumption of a model for the POD(a) function was devised (Ref 3, 4, 5, 6, 7) Analyses of data from reliability experiments on nondestructive inspection (NDI) methods indicated that the POD(a)

function can be reasonably modeled by the cumulative log normal distribution function or, equivalently, the log-logistics (log odds) function The parameters of these functions can be estimated using maximum likelihood methods The statistical uncertainty in the estimate of NDI reliability has traditionally been reflected by a lower (conservative)

confidence bound on the POD(a) function The asymptotic statistical properties of the maximum likelihood estimates can

be used to calculate this confidence bound Details of the mathematics for these maximum likelihood calculations are presented in this article

References

1 B.G.W Yee, F.H Chang, J.C Couchman, G.H Lemon, and P.F Packman, "Assessment of NDE Reliability Data," NASA CR-134991, National Aeronautics and Space Administration, Oct 1976

2 W.D Rummel, Recommended Practice for Demonstration of Nondestructive Evaluation (NDE) Reliability

on Aircraft Production Parts, Mater Eng., Vol 40, Aug 1982, p 922-932

3 W.H Lewis, W.H Sproat, B.D Dodd, and J.M Hamilton, "Reliability of Nondestructive Inspections Final Report," SA-ALC/MME 76-6-38-1, San Antonio Air Logistics Center, Kelly Air Force Base, Dec 1978

4 A.P Berens and P.W Hovey, "Evaluation of NDE Reliability Characterization," AFWAL-TR-81-4160, Vol

1, Air Force Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, Dec 1981

5 A.P Berens and P.W Hovey, Statistical Methods for Estimating Crack Detection Probabilities, in

Probabilistic Fracture Mechanics and Fatigue Methods: Applications for Structural Design and Maintenance, STP 798, J.M Bloom and J.C Ekvall, Ed., American Society for Testing and Materials, 1983,

p 79-94

6 D.E Allison et al., "Cost/Risk Analysis for Disk Retirement Volume I," AFWAL-TR-83-4089, Air Force

Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, Feb 1984

7 A.P Berens and P.W Hovey, "Flaw Detection Reliability Criteria, Volume I Methods and Results," AFWAL-TR-84-4022, Air Force Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, April

1984

NDE Reliability Data Analysis

Alan P Berens, University of Dayton Research Institute

Statistical Nature of the NDE Process

In the application of an NDE method there are many factors that can influence whether or not the inspection will result in the correct decision as to the absence or presence of a flaw In general, NDE comprises the application of a stimulus to a structure and the interpretation of the response to the stimulus Repeated inspections of a specific flaw can produce different magnitudes of stimulus response because of minute variations in setup and calibration This variability is inherent in the process Different flaws of the same size can produce different response magnitudes because of differences

in material properties, flaw geometry, and flaw orientation The interpretation of the response can be influenced by the capability of the interpreter (manual or automated), the mental acuity of the inspector as influenced by fatigue or emotional outlook, and the ease of access and the environment at the inspection site All these factors contribute to inspection uncertainty and lead to a probabilistic characterization of inspection capability

There are two related approaches to a probabilistic framework for analyzing inspection reliability data Originally, inspection results were recorded only in terms of whether or not a flaw was found Data of this nature are called hit/miss data, and an analysis method for this data type evolved from the original binomial characterization (Ref 3, 4, 5) It was later observed that there is more information in the NDE signal response from which the hit/miss decision is made (Ref 6) Because the NDE signal response can be considered to be the perceived flaw size, data of this nature are called data

A second analysis method was developed based on data (Ref 7) Although the analysis frameworks are based on data of different natures, the hit/miss data can be obtained from data Both methods are based on the same model for the

POD(a) function, but different results will be obtained if the two analysis methods are applied to the same data set The following sections present the two approaches to formulating the POD(a) function

POD(a) From Hit/Miss Data. In typical NDE reliability studies, relatively few inspections are performed on each flaw in the specimen set Table 1 presents an example of hit/miss results from fluorescent penetrant inspections by 3 inspectors on 35 cracks in flat plate specimens Because there are only three inspections, it is impossible to obtain more than a general impression of any change in the chances of crack detection as the cracks get larger In a study for the Air Force (Ref 3), inspections of cracked specimens were independently conducted by large numbers of inspectors around the country The data from this program provided considerable insight into the nature of the probability of crack detection

Table 1 Example of summary data sheet of hit/miss results

The example is based on the fluorescent penetrant inspection of flat plates by three inspectors

Crack size Inspector(a) Crack identification

(a) 1 indicates crack was found; 0 indicates crack was not found

Figure 1 shows the results of eddy current inspections by 60 different inspections of a set of 41 cracks around countersunk fastener holes in a 1.5 in (5 ft) segment of a C-130 center wing box (Ref 3) Each data point represents the proportion of times that the crack was found This data set, which is representative of such available data sets, clearly indicates that:

• The chances of detection are correlated with crack size

• Different cracks of the same size can have significantly different crack detection probabilities

• Factors other than size are affecting the chances of detection

Data of this nature also provide the analysis framework for characterizing NDE reliability for the general hit/miss data set