Manual of pesticide residue analysis, volume 2

Derivation of the Limits of Detection and Determination Applying the Calibration Curve Concept 3 Mass-Spectrometric El Data for Confirmation of Results 25 Part 2: Cleanup Methods contd..

Manual of Pesticide Residue Analysis Volume II

VCH

VCH, P.O Box 101161, D-6940 Weinheim (Federal Republic of Germany)

Switzerland: VCH, P.O Box, CH-4020 Basel (Switzerland)

United Kingdom and Ireland: VCH (UK) Ltd., 8 Wellington Court, Cambridge CB1 1HZ (England)

USA and Canada: VCH, 220 East 23rd Street, New York NY 10010-4606 (USA)

DFG Deutsche Forschungsgemeinschaft

Manual of Pesticide

Residue Analysis

Volume II

Edited by Hans-Peter Thier

and Jochen Kirchhoff

Working Group "Analysis"

Pesticides Commission VCH

VCH Verlagsgesellschaft mbH, Weinheim (Federal Republic of Germany)

VCH Publishers Inc., New York, NY (USA)

Translators: J Edwards t and Carole Ann Traedgold

Library of Congress Card No applied for.

A catalogue record for this book is available from the British Library.

Deutsche Bibliothek Cataloguing-in-Publication Data:

Manual of pesticide residue analysis / DFG, Deutsche Forschungsgemeinschaft, Pesticides Commission.

Ed by HansPeter Thier and Jochen Kirchhoff [Transl.: J Edwards and Carole Ann Traedgold] Weinheim; Basel (Swizerland); Cambridge; New York, NY: VCH.

-NE: Thier, Hans-Peter [Hrsg.]; Deutsche Forschungsgemeinschaft / Kommission fur Pflanzenschutz-, Pflanzenbehandlungs- und Vorratsschutzmittel

Vol 2 (1992)

ISBN 3-527-27017-5 (Weinheim )

ISBN 0-89573 957-7 (New York)

© VCH Verlagsgesellschaft mbH, D-6940 Weinheim (Federal Republic of Germany), 1992

Printed on acid-free and chlorine-free paper.

All rights reserved (including those of translation into other languages) No part of this book may be reproduced in any form - by photoprinting, microfilm, or any other means - nor transmitted or translated into a machine language without written permission from the publishers.

Registered names, trademarks, etc used in this book, even when not specifically marked as such, are not

to be considered unprotected by law.

Composition: Filmsatz Unger & Sommer GmbH, D-6940 Weinheim Printing: betz-druck Gmbh, D-6100 Darmstadt 12

Printed in the Federal Republic of Germany

During more than two decades, the Working Group on Pesticide Residue Analysis of the

"Senatskommission fur Pflanzenschutz-, Pflanzenbehandlungs- und Vorratsschutzmittel"(Pesticides Commission), Deutsche Forschungsgemeinschaft (DFG), has edited a loose-leafManual of residue analytical methods

All the methods contained in this Manual were validated prior to their publication, by atleast one independent laboratory Therefore, the Manual has met with acceptance far beyondthe frontiers of the Federal Republic of Germany, particularly since many of the methods areincluded in the List of Recommended Methods of Analysis issued by the Codex Committee

on Pesticide Residues (CCPR) of the FAO/WHO Codex Alimentarius Commission Manyresidue analysts are, however, not well versed in German Therefore, to overcome this languagebarrier and to render the methods accessible to a far wider international circle of analysts,the Working Group decided to translate the most important sections of the Manual intoEnglish This mission was sponsored by the Deutsche Forschungsgemeinschaft

Volume 1 of the English edition was published in 1987 It contained 23 compound-specific("single") analytical methods selected from the 6th and 7th instalments (issued in 1982 and

1984, respectively) of the German edition, 17 multiresidue analytical methods and 6 cleanupmethods (both 1984 status) as well as all pertinent general sections, e g on the collection andpreparation of samples, on the limits of detection and determination, and on micro methodsand equipment for sample processing

The present Volume 2 of the English edition is a direct continuation and completion of thefirst volume It contains 32 single methods, many of them designed for the determination ofrecently developed compounds These methods were adopted, in most cases, from the 8th to11th instalments of the German edition issued between 1985 and 1991 Furthermore, Volume 2contains five new multiresidue analytical methods (coded S) published in German since thefirst volume went to press, and some tables providing supplementary data on the broad ap-plicability of Methods S 8 and S 19 and Cleanup Method 6, both described in Volume 1.Special features of Volume 2 are Part 5, presenting six multiple methods for analysis ofresidues in water (coded W), and Part 6 on analytical methods for determining residues inwater using the Automated Multiple Development (AMD) technique Moreover, two newcleanup methods for the solid-phase extraction of water samples on alkyl-modified silica gelare included An additional chapter introduces a new concept for deriving the limits of detec-tion and determination by the calibration curve technique, thus providing a commendablealternative to the procedure proposed in Volume 1 Finally, a comprehensive table gives mass-spectrometric El data for confirmation of gas-chromatographic results In some cases, theEditorial Committee has also partly changed or updated the original German version in order

to better adjust it to the needs of today's methodology A cumulative index for Volumes 1 and

2 provides easy access to all pertinent compounds and information

The Working Group on Pesticide Residue Analysis had hoped that it would render a majorcontribution to pesticide residue analytical methodology by carrying on the German andEnglish editions However, the Working Group had to terminate its activities in 1989, after

so many years of engagement in matters of pesticide residue analysis, because the Senate of

by Carole Ann Traedgold and edited by the Committee.

The Editorial Committee hopes that this two-volume compilation of procedures andmethods will prove useful to all concerned with the analysis of pesticide residues

Contents of Volume 1 IX Senate Commission for Pesticides, Deutsche Forschungsgemeinschaft XI Working Group on Residue Analysis, Senate Commission for Pesticides XV

Part 1: Introduction and Instructions (contd.)

Derivation of the Limits of Detection and Determination Applying the Calibration Curve Concept 3 Mass-Spectrometric El Data for Confirmation of Results 25

Part 2: Cleanup Methods (contd.)

Cleanup Method 6 Cleanup of crude extracts from plant and animal material by gel

permeation chromatography on a polystyrene gel in an automated apparatus (updated) 31 Cleanup Method 7 Solid phase extraction of water samples on alkyl-modified silica gel

using disposable columns 37 Cleanup Method 8 Solid phase extraction of water samples on alkyl-modified silica gel 41

Part 3: Individual Pesticide Residue Analytical Methods (contd.)

Amitrole, 4-A*) 49 Anilazine, 186 59 Benomyl, Carbendazim, Thiophanate-methyl, 261-378-370 69 Bitertanol, 613-A 77 Bitertanol, Triadimefon, Triadimenol, 613-425-605 87 Bromoxynil, Ioxynil, 264-212 99 Carbendazim, 378 107 Carbosulfan, Carbofuran, 658-344 113 Chlorflurenol, Flurenol, 275-215 127 Chloridazon, 89-A 135 Chlorsulfuron, Metsulfuron, 664-672 145 Copper Oxychloride, 147-A 153 Cymoxanil, 513 157 2,4-D, Dichlorprop, 27-A-38-A 163 Dichlobenil, 225-A 169 Dichlofluanid, Tolylfluanid, 203-371 177 Dichlofluanid, Tolylfluanid, 203-A-371-A 191 Dinobuton, Binapacryl, 255-8 197 Fonofos, 288 205 Fosetyl, 522 211

*) Code numbers according to which the analytical methods are identified in the German issue of the Manual The number without affixed letter corresponds to the BBA registration number of the in- dividual compound.

VIII Contents

Glufosinate, 651 217 Glyphosate, 405 229 Metaldehyde, 151-A 239 Metribuzin, 337 245 Nitrothal-isopropyl, 416 253 Oxamyl, 441 261 Phenmedipham, 233-B 269 Propachlor, 310 275 Propiconazole, 624 281 Sulphur, 184-B 287 Thiabendazole, 256-A 291 Thiabendazole, 256-B 295

Part 4: Multiple Pesticide Residue Analytical Methods (contd.)

Pesticides, Chemically Related Compounds and Metabolites Determinable by the Multiresidue Methods in Parts 4 to 6: Supplement to the Table of Compounds,

pp 221 ff, Vol 1 301

S 8 Organohalogen, Organophosphorus and Triazine Compounds (updated) 313

S 19 Organochlorine, Organophosphorus, Nitrogen-Containing and Other Pesticides (updated) 317

S 22 Natural Pyrethrins, Piperonyl Butoxide 323

S 23 Pyrethroids 333

S 24 Organotin Compounds 343

S 25 Methyl Carbamate Insecticides 349

S 26 Phthalimides 359

Part 5: Multiple Pesticide Residue Analytical Methods for Water

W 4 Phenoxyalkanoic Acid Herbicides 369

W 5 Fungicides 377

W 6 Organochlorine Insecticides 387

W 7 Phenoxyalkanoic Acid Herbicides 393

W 8 Triazine Herbicides 403

W 13 Desalkyl Metabolites of Chlorotriazine Herbicides 413

Part 6: Pesticide Residue Analytical Methods for Water Using the AMD Technique

Thin-Layer Chromatographic Analysis of Pesticides and Metabolites Using the Automated Multiple Development (AMD) Technique 423 Examples for Applying the AMD Technique to the Determination of Pesticide Residues

in Ground and Drinking Waters 435

Cumulative Indexes for Volumes 1 and 2

Index of Determinable Pesticides, Metabolites and Related Compounds (Index of pounds) 449 Index of Analytical Materials 459 List of Suppliers Referenced in the Text-Matter of the Manual 479 Author Index 483

Com-Contents of Volume 1

Senate Commission for Pesticides, Deutsche Forschungsgemeinschaft

Members and Guests of the Working Group on Residue Analysis, Senate Commission forPesticides

Part 1: Introduction and Instructions

Explanations

Notes on Types and Uses of Methods

Important Notes on the Use of Reagents

Abbreviations

Preparation of Samples

Collection and Preparation of Soil Samples

Collection and Preparation of Water Samples

Use of the Term "Water"

Micro Methods and Equipment for Sample Processing

Limits of Detection and Determination

Reporting of Analytical Results

Use of Forms in the Reporting of Analytical Results

Part 2: Cleanup Methods

Cleanup Method 1 Separation of organochlorine insecticides from hexachlorobenzene and

Cleanup Method 4 Cleanup of crude extracts from plant material by gel permeation

chro-matography on polystyrene gels

Cleanup Method 5 Cleanup of large quantities of fats for analysis of residues of

organo-chlorine and organophosphorus compounds

Cleanup Method 6 Cleanup of crude extracts from plant and animal material by gel

permea-tion chromatography on a polystyrene gel in an automated apparatus

Part 3: Individual Pesticide Residue Analytical Methods

Part 4: Multiple Pesticide Residue Analytical Methods

Pesticides, Chemically Related Compounds and Metabolites Determinable by the Multiresidue Methods (Table of Compounds)

S 6 Substituted Phenyl Urea Herbicides

S 6-A Substituted Phenyl Urea Herbicides

S 7 Triazine Herbicides

S 8 Organohalogen, Organophosphorus and Triazine Compounds

S 9 Organochlorine and Organophosphorus Pesticides

S 10 Organochlorine and Organophosphorus Pesticides

S 11 Potato Sprout Suppressants Propham and Chlorpropham

S 12 Organochlorine Pesticides

S 13 Organophosphorus Insecticides

S 14 Triazine Herbicides and Desalkyl Metabolites

S 15 Dithiocarbamate and Thiuram Disulphide Fungicides

S 16 Organophosphorus Pesticides with Thioether Groups

S 17 Organophosphorus Insecticides

S 18 Bromine-Containing Fumigants

S 19 Organochlorine, Organophosphorus, Nitrogen-Containing and Other Pesticides

S 20 Phthalimide Fungicides (Captafol, Captan, Folpet)

S 21 Ethylene and Propylene Bisdithiocarbamate Fungicides

Indexes

Index of Determinable Pesticides, Metabolites and Related Compounds (Index of pounds)

Com-Index of Analytical Materials

List of Suppliers Referenced in the Text-Matter of the Manual

Author Index

Senate Commission for Pesticides,

Dr.-Ing Siegbert Gorbach

Prof Dr Friedrich GroBmann

Prof Dr Hans-Jurgen Hapke

D-6501 SchwabenheimBayer AG, PF-A/CE-RA,Pflanzenschutzzentrum MonheimD-5090 Leverkusen-BayerwerkHoechst AG, Analytisches Laboratorium,Pflanzenschutz-Analyse, G 864

Postfach 800320, D-6230 Frankfurt 80Institut ftir Phytomedizin der Universitat HohenheimOtto-Sander-Stral3e 5, D-7000 Stuttgart 70

Institut ftir Pharmakologie der Tierarztlichen HochschuleBischofsholer Darnm 15, D-3000 Hannover 1

Asta Pharma AGWeismtillerstraBe 45, D-6230 Frankfurt 1Bayer AG, PF-A/CE-Okobiologie,Pflanzenschutzzentrum MonheimD-5090 Leverkusen-BayerwerkDivision Agrochemie der CIBA-GEIGY AGCH-4002 Basel/Schweiz

Lorscher StraBe 10, D-6700 LudwigshafenHoechst AG, Toxikologie-GewerbetoxikologiePostfach 800320, D-6230 Frankfurt 80Bayer AG, Institut ftir ToxikologieFriedrich-Ebert-StraBe 217, D-5600 Wuppertal 1Institut ftir Phytomedizin der Universitat HohenheimOtto-Sander-StraBe 5, D-7000 Stuttgart 70

XII Senate Commission for Pesticides

Prof Dr Fred Klingauf

Dr Claus Klotzsche

Prof Dr Werner Koch

Prof Dr Ulrich Mohr

Prof Dr Friedrich-Karl

Ohnesorge

Prof Dr Christian Schlatter

Prof Dr Heinz Schmutterer

Prof Dr Fritz Schonbeck

Prof Dr Fidelis Selenka

Prof Dr Hans-Peter Thier

Dr Ludwig Weil

Prof Dr Heinrich Carl Weltzien

Biologische Bundesanstalt fur Land- und Forstwirtschaft Messeweg 11-12, D-33OO Braunschweig

Bruelweg 36, CH-4147 Aesch/Schweiz Institut fur Pflanzenproduktion in den Tropen und Subtropen der Universitat Hohenheim

Kirchnerstrafle 5, D-7000 Stuttgart 70 Abteilung fur experimentelle Pathologie der Med Hochschule

Konstanty-Gutschow-Strafle 8, D-3000 Hannover 61 Institut fur Toxikologie der Universitat

Moorenstrafk 5, D-4000 Dusseldorf 1 Institut fur Toxikologie der ETH und Universitat Zurich Schorenstrafle 16, CH-8603 Schwerzenbach/Schweiz Institut fiir Phytopathologie und

angewandte Entomologie der Universitat Ludwigstrafk 23, D-6300 Gieften Institut fiir Pflanzenkrankheiten und Pflanzenschutz der Universitat

Herrenhauser Strafie 2, D-3000 Hannover 21 Institut fiir Hygiene der Ruhr-Universitat Postfach 102148, D-4630 Bochum Institut fiir Lebensmittelchemie der Universitat Piusallee 7, D-4400 Miinster

Institut fiir Wasserchemie und Chemische Balneologie der Technischen Universitat

Marchioninistrafie 17, D-8000 Miinchen 70 Institut fiir Pflanzenkrankheiten der Universitat Nuflallee 9, D-5300 Bonn 1

Permanent Guests

Prof Dr Fritz Herzel Bundesgesundheitsamt

Postfach 330013, D-1000 Berlin 33 Prof Dr Alfred-G Hildebrandt Institut fiir Arzneimittel des Bundesgesundheitsamtes

Postfach 330013, D-1000 Berlin 33

Senate Commission for Pesticides XIII

Secretaries of the Senate Commission for Pesticides

Frau Dr Dagmar Weil Institut fur Wasserchemie und Chemische Balneologie until 1986 der Technischen Universitat

Marchioninistrafie 17, D-8000 Munchen 70

Dr Friedhelm Dopke Institut fur Pflanzenpathologie und Pflanzenschutz from 1987 to 1989 der Universitat

Grisebachstr 6, D-3400 Gottingen-Weende Assessor Wolfgang Deutsche Forschungsgemeinschaft

Bretschneider t 1990 Kennedyallee 40, D-5300 Bonn 2

Working Group on Residue Analysis,

Senate Commission for Pesticides

Members and Guests

Dr Ing Siegbert Gorbach

Prof Dr Fritz Herzel

Landesuntersuchungsamt fur das Gesundheitswesen,Fachabteilung Chemie

Fritz-Hintermayr-Strafle 3, D-8900 AugsburgShell Forschung GmbH

D-6501 SchwabenheimBayer AG, PF-A/CE-RA,Pflanzenschutzzentrum MonheimD-5090 Leverkusen-BayerwerkHoechst AG, Analytisches Laboratorium,Pflanzenschutz-Analyse, G 864

Postfach 800320, D-6230 Frankfurt 80Bundesgesundheitsamt

Postfach 330013, D-1000 Berlin 33Division Agrochemie der CIBA-GEIGY AGCH-4002 Basel/Schweiz

Fachbereich Chemie u Chemietechnik der UniversitatPostfach 1621, D-4790 Paderborn

Institut fur Phytomedizin der Universitat HohenheimOtto-Sander-StraBe 5, D-7000 Stuttgart 70

Prof Dr Hans Maier-Bode Tannenweg 7, D-7884 Rickenbach b Sackingen

XVI Working Group on Residue Analysis

Dr Egon Mollhoff Bayer AG, PF-A/CE-RA,

Pflanzenschutzzentrum MonheimD-5090 Leverkusen-Bayerwerk

Dr Ludwig Weil Institut fur Wasserchemie und Chemische Balneologie

der Technischen UniversitatMarchioninistrafie 17, D-8000 Miinchen 70

Editorial Committee

Prof Dr Hans Zeumer t

(Former Chairman, until 1986)

Parti Introduction and Instructions

Derivation of the Limits of Detection and

Determination Applying the Calibration Curve Concept

(German version published 1991)

1 Introduction

It is a familiar experience in trace analysis that analytical results can become uncertain or even entirely unreliable if the substance to be analyzed (the analyte) is present in very low concen- trations This can be due to various causes which can also occur simultaneously, e g.:

— Co-extractives from the matrix simulate the analyte, thus leading to blank values.

— The analyte is lost during the cleanup in varying proportions, so that the results from allel analyses vary to an unacceptable extent.

par-— The minute amounts of the analyte are not, or are only inadequately substantiated by the measuring system.

Consequently there are three categories in which an analytical result can fall:

A The presence of the analyte is shown; a quantitative determination is possible.

B The presence of the analyte can indeed still be shown, but a reliable quantitative mination is no longer possible.

deter-C The presence of the analyte can no longer be established with sufficient probability; the analyte must, therefore, be considered as "not detectable".

Categories A and B are separated by the limit of determination (LDM), categories B and

C by the limit of detection (LDC).

For these reasons, a convention needs to be established on how to define LDM and LDC *) Both can be used in different respects:

1 By specifying LDM and/or LDC, the author of an analytical method can give other analysts using the method an indication as to its performance.

2 The analyst can more accurately characterize his findings with the aid of LDM and/or LDC, e.g by presenting a result (in case B only!) as "Content of compound X in the sample < [LDM]", or in case C, as "Compound X not detectable in the sample, LDC = [LDC]" The letters in brackets denote the numerical value for either LDM or LDC; see also p 45, Vol 1.

*) In the absence of a defined limit of determination, it may be expedient for the analyst to use the routine limit of determination (RLDM; see p 43, Vol 1) as the reporting level, if the analytical problem per- mits such an approach In this case, however, the analyst must clearly state that the RLDM was used

as the threshold when reporting the result of an analysis as " < ", thus indicating that quantitation below this level was not attempted and there is no evidence whether or not the analyte is determinable when present in concentrations smaller than the RLDM.

4 Limits of Detection and Determination

Numerical values for LDM and LDC are valid only for each special case of the analyst'sinstrumental and operating conditions "Generally valid" statements such as "The methodhas a LDM (LDC) of " are, therefore, not appropriate

The pertinent literature contains numerous recommendations for a mathematical definition

of the LDC and/or LDM Nevertheless, often even the nomenclature is not uniform quently LDC and LDM are used as synonyms; additionally there are other, and sometimeseven incorrect terms in use, e.g "sensitivity"

Fre-Older recommendations, in part still used today, relate the LDC or LDM to the blank value

or instrument noise and their random scatter (standard deviation) The measured signal maythen be considered to be significant if its mean value differs from the mean of the blank ornoise by a given multiple of the standard deviation This kind of evaluation, however, is onlyjustified if the errors inherent in the measurement procedure are caused exclusively by the in-strumental conditions, e.g with photometric measurements after a wet ashing, or withestablishing a calibration curve from standard solutions in gas chromatography It is,therefore, not comprehensive enough for application in residue analysis

The results of residue analyses are decisively affected by primary factors from the precedingextraction and cleanup steps, such as variable "recoveries" For use in this Manual, therefore,the derivation of the LDC and, from it, of the LDM, is based upon results obtained from com-plete analytical procedures Moreover, the additional Requirements II and III were introducedfor the definition of the LDM The LDM is defined as the smallest value for the content of

an analyte in an analytical sample that satisfies the three following requirements:

I The LDM is greater than, and significantly different from the LDC

II The recovery (sensitivity) at the LDM is equal to, or greater than 70%

Ill The coefficient of variation at the LDM, from replicate determinations, is equal to, orsmaller than 0.2 (equivalent to 20%)

The recommendation given in the Section on Limits of Detection and Determination on

pp 37 ff, Vol 1, still required the existence of blank values for estimating LDC and LDM.However, it also demanded the requirement of the recoveries exceeding 70% to be checked (II)

by stepwise fortification and calculation of the regression line In addition, the smallest tification level had to meet Requirement III

for-With the progress of analytical techniques, however, blank values often do not show anymore, or are not significant for the interpretation of an analytical result In order to enable

a convention on the definition of the LDC and/or LDM in these cases, the calibration curveconcept (also familiar from many publications) is proposed here for application in residueanalysis A special advantage of this concept is that the LDC can be determined with actuallymeasured values, and that neither authentic control samples nor blank values are required.This concept will be presented and mathematically sustained For its routine applicationfrom a series of measurements, the use of a suitably programmed computer is recommended

In individual cases, both limits can be derived graphically, with relatively little calculationeffort and with sufficient accuracy, from a plot of the calibration curve and its prediction in-terval For an example, see 9.3

Limits of Detection and Determination 5

2 Calibration curve concept

2.1 Basic considerations

The aim of the concept is to define the limit of detection and, resulting from it, the limit ofdetermination for the results obtained when a given analyte is determined with a particularanalytical method in an individual laboratory The definition proceeds from the calibrationcurve obtained with the analytical method and employs the upper and lower limits of theprediction interval of the curve for deriving LDC and LDM The prediction interval is usedhere as the confidence interval

The operation described for determining LDM permits an appropriate consideration of quirement I Requirement III is integrated into the formulae used to calculate LDM Deter-mining the slope of the calibration curve will check Requirement II

Re-2.2 Establishing the calibration curve

To obtain the calibration curve, a series of fortification experiments is run with k given levels

for which the corresponding signal values

are measured, with m, replicate experiments per level Xt The number of replicate

ex-it

periments may be different on each fortification level X { In total, n = £ m^ value pairs

The given levels (X) of the analyte in the samples and the corresponding signal values (Y)are connected by the method-specific calibration function, which will be linear — at leastlocally — in good approximation In this case, only few value pairs for X and Y from fortifica-tion experiments are required (see 3.3.1)

From the total of the n value pairs obtained, the calibration curve is established It isrepresented by the regression line which is calculated according to the least squares method.The function equation of the regression line is

Y = A + B X

where

Y = measured signal value for the content X

X = content of the analyte in the sample

A = intercept on the signal axis at the point X = 0

B = slope of the regression line (sensitivity of the method)

The prerequisite to deriving LDC and LDM is a certain minimum value for the slope B ofthe calibration curve (see 4.3) Fortification experiments which do not meet this requirementare useless and must be repeated under improved and appropriate experimental conditions

6 Limits of Detection and Determination

Next, the prediction interval is calculated which symmetrically envelopes the linear tion curve (see Figure 1) The curves for the upper (Y + ) and lower (Y_) limits of the predic- tion interval define that interval in which future ("predicted") signal values for any content

calibra-X are to be expected at a selected level of statistical significance.

Fig 1 Calibration curve with upper and lower limits of the prediction interval Intersection of the line

Y = Y o with the prediction interval and the calibration curve; intersection points = X l5 X 3 , X 2

A = theoretical blank value.

The points (Y UP , YLO), where both curves Y + and Y_ intersect with the signal axis, dicate the confidence interval for signal values yielded by samples with a "nil" content Note that this range is extrapolated from the results of the fortification experiments and is not derived from the measurement of blank values.

in-Each signal value Y o > Y UP yields three possible points of intersection with the calibration curve and the limits of its prediction interval They correspond, respectively, to the values

X j , X 3 and X 2 (Figure 1) and form the basis for further derivations When the curvature of both the upper and lower limits of the prediction interval is negligible, X 3 can be considered

the arithmetic mean of X { and X 2 with good approximation X 2 is corresponding to LDC* (Figure 4) and X£ v (Figure 5).

Xj and X 2 can be calculated from the formulae IIa and l i b (see 6.6, see also 3.2).

Limits of Detection and Determination

3 Limit of detection (LDC)

3.1 Definition

The limit of detection is defined by the smallest content of the analyte in an analytical sample, for which the particular analytical method yields signal values which differ, with a selected level of significance a, from signal values obtained from samples with a "nil" content (blank signal values).

The level of significance, a, can be arbitrarily chosen In most cases, values of a between

5 and 1% will allow a sufficient margin of safety In residue analyses, usually a level of

a = 5%, i.e a confidence level of S = 1 - a = 95%, is chosen.

Based on the n results (X^Yj) from the fortification experiments and the given significance level, a, the limit of detection for an analyte is derived from the calibration curve.

It is represented by the smallest value X L D C for which the confidence intervals of the responding signal value Y LDC and of the signal value for a "nil" content do not overlap (Figure 2).

8 Limits of Detection and Determination

— When the measured signal value Y is greater than YLDC, the analytical sample is assigned

a content of X = (Y - A)/B (see X3 in Figure 1) The confidence interval belonging to X

is equivalent to the range from X{ to X2 in Figure 1.

At the limit of detection, the probability for the false proof of detecting the analyte which

in reality is not present in the sample (error of the first kind) is just equivalent to the selected

a of 5% This is illustrated in Figure 2: The distribution curves for the signal values A and

YLDC each overlap by 2.5 area percent, corresponding to a test with a = 5% (two-sided).The error of the second kind (false negative result in spite of a real content of the analyte beingpresent in the sample) depends on the actual content present For X > LDC (corresponding to

Y > YLDC), it is smaller than 50%, for X = LDC, it is exactly 50% (Figure 3), i e a signal value

Y > YLDC will be caused, with a probability greater than 50%, by an analyte content >0

Fig 3 Confidence interval of a result X at the point LDC with distribution curve (schematic illustration).

3.3 Determining the limit of detection

3.3.1 Establishing the calibration curve

For the fortification experiments, it is advantageous to use authentic control samples, ifavailable, but other comparable material can also be used provided it does not contain anysubstances that would interfere with the analysis

The fortification levels extend from the anticipated LDC into the expected working range.Note that the prediction interval which envelopes the calibration curve is narrowest at thepoint X Therefore, by suitable experimental planning and by making use of experienceavailable, a higher degree of precision can be reached through choosing fortification levels inthe neighbourhood of the anticipated LDC

Limits of Detection and Determination 9

For best reliability of LDC, more than 4 evenly spaced fortification levels should be used(k > 4), each with several replicate derminations (up to m = 4) However, for economicalreasons it will often not be possible for this statistically required number of measurements to

be carried out A substantial reason for this is certainly the fact that for a given analyte,depending on the sample material, different values for LDC can result, so that an accordinglygreat number of fortification experiments would be required

In general, it will be sufficient to choose 4-5 different fortification levels if the experimentsare repeated at least once on each level Note that it is beneficial to use fewer replicates on

a greater number of levels, rather than to carry out more repetitions on fewer levels Thenumber of replicates may, however, be different on the individual levels Moreover, increasingthe number of fortification levels can, if need be, render it feasible to check the adequacy ofthe model assumption, e.g linearity

Although to the disadvantage of statistical precision, in practice it may often be unavoidable

to derive the LDC from only one single measurement per fortification level In this case,however, a minimum of 6-8 measured values is required

Measured values are only valid for the calculations if they represent the results of completeanalyses It would be malpractice, for example, to split an extract obtained from a sample intohalves and to analyze these two portions separately in order to get "two" measured values.The performance of the measurement set-up must be thoroughly checked before the for-tification experiments are undertaken Only such instrumentation which is in good condition,complies to the standards, and produces sensitive and reproducible signal values, will besuitable for establishing the calibration curve Note that the instruments often produce quitedifferent signal values for the same amount of the analyte if the analyses are not carried outconsecutively The signals must not be evaluated if they exhibit a drift, or if their qualitydeclines due to other reasons Moreover, the signal values must not be adversely affected byco-extractives from the sample material

Using a suitable programmed computer, the parameters of the linear calibration curve andthe two limits of the prediction interval can easily be calculated from the individual value pairs

X { ,Y { The curves are best drawn by a plotter For illustration, Figure 6 shows a graph and

print-out generated by computer, using Example 1 (cf 9.1) In Section 9.3, a description isgiven on how to proceed without the aid of a computer

3.3.2 Graphical derivation of LDC

In the graph obtained (Figure 2; cf Figure 7), draw a straight line, parallel with the abscissa,from the point of intersection, YUP, to the point where it intersects the lower limit of theprediction interval This point corresponds, on the abscissa, to the value of LDC, the limit

of detection (cf 9.3) For computer calculation of LDC, see 6.6

4 Limit of determination (LDM)

4.1 Definition

Residue analyses are frequently performed to monitor foodstuff for compliance withestablished maximum residue limits For this reason, both risks, namely erroneously to statethe content of a sample either as conforming to, or exceeding the maximum residue limit, must

10 Limits of Detection and Determination

be kept to a minimum This can only be achieved when the coefficients of variation fromreplicate determinations are small and systematic errors can be excluded

It is also for these reasons that the limit of determination must fulfill particularrequirements, especially when the maximum residue limits to be enforced were set at or aboutthis limit

For the purpose of residue analyses, therefore, the LDM is defined as the smallest content

of the analyte satisfying the Requirements I, II and III given in the Introduction

4.2 Consequences of Requirement I

According to Requirement I, the LDM should be greater than, and significantly different fromthe LDC This condition is met when the confidence interval (a = 5%) of a concentration deter-mined at X = LDM does not extend into the range below the LDC If Xj denotes that concen-tration whose confidence interval just borders the LDC, the LDM cannot be smaller than Xj

Xj can be determined graphically in a simple manner from the plot of the calibration curve(Figure 4; cf Figure 8): First, the point of intersection, Yc, of the vertical line X = LDC with

the upper limit of the prediction interval is determined by drawing a parallel to the Y (signal)

axis through the point X = LDC; cf 9.3 (for formulae to calculate Yc, see 6.7) Next, Xj isobtained as the X value of the intersection of the horizontal line Y = Yc with the calibrationline: X! = (Yc - A)/B; see 6.8

The value of Xx can be taken as LDM if it can be shown that the Requirements II and III are

fulfilled at this concentration as well However, a value for LDM being greater than X! may be

exacted through Requirement III in many cases (see 4.4)

Limits of Detection and Determination 11

4.3 Consequences of Requirement II

The usual way to obtain a calibration curve is to plot each signal value (Y) versus the

cor-responding given content (X) of the analyte in the sample material For checking ment II, however, a different ordinate scaling is used for plotting the line

Require-For this purpose, standard solutions are used in order to determine which amount of analyteresults in which signal value (calibration curve for the standard solutions) Next, the individualsignal values obtained from the fortification experiments are converted into the correspondingconcentrations of the analyte

For the given fortification levels (X) and the corresponding measured concentrations thus

obtained (Y), the parameters of the regression line are calculated according to the regressionequation given in 2.2 If the slope B of the regression line is B = 1, the recovery is 100% Re-quirement II asks for a recovery of > 70% which means that the slope must be B > 0.7

4.4 Consequences of Requirement III

According to Requirement III, the coefficient of variation from replicate determinations at thelimit of determination should be equal to, or less than 0.2 (equivalent to 20%)

One way of checking whether this requirement is met could be to calculate the coefficient

of variation for each fortification level individually This can, of course, only be done if severalrepeat measurements (at least m = 4) for each level were made Then, the LDM is given bythe smallest fortification level at which Requirement III is satisfied, provided that Re-quirements I and II are met as well This approach corresponds to the procedure recom-mended on p 40, Vol 1

In practice, however, there may be a need to derive the LDC and/or LDM from only onemeasurement (anyway from m < 4) each per fortification level (see 3.3.1) In such cases, for-tunately, the calibration curve concept offers an elegant possibility to obtain the required in-formation in a different manner

Each future determination of a signal value at a definite concentration can be assigned acoefficient of variation V which is given by

Y(X)

where

SY = standard deviation of a future determination of the signal value Y (X) at

the point X

Y (X) = signal value on the calibration curve at the point X

According to Requirement III, therefore, a position X must be found from which onward

V becomes smaller than the required value Vo = 0.2, presuming that V decreases with ing values of X This position X is given by the intersection of the line

increas-Y = (A + B • X) • (1 + t • Vo)

with the upper limit of the prediction interval (for explanation, see 6.10) Xc v is the X dinate of the intersection point (see Figure 5; cf Figure 9)

coor-12 Limits of Detection and Determination

Fig 5 Limit of determination (LDM) Condition: LDM > X m , with X in = (Y cv - A)/B.

To determine this line, calculate for two points X' and X" the corresponding values Y' andY" on the calibration curve, using the equation Y = A + B • X Both Y' and Y" are eachmultiplied by the factor (1 + 0.2 • t), where t is the factor t from the formula for the predictioninterval, see 6.3 and Table 2 The two points obtained are connected by a straight line

In analogy to 4.2, Ycv is the Y value corresponding to the intersection point Xc v Next,

Xm is defined as the X value of the intersection point of the line Y = Ycv (parallel to theabscissa) with the calibration curve (cf 9.3):

Xi n = ( Yc v- A ) / B

Xm is the second lower bound for LDM.

4.5 Determining the limit of determination

For the final determination of the LDM, the values obtained for Xj (see 4.2) and Xm (see4.4) are compared The larger one of the two numbers is the limit of determination, LDM.For formulae to calculate Xc v, Ycv and X m, see 6.11-6.13.

5 Comments

The concept of deriving the LDC and LDM requires that the calibration curve is linear in therange of the fortification levels used, or that it can be converted to a linear form by a simplecoordinate transformation It is assumed that the variation of the signal values for each for-

Limits of Detection and Determination 13

tification level X follows a Gaussian distribution around the mean value Moreover, the tion of the signal values must be homogeneous for all fortification levels For this reason it

varia-is advantageous to choose such levels which are close to the anticipated LDC

If these conditions are not satisfied, the calibration curve must be fitted with the aid of anon-linear regression The Gaussian distribution may possibly be obtained by suitabletransformation of the signal values

If the variation of the signal values is dependent on concentration, it must be measured andtaken into account in the calculation of the regression, so that the calibration curve conceptcan be maintained as described, even under aggravated conditions, such as non-linear func-tions

The fortification levels chosen may include some which one later discovers to lie below theLDC Although these levels are not significantly distinguishable from blank values, they cannevertheless be used for calculating and constructing the calibration curve

The calibration curve can, alternatively, be obtained by converting the signal values intotheir corresponding concentrations (see 4.3) and plotting these values on the Y axis instead

of the signal values This form of the calibration curve was used in the Examples (see 9)

6 Mathematical formulation of the concept

6.1 General

Here, and in many other aspects of calibration statistics, the low-cost computer programsavailable for most personal computers (or even pocket calculators) are very helpful.For calculating the regression line (calibration curve) by hand, as well as for further calcula-tions required, first compute the following sums:

Formula Auxiliary Term

14 Limits of Detection and Determination

6.2 Derivation of the calibration straight line

The means of the X and Y values are each determined using the sums obtained from 6.1.

G 2

nThe auxiliary term

will also be required

The terms for the slope, B, and the axis intercept, A, of the calibration straight line are tained through:

sR = y —- E (Yj - A - BXj)2 = standard error of estimate: "mean"

deviation of the signal values Y{

from the calibration line

Limits of Detection and Determination 15

With £ ( Y - A - B X )2 = M - L2/K, the standard error can also be expressed as

6.4 Confidence limits of X = (Y - A ) / B for given signal value Y

K (t • s ) 2

C = B 2 - R ; = auxiliary term

K

In Figure 1, the confidence interval for X 3 is shown, with limits X { and X 2

6.5 Intersection point Y U P (Figure 1)

Y UP is obtained by setting X = 0 in the formula for the prediction limits, using the positive portion of equation I:

(III) Y UP = Y - B X + t - s R

-6.6 Derivation of LDC

LDC is obtained by inserting the value for Y UP (6.5) in equation IV:

Equation IV is directly deduced from equation II This, for describing Figure 1, is taking the form:

Substituting Y UP for Y o results in X 2 being identical with LDC, and one obtains (IV).

16 Limits of Detection and Determination

The value obtained for Yc from 6.7, when inserted in equation VI, gives Xji

(VI) XI = (YC-A)/B

6.9 Determination of SY

SY is identical with the product of sR and the square root expression in equation I (cf 6.11).The product t • SY describes half the width of the prediction interval of the calibration line(see Figure 1 and 6.10)

6.10 Definition of X c v

Inserting the expression for V, as given in 4.4, in the formulation of Requirement III (4.4),one obtains:

- | - : g V0 or SYgV0-Y(X)

Multiplying both sides with the factor t, and by adding Y(X) on both sides, one obtains:

Y(X) + t • SY ^ Y(X) + t • Vo • Y(X) =| Y(X) • (1 + t • Vo)

The expression on the left side of the less-or-equal sign is now just identical with the sion for the upper limits of the prediction interval (cf 6.9) On the right hand side, the values

expres-on the calibratiexpres-on straight line are multiplied by the factor (1 +1 • Vo)

In the limit of the equal sign, and in graphical interpretation, the formula describes the tersection of the line Y • (1 + t • Yo) with the curve of the upper prediction limits (Figure 5).The X-coordinate of the point of intersection is Xc v

in-6.11 Derivation of Xc v

Solving the equation Vo • Y = SY for X gives the value for Xc v, whereby

1-

Limits of Detection and Determination 17 and

Squaring both sides gives

(V0)2(Y + BZ)2 = (l + i + ^pj (sR)2

with Z = X c v - X

This is a quadratic equation for Z which can be solved for Z in a straightforward way, leading

to the following solution:

C / Bailey, E.A Cox and J.A Springer, High pressure liquid chromatographic determination

of the intermediates/side reaction products in FD & C Red No 2 and FD & C Yellow No 5:

Statistical analysis of instrument response, J Assoc Off Anal Chem 61, 1404-1414 (1978).

L Oppenheimer, T.P Capizzi, R.M Weppelman and H Mehta, Determining the lowest limit

of reliable assay measurement, Anal Chem 55, 638-643 (1983).

H Frehse and H.-P Thier, Die Ermittlung der Nachweisgrenze und Bestimmungsgrenze bei

Ruckstandsanalysen nach dem neuen DFG-Konzept, GIT Fachzeitschrift fur das

Laborato-rium 35, 285-291 (1991).

18 Limits of Detection and Determination

8 Authors

German version prepared for publication by:

Bayer AG, Research Department TPP 4, Leverkusen, Bayerwerk, H.-E Walter

Hoechst AG, Department of Informatics and Communication - Software, Frankfurt/Main,

K.-H Holtz;

in collaboration with H Frehse, S Gorbach and H-R Thier

English version prepared for this Manual by H.-P Thier and H Frehse

9 Examples

In this chapter, examples will be given on how to derive the LDC and LDM from a series ofmeasurements The measured values listed in Table 1 will be used for this purpose.Table 1 Measured values from recovery experiments.

Example 1 Example 2

Concentration [|ig/kg] Concentration [mg/kg]

Added (X) Found (Y) Added (X) Found (Y)

0.03 0.03 0.03 0.03 0.05 0.05 0.05 0.05 0.1 0.1 0.1 0.1 0.2 0.2 0.2 0.2

0.031 0.027 0.029 0.025 0.037 0.042 0.045 0.047 0.088 0.080 0.093 0.080 0.159 0.177 0.159 0.186

For the calculations to be made, all individual value pairs (measured values vs respectivefortification level) must be used The degrees of freedom increase with the number of calibra-tion points, n, whereby LDC or LDM may shift to lower values Therefore, do not formarithmetic means from the individual Y values obtained on a given level X!

The formulae needed for deriving the terms and quantities at the different steps of thecalculation are given in the text They will be quoted here either as numbers of equations or

in the form of the auxiliary terms as given under 6 For better legibility, the dimension terms(mg/kg or M-g/kg, respectively) are omitted

9.1 Example 1: Calculation of LDC and LDM

Note that in this case only one measurement per fortification level was made Number ofmeasurements: n = 8

Limits of Detection and Determination 19

When proceeding according to 6.1 and 6.2, the following results will be obtained:

The Intersection Point, Y UP , is obtained from eq Ill in 6.5, where A can be inserted for

Y - BX; t = 2.45 for f = n - 2 = 6, see Table 2:

Y UP = 31.21.

Accordingly, Y LO is obtained through

YLQ = A — t • s R • square root term in eq Ill = —11.91

(for quick calculation, use Y LO = 2 • A — Y UP ).

Note that the calculations outlined thus far must also be made when the procedure is tinued graphically "by hand" (see 9.3) The following steps are, however, given for illustration,

con-so that users carrying out the estimation of LDC or LDM either graphically or by computer (cf Figure 6) can check their results versus the figures given here.

Using eqs II (for C) and IV, one obtains

20 Limits of Detection and Determination

Limits of Detection and Determination 21

example In the first case, all values given in Table 1 are used (n = 16), while in the second case the calibration points obtained for the fortification level 0.2 were disregarded (n = 12) Calculations in analogy to the one outlined in Example 1 will yield the following results:

n = 12

(t = 2.228) 0.0026 0.8240 0.0045 0.0146 -0.0094 0.0280 0.0414 0.0269 0.0358 0.0403 0.04

In both cases, recoveries were_fully sufficient (84 and 82%).

In the second case (n = 12), X ( = 0.06) is closer to LDC and LDM, and the prediction terval is a little narrower (cf the difference in the values for s R ) than in the first case (n = 16), where X = 0.095.

in-9.3 Example 1: Graphical derivation of LDC and LDM

The mathematical expressions (Chapters 6.6-6.13) were given in order to describe the model,

at the same time serving as a basis for establishing suitable computer programs.

Without computer support, it is rather laborious to construct the upper and lower tion limits In general, however, these limits are only needed in a region between X = 0 and

predic-X = predic-X 2 (see Figure 1) For a graphical derivation of the LDC and LDM it is, therefore, missible to substitute the curves for these limits by straight lines, which are drawn, parallel

per-to the calibration line, through the points Y UP and Y ^ (A mathematical check of the quality of the approximation can be made by calculating some values for Y+ and Y_ using equation I.)

This simplification permits the derivation of the LDC and LDM "by hand" on graph paper The graphical procedure is described here, using Example 1:

- Calculate the terms E through N, as well as X, Y, A, B, Y UP and Y ^ as described in 9.1

— Draw the calibration line by connecting two points, e g X = 0, Y = 9.65 and X = 100,

Y = 94.8, using the linear equation Y = 9.65 + 0.85 • X to obtain the Y values

- Draw two lines, parallel to the calibration line, through Y UP and Y ^

- Derive LDC as illustrated in Figure 7, obtaining a value of approx 50 on the X-axis

— Derive Xj as shown in Figure 8, obtaining a value of approx 75 on the X-axis

— For deriving X m , draw a straight line through two points, e.g X', Y* and X", Y**, so that

it intersects the upper parallel as illustrated in Figure 9 Y* and Y** are obtained from the equation of the calibration line by multiplying the resulting Y values (Y' and Y") by

1 + 0.2 • t (for explanation, see 4.4); in this case, t = 2.45 (see Table 2).

22 Limits of Detection and Determination

Example: X' = 20, Y' = 26.7, Y* = 39.8

X" = 60, Y" = 60.8, Y** = 90.5

The point of intersection corresponds to Y c v and yields X m being approx 62.

The outcome of the graphical derivation is in good agreement with the results obtained from calculation (9.1) The approximation achieved by such a procedure will be sufficient for residue analyses in most cases.

LDC

Fig 7 Graphical derivation of LDC.

UX X

Fig 8 Graphical derivation of

Fig 9 Graphical derivation of X

Limits of Detection and Determination 23

f

10 11 12 13 14 15 20 30 40

t two-sided 2.228 2.201 2.179 2.160 2.145 2.131 2.086 2.042 2.021