5.3.1 General
The requirements for the overall quality of the sensory measurement of odour concentrations in one laboratory shall be assessed by performance testing to demonstrate and ensure compliance on a regular basis. The
procedure and calculations are defined in 5.3.2.3. The frequency of performance testing shall be determined taking into account the performance history of the laboratory, but shall be no less than once a year.
Compliance with the quality criteria shall be determined on the basis of test results obtained for certified reference materials for which an accepted reference value is available.
correcting measurement results on the basis of the relation between the actual panel threshold and the accepted reference value.
To determine whether a laboratory complies with this European Standard only criteria applying to accuracy, trueness and precision for reference materials are considered. In addition, the precision expressed as repeatability for non-reference odorants can be assessed and should comply with the same criterion as set for the reference odorant.
For environmental odours, however, an accepted reference value is not available. This implies that for non-reference odorants, such as environmental odours, only precision can be tested. Trueness cannot be determined in these cases.
If a laboratory complies with the overall sensory quality criteria for the reference material, this standard assumes that this quality level is transferable to other, environmental, odours (van Harreveld, Heeres 1995, see
Bibliography).
NOTE 1 To confirm this assumption on environmental odours, interlaboratory comparisons should be conducted to assess precision and compare precision with that obtained for reference material.
When conducting inter-laboratory comparisons the precision shall be assessed in terms of repeatability and reproducibility and in terms of trueness (laboratory bias). These values for a laboratory shall be compatible with the values found for the reference material.
'When calculating statistical parameters the decimal logarithms of the measured odour concentration values shall be used. To obtain a value of non-logarithmic units, the value can be re-converted into its antilog. For assessing instrumental quality requirements a logarithmic conversion shall not be used.
NOTE 2 Examples of the calculations for performance testing are given in annex D.
NOTE 3 The frequency distribution for detection thresholds for an odorant is log-normal. The exponential nature of the data can be understood by realising that thresholds are determined by presenting an odour panel with a sequence of dilutions that are a fixed factor apart. The multiplication of dilution factors can be expressed on a linear scale after logarithmic transformation.
NOTE 4 It is useful to plot test results obtained on reference material on quality control charts, enabling a visual check to assess whether the quality is sufficiently under control.
5.3.2 Quality criteria for the performance within one laboratory on reference material (odorant) 5.3.2.1 Quality criterion for accuracy
The accuracy reflects both the trueness (expressed as bias) and the precision (random error). The test variable for accuracy is Aod.
To assess if a laboratory complies with the accuracy criterion, the 95 % confidence interval for the within-laboratory bias δw is first calculated as:
r A d r
A
dw − w⋅ ≤ δ w ≤ w + w⋅ (4)
where
A n
= ⋅ 2
1
w (5)
where
Aw is a statistical factor;
dw is the trueness, expressed as the estimate of within-laboratory bias;
n is the number of test results;
r is the repeatability limit.
The calculations of trueness and precision are described in 5.3.2.4 and 5.3.2.5.
The test variable Aod is then calculated. The criterion for accuracy of the odour concentration is:
( w ) 0 , 217
w
od = d + A ⋅ r ≤
A (6)
NOTE Examples of the calculations for performance testing are given in annex D.
5.3.2.2 Quality criterion for precision (repeatability)
In addition to the overall accuracy criterion, the precision, expressed as repeatability limit, shall comply with:
477 ,
≤ 0 r
This repeatability limit can also be expressed as:
0 , 3 10r ≤
NOTE 1 This requirement implies that the factor that expresses the difference between two consecutive single
measurements, performed on the same testing material in one laboratory under repeatability conditions, will not be larger than a factor 3 in 95 % of cases.
NOTE 2 Examples of the calculations for performance testing are given in annex D.
5.3.2.3 Procedure for testing compliance
To assess if a laboratory complies with the criteria for accuracy and precision, expressed as repeatability, the following procedure applies. Odour concentration measurements shall be performed on a certified reference material of the reference odorant, under repeatability conditions, using one or more concentration levels compatible with the measuring range. At least 10 test results, all measured within the most recent twelve months, shall be used for testing compliance.
The expanded uncertainty of the concentration of the reference material shall be ± 5 % or less in terms of concentration. The operator should not be aware of the concentration of the samples.
NOTE The test results can be obtained on reference material at one concentration, as the dilution performance of the olfactometer is assessed in a separate procedure. However, to obtain an overall quality assessment the use of multiple concentration levels within the measuring range is advisable.
5.3.2.4 Calculation of precision (repeatability)
First the repeatability limit r is calculated from the test results.
The repeatability standard deviation for the laboratory sr is calculated using:
) 1 (
) (
1
2 w
r −
−
= ∑
=
n y y s
n
i i
(7)
where
n is the number of test results;
yw is the average of test results;
yi is the test result.
The repeatability limit r is then calculated using:
2 sr
t
r = ⋅ ⋅ (8)
where
t is a factor from the Student's t-distribution for n - 1 degrees of freedom, and a confidence level of 95 %, (Lide, see Bibliography).
The repeatability limit is then compared with the quality criterion.
5.3.2.5 Calculation of trueness (bias) The within-laboratory bias δw is estimated by:
à
−
= w
w y
d (9)
where ywis the average of the test results.
5.3.3 Assessment of the performance on non-reference materials (odorants) 5.3.3.1 General
The accuracy reflects both the trueness (expressed as bias) and the precision (random error). However, for non- reference odorants, for which no accepted reference value is available, the bias of the measurement method (the term δ in the statistical model) cannot be assessed.
The precision expressed as repeatability and as reproducibility however can be assessed and shall be calculated as specified below. The results shall comply with the same criteria as those for reference materials.
5.3.3.2 Assessment of precision within one laboratory (repeatability)
Performance testing to assess compliance with the criterion for precision (expressed as repeatability) is done by performing measurements on identical samples of the odorant(s) concerned, under repeatability conditions. At least 10 test results are required.
The calculation of the repeatability limit from the test results is identical to that in 5.3.2.4.
The repeatability limit shall comply with the same quality criterion that applies to the reference odorant:
477 ,
≤ 0 r
5.3.3.3 Assessment of precision between laboratories (reproducibility)
For non-reference odorants an accepted reference value is not available. Consequently the bias of the
measurement method cannot be quantified. To assess accuracy for odorants or environmental odours when no agreed reference value is available an interlaboratory comparison should be held where a suitable number of laboratories all analyse identical samples.
The geometric mean of the odour concentrations in ouE/m3 of all the participating laboratories is then considered to be the best estimate of the reference value à.
Using that best estimate the analysis of accuracy can be made along the same lines as for the reference material.
However, the precision in this analysis will be expressed in terms of reproducibility.
To carry out assessment of trueness (laboratory bias) and precision (expressed as reproducibility) measurements are performed on identical samples under reproducibility conditions. These samples can be produced for
environmental odours by diluting a concentrated sample using a calibrated dilution apparatus. At least 10 test results per laboratory are required. Such results should be obtained in a proficiency test, following the guidance of ISO 5725-2.