Annals of Occupational Hygiene Advance Access originally published online on April 21, 2005
Annals of Occupational Hygiene 2005 49(4):325-334; doi:10.1093/annhyg/mei017
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Published by Oxford University Press (2005);
Original Article |
Proposal to Adapt the Workplace Analysis Scheme for Proficiency (WASP) Programme to Fibre Counting Tests
Institut National de Recherche et de Sécurité (INRS), Avenue de Bourgogne, BP27, 5450L Vand
uvre lès Nancy, France
* Author to whom correspondence should be addressed. E-mail: michel.grzebyk{at}inrs.fr
ABSTRACT
Three methods of classifying laboratories during fibre counting proficiency tests were compared. The first two are those used in France (classification according to the mean and coefficient of variation of the results) and in Great Britain (classification according to the proportion of normalized results situated within predefined limits). The third is a variation of the Workplace Analysis Scheme for Proficiency (WASP) programme adapted to fibre counting tests. In the latter case, the laboratory classification is based on comparing the variance characterizing the dispersion of the results of a laboratory with a reference variance, which is considered as the variance of experienced analysts or laboratories. This mode of processing has the advantage of allowing the comparison of magnitudes. For example, the variance of the reference value can be compared with the reference variance. The same applies if a proficiency test is organized on the basis of replicas distributed to different analysts, the variability of these replicas can be compared with the reference variance. It emerged that the modified WASP method produces results close to those obtained by the other two methods. Moreover, the selectivity of the three methods is evaluated.
Keywords: fibres proficiency testing scheme quality control
INTRODUCTION
The method most commonly used to evaluate the airborne fibre concentration is the membrane filter method. The fibres are sampled on a filter and if identification is unnecessary they are counted by means of phase contrast optical microscopy (PCOM). The fibres counted are generally those longer than 5 µm with a width of <3 µm and a length to width ratio >3:1. Counting fibres over a certain length not only reduces the variability due to the limitations of the optical microscope, but also takes into account the fact that the longest fibres appear to be the most dangerous (Lynch et al., 1970
). Cut-off at a width of 3 µm allows separation of respirable fibres from non-respirable fibres. Since the start of the 1970s, this method has been gradually improved to the point where there has been a proposal for international harmonization (WHO, 1997
). Other methods employing transmission electron or scanning electron microscopy can be used, if necessary, to identify the fibres present on the sampling filter (VDI, 1991
; ISO, 1995
, 1999
).
In 1976, it appeared necessary to compare the counts obtained by different analysts (Walton et al., 1976
). Contrary to most analytical methods, fibre counting leaves much to interpretation and, due to the distribution of fibre sizes, correct adjustment of the microscope is particularly important. Since then, numerous proficiency tests have been organized to ensure the coherence of the counts made. These include those carried out in Belgium (Grosjean, 1998
), France (Carton et al., 1981
; Kauffer, 1989
; Kauffer et al., 2001
), Spain (Arroyo and Rojo, 1998
), Great Britain (Brown et al., 1994
, 2002
; Brown and Jones, 2001
), and the United States (Schlecht and Shulman, 1995
). The classification of laboratories during these tests is based on different principles. In this respect, in the Proficiency Analytical Testing (PAT) in the United States, the acceptance limits are set at ±3 times the standard deviation of the mean of the results. For the tests carried out in Belgium, Spain and Great Britain, the classification is based on the proportion of the results situated within predefined limits, which can depend on the density of the fibres on the filter undergoing test. In France, for counting by phase contrast optical microscopy (Kauffer, 1989
), analysts are classified on the basis of the mean and the coefficient of variation of their results.
Differences in both the evaluation and the organization of these tests make them difficult to compare. Recently, a proposal for harmonization was put forward (Arroyo and Rojo, 2001
). In this respect, in the different proficiency tests, a standardization of the number of samples used for laboratory assessment is recommended, as are acceptance limits chosen in a way that the probability of success is the same whatever the proficiency test in question. This same concern has also resulted in a comparison of the proficiency tests organized in Great Britain and the United States (Song and Schlecht, 2000
).
Independent of the differences that may exist in the organization of proficiency tests from one country to another, the principles retained for the same country can be different depending on whether fibres or other pollutants encountered in occupational safety and health are involved. Great Britain, for example, employs the Workplace Analysis Scheme for Proficiency (WASP) programme (Jackson and West, 1992
), while in France it is Aptitude des Laboratoires pour l'Analyse des Substances Chimiques dans l'Air (ALASCA) that is used. In this proficiency test the classification is based on comparing the variance characterizing the dispersion of the results of a laboratory to a reference variance, for a given analytical technique and pollutant.
The aim of this article is to show that the principle adopted for the WASP programme can be extended to judge the performance of laboratories involved in fibre counting. The model proposed will be compared with the data of the last three proficiency tests organized in France for fibre counting by PCOM. During these tests, 21 microscope slides ready to be observed were evaluated by the participants. These slides were divided into six batches, the slides of the same number of each batch were assumed to be identical (Kauffer, 1989
). The slides were counted by all the analysts of each laboratory. For the years 2001, 2002 and 2003, 95, 85 and 90 analysts from 35, 31 and 30 laboratories took part in the tests. This allows the evaluation of the performance of each analyst. In some other countries each laboratory has to count all the test samples, but individual analysts count only some of the samples. In these cases, it is of course the performance of each laboratory which is evaluated.
METHODOLOGY
Three methods allowing an appraisal of the quality of the results of analysts involved in fibre counting by PCOM were compared. The first method is that used in France, the second is that employed in Great Britain in the Regular Interlaboratory Counting Exchanges (RICE) test and the third is an adaptation of the WASP programme. The principle of these methods is described below. To facilitate the comparisons, for a given level of loading, the reference value Rf was always taken as equal to the median of all the results of the participants. In addition, the procedure used to calculate the selectivity of these methods is also described.
Method 1
For all the N results of a proficiency test, normalized results [result obtained by an analyst (Rs) divided by the corresponding reference value (Rf)] are calculated.
The classification of an analyst is based on the mean M and the coefficient of variation CV of these normalized results:
![]() |
![]() |
- Group 1: 0.75 < M < 1.33 and CV < 0.4,
- Group 2: 0.50 < M
0.75 or 1.33
M < 2.00 and CV < 0.4,
- Group 3: M
0.50 or M
2.00 or CV
0.4.
Method 2
For this method (Brown et al., 1994
), it is the proportion of results falling within predefined limits that determines the classification of the laboratory or analyst. These limits depend on the density of the fibres on the filter. They are defined as follows:
- High densities:
- internal limits: 0.65 x Rf to 1.55 x Rf,
- external limits: 0.50 x Rf to 2.00 x Rf.
- internal limits: 0.65 x Rf to 1.55 x Rf,
- Low densities:
- internal limits:
to
,
- external limits:
to
.
- internal limits:
Classification in groups 13 depends on the percentage of the results located within the limits:
- Group 1: 75% of the results within the internal limits,
- Group 2: 75% of the results within the external limits,
- Group 3: other cases.
to
and from
to
, respectively. As the number of filters distributed each year in the French proficiency test is equal to 21, the limit in terms of the results to define the classification in the different groups was taken as 16 (75% of 21 equals 15.75, rounded up to 16).
Method 3
The quantity R is calculated as:
![]() |
In practice, for a proficiency test where the same filters are analysed, Vari(Rf) is the variance of analysts or laboratories normally using the method. In the case of replicas being distributed to the participants, this variance includes the variability of the replicas.
Let us assume that this quantity R follows a
2 law with N degrees of freedom. In this framework, the classification in groups is carried out on the basis of comparing R with the values
(group 1) and
(group 3) for N degrees of freedom.
This formula is identical to that presented in the WASP programme if
.
In the case of fibre counting when the densities are high, the stopping criterion is the number of fibres counted, and in this case the standard deviation of the results is proportional to the reference value (Rf). The variability is then constant on a logarithmic scale. When the densities are low, the stopping criterion is the number of fields counted. In this case, the standard deviation of the results is proportional to the square root of the reference value, and the variability is constant on a square root scale (Miller, 1984
; Brown et al., 1994
, 2002
).
Thus, we obtain:
- for high densities Vari(Rf) = a x (Rfi)2
- for low densities Vari(Rf) = a' x Rf i.
![]() |
- For high densities:

As an approximation (development limited to first order), the variance of a function g(x) is equal to the square of the derivative of the function for a mean value
of the variable times the variance of x (Kendal and Stuart, 1969
):

Taking the reference value (Rf) as the mean value, we deduce:
and hence,

- For low densities:

Hence,

Laboratories can therefore be classified in three groups by calculating, for all the results, the quantity
where fi = 0 for high densities and fi = 1 for low densities.
In practice, the WASP programme is a sliding system that only takes into account the best of the most recent rounds. To take account of this particularity, the 21 results yielded by the analysts for a given year were split into four groups to simulate four fictitious rounds:
- Round 1: Results corresponding to counting of the slides 15.
- Round 2: Results corresponding to counting of the slides 610.
- Round 3: Results corresponding to counting of the slides 1115.
- Round 4: Results corresponding to counting of the slides 1621.
This grouping allows the following calculations:
![]() |
![]() |
![]() |
![]() |
In addition, to ensure that one poor set of results does not automatically result in a category 3 rating for a long time, irrespective of any improvement in performance, a ceiling value for individual rounds is applied in the calculation of R.
For each fictitious rounds, the ceiling value was set by adopting the following rule:
![]() |
2 for 15 degrees of freedom and 5 is the mean of
2 for 5 degrees of freedom. Here it was supposed that the worst round was Round 4 (hence 15 degrees of freedom). In the other cases, the ceiling values were computed in the same way with 16 degrees of freedom for the 97.5% percentile of
2.
Let us now take into account the results of the best three of the last four rounds in the calculation
![]() |
If the number of slides distributed in each round is identical, the calculation is more simple, as shown in Appendix 1.
Selectivity of the different methods
The selectivity of the three methods was determined for a proficiency test involving four rounds of eight density levels (of which four are low density levels and four are high density levels). Selectivity consists in the probability that an individual analyst will be rated in a particular group, given its performance. We focussed on the probability of being rated in group 1 or 2 because being rated in group 3 means not passing the test. The performance of an analyst is characterized by his (multiplicative) bias b and his coefficient of variation CV, observed for high densities.
For high densities (>127.32 fibres mm2), it is assumed that the results are log-normally distributed. Thus, as shown in Appendix 2, the mean and the standard deviation of the logarithm of the measured densities are respectively:
![]() | (1) |
![]() | (2) |
For low densities (<127.32 fibres mm2), it is assumed that the square root of the results is distributed as the absolute value of a normal random variable whose standard deviation is independent of the density level. This hypothesis is slightly different from the usual hypothesis that the square root of the results are normally distributed (Miller, 1984
; Brown et al., 1994
, 2002
) but when the mean of the square root is high with respect to the standard deviation, the difference is numerically small. Thus, as shown in Appendix 2, the coefficient of dispersion of the analyst (the ratio of the standard deviation and the square root of the mean value) is
and the mean and standard deviation of the normal distribution corresponding to the square root of the results of an analyst with bias b and coefficient of variation CV at density level Rf are respectively:
![]() | (3) |
![]() | (4) |
- 4 rounds of 8 high density levels;
- 4 rounds of 8 low density levels;
- 4 rounds of a mixture of 4 high and 4 low density levels.
RESULTS AND DISCUSSION
Figures 1
3 show the selectivity of the three methods for different density patterns of the slide (classification based on 32 high- or 32 low-density slides or a mixture of 16 high- and 16 low-density slides). In these figures, the bias b and the coefficient of variation CV of an analyst must lie in the region under the contour line to have more than 95% chance of being rated in group 1 or 2. Figure 4 compares the selectivity of the three methods in the case where high and low densities are mixed. The coefficient of variation CV for high densities is reported on the left vertical axis. The coefficient of dispersion for low densities (ratio of the standard deviation to the square root of the mean value) is reported on the right axis. It appears that the selectivity of methods 2 and 3 are comparable whereas method 1 is more selective with respect to the coefficient of variation. This is expected as the selection process in method 1 is based on the estimated coefficient of variation of each analyst, no distinction being made between high and low densities. Provided that an analyst knows his bias and precision, Figs 1
3 can also be used to evaluate his probability of success in one or the other of the three proficiency tests (less or more than 95%). For high densities the precision is characterized by the CV. For low densities the precision is characterized by the ratio of the standard deviation to the square root of the mean value. Of course if necessary other graphs with different probability levels could be computed. Inversely, this model can be used by the organizer of proficiency tests, to estimate the bias b and the coefficient of variation CV of each laboratory using its results. This can be done by means of the maximum likelihood estimation method.
|
|
|
|
The comparisons of the three methods described in this paper are based on data acquired during proficiency tests organized in 2001, 2002 and 2003. For these data, the density ranges which divide the results into three equal lots were: 57119, 119171 and 171314 fibres mm2. Furthermore, for 41% of the slides the loading was less than the density (127.32 fibres mm2) separating (for the counting rules used) low densities from high densities.
In method 3, the quantity a is the reference relative variance. Figure 5 shows the number of analysts classified in group 3 as a function of a, by distinguishing the case where all the results were taken into account for the classification (21R for 21 results), from that where only the best three of the last four fictitious rounds were taken into account (15R for 15 results). In fact, depending on whether the worst of the four fictitious rounds was constituted of six or five slides, the overall judgement was based on 15 or 16 results, but to avoid complicating the explanations unnecessarily, these will be referred to as the 15 results.
|
Table 1 gives the list of analysts identified by a number that were classified in group 3 for the three methods described earlier. For method 3, the results are given for different values of parameter a (a varying between 0.14 and 0.18) and for each time two cases were envisaged (taking into account all the results (21R) or the best three of the last four fictitious rounds (15R)). On the whole, the number of analysts classified in group 3 by methods 1 and 2 was by and large the same (19 for method 1, 21 for method 2), 15 analysts having an identical classification. The differences observed concern analysts with results close to the limits set for the classification or are the result of the rules adopted in the two methods. For example, for method 1, where the coefficient of variation CV is involved in the classification, a very bad result can lead to an analyst being classified in group 3, which is not possible with method 2. If the results obtained with method 3 are now considered, it can be seen that a reference relative variance between 0.14 and 0.18, for the data analysed in this article, yields a classification close to those obtained with the two other methods. For a equal to 0.18, 18 analysts were classified in group 3 if all the results are taken into account (21R), 14 if only the best three of four fictitious rounds (15R) are considered. All the laboratories with the exception of one, classified in group 3 by method 3, are also classified in group 3 by method 2.
|
Table 2 gives the relative proportions of analysts classified in groups 1, 2 and 3 for the three methods. On the whole, the agreement is good between the results yielded by the three methods. A balancing-out effect favouring group 2 was observed for method 3.
|
The comparison of the three methods has shown that it is possible to adapt the principle retained in the WASP programme (comparison of variance) to proficiency tests dedicated to fibre counting. The reference relative variance determined in method 3 is coherent with the dispersion expected between laboratories for fibre counting. A value of a equal to 0.18 corresponds to a coefficient of variation of 0.42, which is very close to the value indicated for the inter-laboratory coefficient of variation (0.45) in method 7400 of National Institute of Occupational Safety and Health (NIOSH, 1994
The method proposed has the advantage of examining with the same model all the pollutants found in the field of industrial safety and health, whether fibrous or not, which favours the comparison of different proficiency tests. Knowledge of the reference relative variance, and where applicable of the relative variability of replicas, allows the different proficiency tests organized in the different countries to be compared for the same pollutant. As regards tests intended for fibre counting, this method also allows direct comparison of different analytical techniques (e.g. PCOM and transmission electron microscopy (TEM)). Arroyo et al. (2001)
, further to a suggestion made by Ogden (1984)
, proposed that fibre counting proficiency tests be based on counting 32 filters. In method 3, the number of filters analysed during the proficiency test is directly taken into account. The values of
2 allowing the classification of laboratories into different groups depend on the number of degrees of freedom, and therefore on the number of filters analysed.
Brown et al. (2002)
pointed out that it was easier to pass the RICE test with low-density slides than it was with high-density slides. This is indeed what is observed in the data of the present report. If method 2 is considered, 6.29% of the normalized results (Rs/Rf) are outside the external limits for the high-density slides as opposed to 4.89% for the low-density slides. The same tendency was noted with method 3. The proportion of individual values of factor R exceeding 5.02 (value of
for one degree of freedom) is equal to 2.81% for high-density slides compared to 1.72% for low-density slides. This could also be seen in Figs 2 and 3 where for a CV <
0.45 the selectivity for low-densities slides is smaller than the one for high-densities slides.
CONCLUSION
This article has demonstrated the possibility of using a method based on comparing the variance dispersion of a laboratory with a reference dispersion to classify the results obtained during proficiency tests organized for fibre counting. This reference dispersion, if the dispersion variability of replicas is negligible, is that characterizing the dispersion of the results of a group of laboratories normally employing the same method. A reference relative variance of between 0.14 and 0.18, for the data analysed in this article, allowed results to be obtained close to those obtained by traditional methods.
This processing mode has the advantage of allowing certain comparisons. For example, the variance of the reference value can be compared with the reference variance. In addition, if the proficiency test is organized on the basis of replicas distributed to the different participants, the variability of these replicas can be compared to the reference variance. It also allows processing of all pollutants, whether fibrous or not, with the same model whatever the analytical technique: counting, weighing, conventional analysis. If it is assumed that the reference relative variance is an approximation of the square of the coefficient of variation of the method, this mode of processing facilitates the comparison of different analytical techniques.
APPENDIX 1
Adaptation of the WASP model to fibre counting proficiency tests; Case of a test spread over four rounds during which the laboratory counts eight slides
For each test (every 3 or 4 months), the laboratory receives eight microscope slides for counting. The evaluation of the laboratory is based on taking into account the best three of the last four rounds and the result of each test if this leads to too high a value of R, is replaced by a ceiling value.
For each round, the quantity R is calculated as:
![]() |
![]() |
![]() |
The preceding calculation is repeated for the four rounds, hence R1, R2, R3 and R4.
If for one of the rounds the result is higher than the ceiling value C, this result is replaced by this value.
The ceiling value is determined in the following way:
![]() |
2 for 24 degrees of freedom and 8 represents the mean of
2 for 8 degrees of freedom.
Taking into account the best three of the last four round results in the calculation
![]() |
- If R is lower than 12.4 (
for 24 degrees of freedom), the laboratory is classified in group 1.
- If R is higher than 39.4 (
for 24 degrees of freedom), the laboratory is classified in group 3.
- If R is between 12.4 and 39.4, the laboratory is classified in group 2.
APPENDIX 2
Determination of the parameters for the simulation
The performance of an analyst is characterized by his (multiplicative) bias b and his precision. As the bias b of an individual analyst is multiplicative, the mean value of a result is therefore b x Rf when the density level is Rf. This relation stands whether the density is high or low. The dispersion of an individual analyst is characterized as explained above when describing the third method. For high densities (higher than 127.32 fibres mm2), the dispersion is characterized by a constant coefficient of variation CV. For low densities (less than 127.32 fibres mm2), the dispersion is such that the ratio of the standard deviation of a result and the square root of the mean value is constant (Miller, 1984
; Brown et al., 1994
, 2002
): this constant coefficient is called the coefficient of dispersion. As in method 3, the equality of the variances for this density implies that the coefficient of dispersion is
, where Dl is the limit between high and low densities. This constitutes the measurement hypothesis and are the basis of the simulations.
We derive now the parameters for the simulations.
When the stopping rule is the number of counted fibers (high densities), the measured density Rs is assumed to be higher than the density limit Dl = 127.32 fibres mm2. Then the distribution of its logarithm is assumed to be normal:
![]() |
.
These parameters are related to the bias b and coefficient of variation CV by:
![]() |
![]() |
.
![]() |
is the non-central
2 distribution with one degree of freedom and non-centrality parameter
(Kendal and Stuart, 1973). The results show that the mean value and the variance of Rs are:
![]() |
![]() |
![]() |
![]() |
- 3.
- 4.
.
- 4.
2 requires that
. Received April 8, 2004; in final form February 17, 2005
REFERENCES
Arroyo MC, Rojo JM (1998) National versus international asbestos fibre counting schemes: comparison between the spanish interlaboratory quality control programme (PICC-FA) and the asbestos fibre regular informal counting arrangement (AFRICA). Ann Occup Hyg; 42: 97104.
Arroyo MC, Rojo JM. (2001) A proposal for harmonising laboratory performance assessment criteria in national asbestos fibre counting schemes. Ann Occup Hyg; 45: 44755.
Brown PW, Crawford NP, Jones AD et al. (1994) Development of the regular interlaboratory counting exchanges (RICE) scheme to include visual reference counts and samples from asbestos clearance operations. Ann Occup Hyg; 38: 687703.
Brown PW, Jones AD. (2001) Interlaboratory comparison schemes for fibre counting: AFRICA and the WHO/EURO reference scheme. GefahrstoffeReinhaltung des luft 61: 1014.
Brown PW, Jones AD, Miller BG. (2002) Developments in the RICE asbestos fibre counting scheme. Ann Occup Hyg; 46: 32939.
Carton B, Kauffer E, Derzko G et al. (1981) Results of an asbestos counting trial performed in France in 1979. Ann Occup Hyg; 24: 921.
Grosjean R. (1998) Proficiency testing schemes for workplace measurements, In: European News. Spring 1998. WASP Steering Committee, Health and Safety Executive, 1998.
ISO 10312. (1995) Ambient airDetermination of asbestos fibresDirect transfer transmission microscopy method.
ISO 13794. (1999) Ambient airDetermination of asbestos fibresIndirect-transfer transmission microscopy method.
Jackson HM, West NG. (1992) Initial experience with the workplace analysis scheme for proficiency (WASP). Ann Occup Hyg; 36: 54561.
Kauffer E. (1989) The French asbestos quality control scheme, In: Proceedings of gthe Sixth International colloquium on dust measurement technique and strategy, Jersey, Channel Islands, 2830 November 1989. Paris: Asbestos International Association; 1989, 15663.
Kauffer E, Martine M, Grzebyk M et al. (2001) Performance of asbestos fibre counting laboratories by transmission electron microscopy. Ann Occup Hyg; 45: 619.
Kendall MG, Stuart A. (1969) The advanced theory of statistics, third edition, Vol. 1. London: Charles Griffin, p. 232.
Kendall MG, Stuart A. (1973) The advanced theory of statistics, third edition, Vol. 2, London: Charles Griffin, p. 237.
Lynch JR, Ayer AE, Johnson DL. (1970) The interrelationships of selected asbestos exposure indices. Ann Ind Hyg Assoc J; 31: 598604.
Miller BG. (1984) A statistical method for analysis of membrane filter samples of airborne fibre. Ann occup Hyg; 28: 21727.
NIOSH (1994) Asbestos and other fibers by PCM. NIOSH Manual of Analytical Methods (NMAM), 4th edn. 8/15/94.
Ogden TL. (1984) Statistical uncertainties in asbestoslaboratory classification through the RICE scheme. Ann Occup Hyg; 4: 44952.
Schlecht PC, Shulman SA. (1995) Phase contrast microscopy asbestos fiber counting performance in the proficiency analytical testing program. Am Ind Hyg Assoc J; 56: 4809.[Web of Science][Medline]
Song R, Schlecht PC. (2000) Comparison of performance rating criteria in proficiency testing programs. In: Proceedings of the 1999 American Statistical Association, Section Physical and Engineering Sciences, 2000, Baltimore, MD: American Statistical Association, pp. 179184.
VDI 3492. (1991) Measurement of Inorganic Fibrous Particles in Ambient Air (Scanning Electron Microscopy Method) VDI-Handbuch Reinhaltung der Luft, Band 4.
Walton WH, Attfield MD, Beckett ST. (1976) An international comparison of counts of airborne asbestos fibres sampled on membrane filters. Ann Occup Hyg; 19: 21524.
WHO. (1997) Determination of airborne fibre number concentrations. A Recommended method by phase-contrast optical microscopy (membrane filter method). Geneva: World Health Organization.
This article has been cited by other articles:
![]() |
M. GRZEBYK, E. KAUFFER, and L. FREVILLE Proposal to Adapt the WASP Programme to Fibre Counting Tests Ann. Hyg., June 1, 2006; 50(4): 427 - 427. [Full Text] [PDF] |
||||
![]() |
B. PREAT Comments About the Proposal to Adapt the WASP Programme to Fibre Counting Tests Ann. Hyg., January 1, 2006; 50(1): 105 - 105. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
































