Annals of Occupational Hygiene Advance Access originally published online on May 6, 2005
Annals of Occupational Hygiene 2005 49(7):549-561; doi:10.1093/annhyg/mei018
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Original Article |
Meaningful Workplace Protection Factor Measurement: Experimental Protocols and Data Treatment
1 Health and Safety Laboratory, Harpur Hill, Buxton SK17 9JN, UK; 2 Health and Safety Executive, Bootle L20 3QZ, UK
* Author to whom correspondence should be addressed. vaughan{at}npvaughan.demon.co.uk
 |
ABSTRACT |
|---|
TOP
ABSTRACT
INTRODUCTIONWorkplace performance measurement of respiratory protective equipment (RPE) is fundamental to the understanding of how well wearers are protected. It forms the basis for guidance on which the selection of appropriate equipment is based. However, the measurement of this performance is open to many sources of interference and inaccuracy, reducing the value and relevance of the results, and is most difficult for devices providing the highest levels of protection. In this paper, a method for critically assessing collected workplace protection factor (WPF) data is validated. This method rejects unreliable data, using criteria based on the detection limits of the analytical measurement system. An iterative approach is also described which arrives at a supportable estimate of given non-parametric percentiles of the distribution of measured WPFs [such as the fifth percentile, conventionally taken to be the assigned protection factor (APF)]. Further pragmatic criteria, based on a combination of experimental experience and consideration from first principles, are suggested as the basis for any future studies of RPE performance. These will maximize the chances of valid measurements being made, and also provide insight into the level of confidence which can be placed on any of the results. A consequence of these criteria is that typical working environments and measurement methods are incapable of generating WPF data which can be considered reliable.
Keywords: respiratory protection • workplace protection factor • assigned protection factor • measurement protocol
|
INTRODUCTION |
|---|
Measurement of the workplace protection factor (WPF) for respiratory protective equipment (RPE) depends on accurate measurement of simultaneous contaminant concentrations inside and outside the facepiece of the equipment. WPF is a derived value, calculated from the ratio of outside to inside concentrations. Frequently, in occupational situations where exposure controls are in place in addition to RPE, concentrations inside or outside the facepiece, or both, are low and may be just above, at or below the detection limits of the analytical method which is being applied. In these cases, calculated protection factors (PFs) are inherently unreliable, and may need to be excluded from consideration. Criteria are needed to identify, and establish sound rejection criteria for, data which fall into this category. Similarly, measurement protocols need to take into account the analytical detection limits of the methods being used, and the limitations they impose on the reliability of measured PFs.
The important parameters which determine acceptability of data are the detection limit (DL) of the analytical system used (including both instrumental precision and sampling variability), and the actual level of protection (or distribution of protection levels achieved over a period of time) of the RPE. However, in real situations (whether in a workplace or simulated workplace), it is almost invariably the second of these parameters, the PF, which the workplace study is trying to measure. In this paper, criteria are described to assist in the design of WPF measurement programmes, to ensure that the PFs obtained are reliable and meaningful.
Even with these precautions, concentrations measured in the workplace may be in the region of the detection limit of the analytical system(s) used. A data treatment method is required which identifies and rejects data pairs from a set of simultaneous challenge and in-facepiece measurements which are unduly affected by proximity to the detection limits of the measurement method. Such measurements are fundamentally unreliable and produce values of PF which are unrealistic and invalid. This is a logical extension of the recommendation of Johnston et al. (1992)
.
A possible data treatment method was developed during analysis of WPF data from paint spraying environments where compressed air-fed visor breathing apparatus was in use (Bolsover et al., 2003). This data treatment method successively examines the dataset, applying a measured analytical system detection limit (DL), to arrive at an estimate of the PF for a given type of RPE. It is a non-parametric approach, and makes no assumptions about the form of the measured distribution of PFs. However, as in most real workplace measurements, data came from situations where the many variables which could influence measurement accuracy were uncontrolled. Systematic study and validation of the data treatment method required generation of data under better defined and better controlled conditions in the laboratory. These validation studies are described in this paper.
What is under scrutiny here is the data treatment method in relation to measurement uncertainties, detection limits and challenge concentrations in general. The principles will apply to any measurement system used to measure the PF of any RPE. It is not specific to air-fed visors, or to the test agents and detection systems used here.
|
APPROACH USED FOR VALIDATION OF DATA TREATMENT |
|---|
Field measurements are known to suffer interference from a number of sources, e.g.:
- Variability in humidity of sampled air;
- Variability in concentration of other substances which may interfere with detection of the chosen challenge substance;
- Bias in sampling of in-facepiece and ambient concentrations, including sample probe position;
- Unpredictable and uncontrolled variation of the challenge concentration in both time and space.
The first two of these points are particularly true for in-facepiece samples where exhaled breath (moisture and CO2-laden) is intermittently present. To avoid such complicating factors, the complexity of the validation test system was reduced to a minimum.
Elimination of wearer and RPE
Measurements were made on a rigid engineering system, eliminating variability due to device and wearer effects altogether. A Wosthoff pump (Wosthoff GmbH, Bochum 1, Germany; a controllable precision dilution system), set to a fixed and invariant nominal dilution ratio of 1010, was substituted for the device/wearer system. This effectively simulated an RPE/wearer system with a nominal known, constant, PF of 1010. Moisture and CO2 effects noted above for real RPE users were also eliminated.
Test agent
For these studies, we used sulphur hexafluoride (SF6) gas as the test agent. This is an established test agent for RPE certification purposes, which can be reliably detected over a wide range of concentrations using either infra-red absorption or electron capture instrumentation. Variations in oxygen level are known to affect measurements, but as we have eliminated wearers from the system, this complication does not arise. (Unlike salt, SF6 is not absorbed by those breathing it, so this complication would not have arisen anyway.)
Challenge concentration
Using SF6 we were able to generate well-defined concentrations in a variety of ways:
- Direct sampling of certified calibration gas mixtures to give a small number of discrete constant challenge concentrations;
- Precision dilution of the certified calibration gas mixtures, to give intermediate discrete constant concentrations;
- Generation and gradual decay of a well-mixed challenge concentration to give a monotonically decreasing challenge.
30 to 5000 p.p.m. were used in these studies.
Data collection
Challenge concentrations were measured using a BINOS infra-red absorption analyzer (Rosemount Analytical, Orrville, OH USA). ‘In-facepiece’ concentrations (i.e. the diluted output of the Wosthoff pump) were measured using a Q200 (Uson Ltd, Bury St Edmonds, UK) electron capture detector (ECD). Analogue voltage outputs of both devices were logged using Squirrel 12-bit dataloggers (Grant Instruments, Cambridge, UK) at one second intervals, on either a ±2 or a ±20V channel. Ultimate resolution of the logged data was therefore 0.001 and 0.01V (4V range divided by 4096 bits, or 40V range divided by 4096 bits), respectively. Datasets for subsequent analysis typically contained a few thousand data pairs.
|
DETECTION LIMIT OF MEASUREMENTS |
|---|
Definition of DL
For the data analysis method under examination, it is essential to establish what the limit of detection of the measurement method is. Where challenge and ‘in-facepiece’ concentrations are measured by different methods, separate measurements of DL must be established.
The DL of an analytical method is commonly defined as three times the standard deviation of a blank measurement (Royal Society of Chemistry, 1987). In the context of our analysis, the standard deviation of blanks has been substituted by the standard deviation of measurements made at known constant challenges (e.g. of several hundred measurements of a known calibration gas concentration after a fixed dilution). Standard deviations for all available diluted constant challenges were combined to give an overall value for a given detection method/logging channel resolution, by calculating the square root of the sum of the individual variances. The DL was taken to be three times this overall standard deviation.
BINOS detection limit
The BINOS was used to measure the relatively large challenge concentrations, invariably on a ±20V squirrel channel. The overall DL for BINOS measurements was 0.1128 p.p.m. At worst (for the lowest challenge concentration used), this represents a possible error source of one part in 265, falling to one part in 44 000, (respectively, 30/0.1128 and 5000/0.1128) and is considered to be relatively unimportant.
ECD detection limit
ECD outputs were logged on either the ±20 or the ±2V squirrel channels, depending on the range of values being measured. Calculated detection limits are given in Table 1.
|
On the ±2V channel, the limiting factor is the stability of ECD measurements. On the ±20V channel, the limiting resolution of the logging system begins to influence the reliability of measurements.
|
TESTS CONDUCTED |
|---|
Three means of generating a variable challenge concentration were used. (In each case, actual measured values of the challenge concentration were used in the calculation of PFs, and not the nominal concentration. Challenge concentration measurements typically exhibited a standard deviation of a few percentage points.)
Calibration gas mixtures only
Certified gas mixtures of 5000, 500 and 50 p.p.m. SF6 in air were successively used as challenge inputs to the Wosthoff pump. Each mixture was sampled and data were logged for at least 15 min. Data from all three challenge concentrations were amalgamated for treatment as a single set.
Calibration gas with step dilution
A challenge concentration of 5000 p.p.m SF6 in air was fed to the Wosthoff pump through a further gas flow divider (Signal Instrument Co. Ltd, Camberley, Surrey, UK). The setting of the gas divider could be selected in steps of 10% from 10 to 100% of the input challenge. Resultant challenge concentrations of 5000, 4000, 3000, 2000, 1000 and 500 p.p.m were used. Each concentration was sampled for 4 min, and the whole set was amalgamated for treatment.
Continuous ramp
An initial challenge concentration of 1000 p.p.m SF6 in air was established in a well-mixed chamber, and was fed to the BINOS and through the Wosthoff pump to the ECD. Through recirculation of the chamber air with a small diluting air flow, the concentration of SF6 reduced continuously and monotonically over a period of 30 min.
Unlike the other test situations which were based on successive steady-state concentrations, both challenge and ‘in-facepiece’ concentrations varied continuously with time. Instrumental response time therefore became important in the calculation of PF. Passage of sample through the Wosthoff pump introduces a significant time lag between the measurement of a challenge concentration and the equivalent measurement of ‘in-facepiece’ concentration. In this set of tests, simultaneous samples of challenge and in-facepiece air took different lengths of time to reach their respective measurement and recording systems. This difference had to be corrected for in the subsequent calculation of PF, to avoid calculated values being systematically biased low. Chart records of the sensor outputs suggested a minimum delay of 60 s and a probable maximum of 72 s. These two values were used to correct recorded data for analysis, in effect synchronizing the challenge and in-facepiece detection systems.
|
DATA TREATMENT |
|---|
The process followed for each dataset was essentially the same. Data were transferred to Microsoft Excel spreadsheets using proprietary software. Recorded voltages were converted to p.p.m SF6 by application of calibration factors, established at the beginning of each test set. Any significant negative drift in zero over the period of the test was also corrected for at this stage. For ramp test data, a correction was made for difference in instrument response time.
Raw PF values were then calculated by dividing challenge concentration by the simultaneous ‘in-facepiece’ concentration. An example of this raw data is given in Fig. 1. Clearly, values of PF calculated from low challenge concentrations are highly variable, and reliability of PF value increases with increasing challenge concentration. The origin of most of this variability at low challenge concentrations lies in the similarity between measured ‘in-facepiece’ concentrations and the DL of the ECD measurement system.
|
Application of detection limits
A basic assumption of the data treatment method is that any recorded ‘in-facepiece’ data of less than the relevant calculated DL could in fact take any value up to the DL itself. Substituting such data points with the DL value introduces a worst possible (most pessimistic) case for calculation of PFs, and also clearly demonstrates that the reliability of PF calculated from low challenge concentrations is fundamentally limited by the challenge concentration itself. Raw data from Fig. 1 have been recalculated on this basis, assuming the relevant in-facepiece DL (0.156 p.p.m), and are shown in Fig. 2.
|
The initial straight line portion of the plot for challenge concentrations below
160 p.p.m represents the region where challenge concentration alone effectively controls the PF value. Figure 2 shows that inclusion of such data points in any overall calculation of assigned protection factor (APF)—the fifth percentile of the observed distribution of measured PF values; see BS 4275 (1997) for the full definition) would invariably bias the results very significantly to lower than truly representative values. This region also corresponds to the region of greatest variability for the raw data in Fig. 1. Data from higher challenge concentrations are essentially unaffected by this DL data substitution. Although not an important issue in these laboratory tests, any PF data derived from ‘in-facepiece’ concentrations above DL have to be considered valid, regardless of the challenge concentration. This has more significance for measurements made in real situations, where PF is unknown and almost certainly variable. In these situations, ‘in-facepiece’ concentrations above DL must be regarded as valid, irrespective of the challenge concentration. If the challenge concentration is also low, a valid low PF value will result.
For the purposes of this validation study, we must now assume that the PF of the system is unknown (rather than the fixed 1010 nominal dilution factor to which the Wosthoff pump was set). Can we now identify the challenge concentration below which measurements are unreliable? The following steps set out to establish this, in effect testing what is shown in Figs 1 and 2 in order to establish the point at which PF measurements become reliable.
Sorting of data
Until this point, data have been arranged in the order in which they were collected, as a time sequence of data pairs and derived PF measurements. As we are only trying to establish the observed PF distribution for a given type of RPE, the time sequence of measurements is irrelevant.
With the dataset entered into a spreadsheet, it is a simple matter to sort it according to values in any given column or columns. No assumptions are made about the form of the measured PF distribution. Sorting according to PF allows simple identification of salient points in the distribution of the data—for our purposes we have identified the 50th percentile (median) and 5th percentile (the value corresponding to the APF) values of the PF distributions. Sorting data first by ‘in-facepiece’ concentration and then by challenge concentration is required to simplify rejection of invalid data in subsequent steps.
Assumption of a protection factor
Values of PF must now be assumed which, in conjunction with the measured system DL, can be used as the basis for rejection of data. At this stage the assumed PF value has no significance; it is simply the starting point for the calculation process. For the datasets in this study, this process began with a low assumed PF value of typically around 200, and was repeated in successive equal increments. (In real WPF studies, where the PF of the RPE is completely unknown, it may be valid to begin with an assumed PF of 1 and increment from this value.) Assumed PF can in fact take any value; it does not have to follow any incremental or stepwise pattern. The approach chosen here was simply a balance between resolution in the calculated data and the processing effort required. Given time and resource, the whole data reduction process could be automated (e.g. by spreadsheet macro).
Having selected a PF value, the dataset is sorted first by ascending ‘in-facepiece’ concentration and second by ascending challenge concentration. Data pairs are eliminated where:
- ‘in-facepiece’ concentration is at the DL, and
- challenge concentrations are less than the product of the assumed PF and the DL.
Resulting PF distribution
The remaining dataset is then sorted according to PF value, and the 5 and 50% points of the ranked PF data distribution is determined by inspection.
Iteration
A higher value of assumed PF is selected, and the PF x DL product is calculated. Data is re-sorted according to ‘in-facepiece’ and challenge concentration, and the new PF x DL product is used to reject data pairs. The 5 and 50% values are then determined from the remaining dataset and sorted according to PF.
This process is repeated until insufficient data points remain (e.g. <200), all the ‘in-facepiece’ data at the DL are eliminated, or (for the datasets in this validation study at least) the derived 5 and 50% values stabilize.
Derivation of ‘real’ PF
Accumulated values of 5th and 50th percentiles are plotted against the assumed PF value used to derive them. By definition (see Fig. 2) residual datasets which include values derived from data pairs where ‘in-facepiece’ concentration is at the DL and challenge concentration is below the product of DL and the ‘real’ PF, will be biased low. Those derived only from challenge concentrations above this value will be unbiased. The point at which the transition from biased to unbiased values occurs can be identified by the intersection of the plotted data with a 1:1 line representing ‘Calculated PF = Assumed PF’, and should hold for any given percentile of the PF distribution. Figure 3 shows a schematic representation of such a graph based on a small hypothetical dataset, and how it relates to collected data.
|
Having established this transition point, it will be valid to accept measured values of PF calculated from challenge concentrations which exceed the product of DL and this derived ‘real’ PF value, as well as PF values derived from any of the data pairs with ‘in-facepiece’ concentrations above DL.
|
APPLICATION TO COLLECTED VALIDATION DATASETS |
|---|
Brief descriptions of the data resulting from the three laboratory test runs are given below. Derived values of the ‘real’ PF obtained by interpolation from the various 5 and 50% plots are given in Table 2.
|
Calibration gases only
Essentially there were only three challenge concentrations on which to base data points, giving a coarse step-function and low resolution to the derived PF data.
Calibration gas with dilution
Better resolution of the PF relationship, resulting from the increase in the number of challenge concentrations used, was apparent, particularly for data derived from low challenge concentrations. This dataset was obtained using the lower resolution (±20V) logger channel, which resulted in a broadening of the distribution of valid PF measurements.
Ramp data
Figure 4 shows the PF derivation plot resulting from the above data treatment to challenge concentration ramp data corrected for 72 s time lag. The principal difference between data for 60 and 72 s lag time lies in the absolute value of the calculated PF values in the ‘stabilized’ regions of the plots. This represents a small but systematic error caused by the difference between actual and assumed lag times in the original data. For the purposes of the data treatment validation, this effect is relatively unimportant. Both sets of data were recorded using the higher resolution (±2V) logger channel. The essentially continuous nature of the challenge concentration data also largely eliminates the ‘step-function’ appearance of the plots when compared with ‘calibration gas’ data, with and without dilution.
|
|
DISCUSSION OF VALIDATION TEST RESULTS |
|---|
The nominal PF of the simulated respiratory protective device in these studies was 1010, as determined by the setting on the Wosthoff gas dilution pump. As with all practical measurement systems, PFs calculated from pairs of challenge and ‘in-facepiece’ concentration samples are subject to sampling and analytical uncertainty. Both BINOS and ECD detectors have inherent measurement uncertainties; PF values derived from them will similarly be subject to a combined uncertainty. It is therefore to be expected that a distribution of PF values will be obtained from repeated measurements. This measured distribution would be expected to have a median close to the nominal 1010 dilution ratio of the Wosthoff, with a spread related to the combined standard deviation of the measurement systems.
The fifth percentile of the measured PF values should occur at a value of the mean minus 1.645 x the standard deviation, if the measured PFs are normally distributed. For the datasets considered here, in the data region corresponding to the known PF of the system, the corresponding predicted (log-normal parametric) values are shown in the final column of Table 2, for comparison with the measured (non-parametric) fifth percentile. It is clear that there is generally an excellent level of agreement between predicted and estimated values where data is of a more continuous nature.
The proposed technique for eliminating unreliable data and deriving the ‘real’ PF of the datasets produces estimates which are accurate to within 0.5%. The accuracy of this estimation is likely to be most reliable for large datasets with continuously variable challenge concentrations.
|
APPLICATION TO REAL WPF DATA |
|---|
There are two fundamental differences between these simplified instrumentation-generated studies, and real WPF measurements.
- Inclusion of wearers in the system will introduce significant opportunities for increasing the variability of measurement, the net result of which will be to elevate the DL.
- In practice, RPE/wearer systems do not exhibit nominally constant PF values; they change with time. There will be a continuum of in-facepiece values varying from zero to a finite maximum. The resulting PF values will therefore theoretically vary from infinite to a somewhat lower figure.
The suggested data reduction technique has been shown to work for systems with a nominally fixed PF performance. There is no reason why it should not work equally well for systems with variable PF performance, like practical RPE/wearer combinations in use in the field. The difference here is that the point at which data become valid is not apparent simply by examining a plot of PF versus challenge concentration like in Fig. 2. As noted above, the PF variability blurs this transition.
Application of this technique to the air-fed visor workplace data (Bolsover et al., 2003) yields the plot in Fig. 5. The estimated APF value for this large dataset (>8000 data pairs) is 41. It is worth noting that in this dataset in-facepiece measurements at DL are apparent throughout the observed range of challenge concentrations. As explained above, the distribution of PF is still constrained by substituting DL for low/zero ‘in-facepiece’ concentrations. This is likely to be a common finding for all practical RPE systems, particularly those providing high levels of protective performance.
|
|
PROTECTION FACTOR/CHALLENGE RELATIONSHIPS |
|---|
Sources of relationship
During studies of RPE performance, some researchers have noted an apparent relationship between measured PF and the challenge concentration (Co). For example, Howie et al. (1996)
reported a relationship between measured WPF for asbestos workers and the ambient fibre concentration. Potential explanations for this effect, some or all of which may apply, include:
- when challenge concentrations are known to be high, RPE wearers take more care to fit and use their RPE correctly;
- where work environments contain substances which are readily detected by the wearer, higher challenge concentrations effectively represent a continuous qualitative fit test of the RPE being worn, and consequently the quality of facepiece fitting is improved;
- only those data from high ambient concentrations are reliable, and the equipment consistently provides a good level of protection.
 | (1) |
This is a system with two degrees of freedom: two of the terms can take any (positive) value, but the third is defined by the above relationship. If any one of the terms is constrained by the experimental method or data treatment process, the system reduces to one degree of freedom. A linear relationship will be apparent between the two other terms.
In experiments to measure PF from Co and Ci, the three mechanisms described below can act to partially constrain the values of Co and Ci. In practice, the constraint on Co (usually a relatively large quantity) is unlikely to be significant. Collectively or separately, these mechanisms may give rise to a spurious apparent relationship.
Detection limit (noise floor)
All detection systems have a level below which any measurement is obscured by the background signal of the system itself. Examples include:
- the number of fibres counted on blank filters;
- weight changes of blank filters with time;
- for measurement of SF6 by electron capture or infra-red detector, high frequency noise on the voltage output, and zero/sensitivity drift with time.
|
The effects of assigning a fixed value to in-facepiece data which are within the noise level are apparent in Fig. 2. Similar effects are seen in Miller (2000
, Fig. 2a of that paper), where an arbitrary very small fibre concentration (0.00005 fibres ml–1) has been substituted for zero in-facepiece values to assist with the calculation (avoidance of infinite calculated PF values). This treatment imposes a constraint on Ci, generating a spurious correlation between PF and Co; the constrained data points plot on a straight line with a perfect positive correlation between PF and Co, and impart a weaker correlation to the entire dataset.
Quantization or rounding errors
In digital systems, or where analogue data is rounded to a fixed number of decimal places, the precision with which Ci is measured can result in quantized steps. At each step, the value of Ci is held constant and the resulting PF is proportional to Co. Examples include:
- the number of fibres counted on a filter;
- recording of a continuous analog voltage output using a digital datalogger.
Data presentation
Some forms of data presentation can appear to display a spurious relationship, particularly where the two variables concerned are not truly independent. Figure 7 demonstrates this effect for a set of entirely random pairs of variables (A and B), treated as though they were PF measurement data, and plotted on log-log axes. There appears to be a marked correlation in the data which is entirely spurious.
|
Hayes (1988)
and Oldham (1962) have identified this effect and have discussed in depth its lack of validity.
Importance of PF versus Co relationships
The mechanisms described can lead to apparent relationships between PF and Co. Only if these mechanisms have been addressed fully could any observed relationship be considered as potentially real. The significance of such a relationship is, however, secondary to other consequences of the noise floor and quantization effects noted above; they can result in incorrect measurements of PF. In practice, the presence of such an apparent relationship is likely to be a good indication that the dataset contains quantities of data which are unreliable, and which should be eliminated before drawing conclusions concerning the distribution of measured PFs.
|
CRITERIA FOR MEANINGFUL PF MEASUREMENT OF RPE |
|---|
Rationale
It is essential that PF studies are designed such that the Co and Ci values which will be encountered can be reliably measured by the instrumentation being used, so that reliable estimates of PF can be calculated. The following section sets out specifications for instrumentation and situations which are likely to provide reliable data on which to base estimates of the distribution of PF measurements. In all cases, values of Co and Ci are assumed to have been corrected for blank measurement.
These specifications are based partly on the findings of this validation study and the related laboratory work we have conducted to assess in-house instrumental measurement uncertainties, and partly from first principles. Pragmatic criteria have emerged which must be satisfied if meaningful PF measurements are to be made. Building such criteria into PF measurement strategies will ensure that resulting calculations do not suffer from avoidable bias arising from detection limit and uncertainty effects. This applies equally to both laboratory and field situations. Adhering to these specifications may not eliminate the need to apply the data reduction technique described earlier, but it should minimize the quantity of data which needs to be rejected on unreliability grounds.
There are two probable measurement conditions
Condition 1. Percentage uncertainty of challenge measurement < that in-facepiece.
This will normally be the situation (e.g. where the same sampling analysis technique is applied to both ambient and in-facepiece measurements, and PFs are appreciable).
- The uncertainty of the ‘in-facepiece’ concentration (Ui) must be determined, and expressed in the relevant units of concentration measurement (e.g. p.p.m, fibres/ml etc.).
- The expected PF which is intended to be measured must be chosen. For practical purposes this value will normally be taken to be the currently accepted APF for the RPE under investigation.
- The precision of the in-facepiece sample measurement system (Pi) must be known. This is equivalent to the smallest digit on the readout of a digital display, the smallest graduation on an instrument dial or the 1-bit steps on a digital logging system.
- The challenge concentration and the likely total uncertainty in measuring the ‘in-facepiece’ concentration should satisfy the condition:
Challenge concentration 
Cu, where:
 | (2) |
- The challenge concentration and the precision in measuring the ‘in-facepiece’ concentration should satisfy the condition:
Cp, where:
 | (3) |
- Both the uncertainty of the ‘in-facepiece’ concentration and the precision contribute to the overall uncertainty in the inward leakage measurement. If the above criteria give similar values, the critical challenge concentration (Cc) is given by:
 | (4) |
Condition 2. Percentage uncertainty of challenge measurement that in-facepiece.
This may occur if different sampling or detection systems, or different sensitivity ranges of the same instrumentation, are applied to in-facepiece and challenge measurements.
- The uncertainty of the challenge concentration (Uc) must be determined and expressed in the relevant units of concentration measurement (e.g. p.p.m, fibres/ml etc).
- The precision of the challenge sample measurement system (Pc) must be known. This is equivalent to the smallest digit on the readout of a digital display, the smallest graduation on an instrument dial or the 1-bit steps on a digital logging system.
- The challenge concentration and the likely total uncertainty in measuring the challenge concentration should satisfy the condition:
Cu, where:
 | (5) |
- The challenge concentration and the precision in measuring the challenge concentration should satisfy the condition:
Cp, where:
 | (6) |
- Both the uncertainty of the challenge concentration and the precision contribute to the overall uncertainty in the PF measurement. If the above criteria give similar values, the critical challenge concentration (Cc) is given by:
 | (7) |
In either case
- The response times of all instrumentation should be considered where real-time measurement systems are used. If the response time of the challenge measurement is different from that of the in-facepiece measurement, spurious PF values may be observed if appropriate correction is not made. No transient effect lasting less than the longest instrument response time can be reliably measured.
- If the values of Cc derived from equations (4) and (7) are similar (e.g. of the same order of magnitude), the two values should be added together to give the required challenge concentration needed to derive meaningful results in a WPF study.
- If the challenge concentration exceeds Cc, the measured PFs will be within ±10 % of the true value.
- If data from situations where any of the above criteria do not apply are included in the assessment of RPE performance:
- Large uncertainties may render the results meaningless or unreliable.
- Digitization errors may add to the overall uncertainty.
- Spurious relationships between measured PFs and the challenge concentration may be observed.
- Large uncertainties may render the results meaningless or unreliable.
Application to air-fed visor data
The workplace data on air-fed visor performance (Bolsover et al., 2003) were examined in accordance with the principles given above. In this study, the percentage uncertainty of the challenge concentration was very much less than that inside the facepiece. (Ui = 0.5 p.p.m and Pi = 0.01 p.p.m; the limiting factor on measurement uncertainty is Ui, and Pi can be ignored.)
As the criteria are being applied retrospectively, we have a choice of what value to apply for expected PF, and two possibilities suggest themselves. The nominal protection factor for this class of equipment (based on the maximum allowable inward leakage during certification tests) is 200, whereas the current APF for this class of device is 40. Applying these values to equations (2) to (4) gives a challenge concentration (Cu) below which calculated PF values are unlikely to be meaningful and should be excluded from analysis of the measured PF distribution. The dataset was examined on this basis, and the results are described in Table 3.
|
With a value of 200 for expected PF, the required challenge concentration for confident measurement is 1000 p.p.m. None of the challenge concentrations encountered in the field measurements were high enough to confidently measure a PF of 200.
Assuming a value of 40 for expected PF, only 11% of data (895 data pairs) exceeded the required challenge concentration of 200 p.p.m. This remaining data gave a fifth percentile (APF) value at 312. However, by definition we can only have confidence in measured PFs up to 40 from this set of data. We cannot attach any significance to this value of 312. All we can say with confidence is that the field data supports an APF value somewhere in excess of 40. Had the remaining data exhibited a fifth percentile below 40, we would be justified in accepting this lower value as valid.
This highlights what is likely to be a common limitation on RPE performance data measured in real situations. Particularly for high efficiency RPE, the required challenge concentrations for confident measurement of protective performance are unlikely to be encountered in normal working environments. The inevitable consequence is that performance of these devices simply cannot be assessed in real working situations, and laboratory-based work simulation using harmless surrogate test agents at high concentration is the only ethically viable alternative.
Application of these criteria to other published WPF data
Application of these principles to published WPF data may ultimately result in revision of the measured protection factor distributions and derived APF values. Some examples are given in Appendix 1. However, most published data do not contain sufficient information on which to base a retrospective assessment – details of precision and detection limits are frequently missing and cannot be determined. For clarity in all future studies of this nature, the parameters outlined above should be established and reported.
|
CONCLUSIONS AND RECOMMENDATIONS |
|---|
An inescapable consequence of the criteria developed here is that it will frequently be impractical to reliably measure PFs in working situations by relying on the levels of contaminant typically present in the ambient atmosphere and common measurement techniques; work simulation studies would be the only viable alternative in these situations.
The data rejection technique and method for estimating the unknown PF from a dataset has been shown to yield accurate values on a well controlled system of constant performance. It will be equally valid when applied to less well controlled situations, such as real WPF studies. Best results will be obtained from large datasets with continuously variable challenge concentrations and low detection limits.
The necessary data processing is relatively simple to carry out manually in a spreadsheet programme, and is amenable to automation through a spreadsheet macro. It is essential that large bodies of data are available, and continually logged simultaneous challenge and ‘in-facepiece’ concentration measurements are ideal for this. However, most previously reported WPF studies contain insufficient bodies of data to allow the PF estimation procedure used here to be applied to them.
The principle of rejecting data where ‘in-facepiece’ concentration is at the DL and the challenge concentration is less than the product DL x real PF is universally applicable to any WPF study. Johnston et al. (1992) similarly suggest that relationships of this nature must be built into WPF study protocols to ensure that meaningful data are obtained.
Pragmatic criteria have been established which should optimize collection of reliable data on which to base calculation of PF distributions. Adherence to these criteria should eliminate several sources of bias and spurious correlation. They should be incorporated at the design stage of all workplace performance studies on RPE, and included in study reports and publications. The criteria to be included in reporting of studies should include:
- The overall uncertainty of measurement of the in-facepiece and challenge concentrations;
- The precision of the ‘in-facepiece’ and challenge measurement systems;
- The PF which the study has been designed to measure;
- The response times of any ‘real-time’ instrumentation used to measure concentrations;
- The range of challenge concentrations encountered, and the lower limit considered to provide valid PF data;
- The number of valid PF data points.
|
APPENDIX 1: APPLICATION OF CRITERIA FOR MEANINGFUL PF MEASUREMENT TO A SMALL SELECTION OF PUBLISHED WPF DATA |
|---|
Application to powered respirator data
Data from Howie et al. (1996)
were considered in the light of these criteria. The difficulty here is that data are in the form of pairs of asbestos fibre concentrations, derived from fibre counts on individual membrane filters. Because there is no consistency in the volumes of air sampled onto the filters (e.g. in-facepiece sample volumes varying by more than a factor of 25), each individual measurement effectively has a different detection limit. While both fibre counts and calculated fibre concentrations are available for in-facepiece samples, only fibre concentrations are available for challenges. The quantity of data is also considerably lower than in the air-fed visor study; only 179 data pairs are available in total.
Howie et al. (1996) quotes an overall in-facepiece DL of 0.0014 fibres ml–1, based on average sampling times and a minimum acceptable fibre count of 11 fibres in 200 fields. Adopting this value as Ui, and applying the criteria at equations (2) to (4) above to this data yields the results in Table A1, which also shows the effect of adopting the usual minimum acceptable fibre count of 20 fibres in 200 fields (equivalent to 0.0025 fibre/ml).
|
While this dataset suffers from a lack of precision because of the relatively small number of data points, and probably from the assumption of an ‘average’ DL, some interesting features are apparent.
- Much of the raw data is derived from concentrations which are too low for the calculated protection factor to be considered valid, although almost all of the lowest PF values do not fall into this category.
- If a value of 40 is assumed for the expected PF, regardless of the value of Ui, the fifth percentile of the remaining data is 43. This is close to, but larger than, the expected APF, so cannot strictly be considered a valid estimate.
- Using the value of Ui assumed by Howie et al. (1996), and applying an expected PF of 200 to the data, still results in a fifth percentile for the remaining dataset of 43, but as this is lower than the assumed expected PF value, it must now be considered valid. Despite the relatively small number of data points available in the measured PF distribution, this value appears to be quite robust, and based on valid data.
- Applying an expected PF of 400 at the value of Ui assumed by Howie et al. (1996) gives a fifth percentile value of 117. As this is below the expected PF value, it must also be considered valid. So what is the ‘real’ APF for this dataset, 43 or 117?
- With an assumed value of 0.0025 for Ui, applying expected PF values of 100, 300 and 400 results in fifth percentile values which are respectively higher and hence invalid. However, the value for an expected PF of 200 is 196, and hence may be valid. Again, the relatively small number of data points available in the measured distribution may be a complicating factor here.
If we are to err on the side of caution, and we adopt the Howie et al. estimate of the DL, we must accept the lowest valid estimate of the APF, i.e. 43. It seems sensible to round this down to the currently accepted APF of 40 for this class of RPE.
Application to full face mask respirator data
Groves and Reynolds (2003) describe some preliminary field measurements of full face negative pressure respirator performance against ammonia, when used in indoor animal rearing environments. Values for the measurement uncertainty of in-facepiece samples can be derived from their data, and can be used to assess the validity of their WPF measurements. Table A2 summarizes the outcome of applying our criteria for valid measurement.
|
These authors recognize the limitations on their measurements posed by the similarity between in-facepiece values and the measurement uncertainty, stating that the calculated values can only be regarded as ‘semi-quantitative’. They also stress the importance of the sensitivity of the analytical system used when performing WPF studies. This assessment is in full agreement with the outcome of applying our criteria for reliable measurement to their data. None of their calculated WPF values lie in the range where we would have confidence in the results.
Application to the data of Myers 1984 and Myers and Zhuang 1998
Unusually in this study, an analytical technique with an extremely high level of precision has been used to measure the collected challenge and in-facepiece concentrations (PIXE). Applying our criteria to these datasets suggests that all the reported data are valid. Analytical constraints are not a limiting factor for this data, where both in-facepiece and challenge concentrations are both confidently measurable. None of the data pairs which result in low calculated PFs are affected by DL substitution or relatively low challenge concentration. The fifth percentiles for these datasets are shown in Table A3. Linear interpolation has been used to derive these values because of the relatively small numbers of datapoints available.
|
These results show the benefit, in terms of the proportion of usable data, of using extremely sensitive analytical techniques. However, PIXE analysis is also relatively expensive and requires access to uncommon and highly specialized equipment. It is impractical for general use.
Received November 19, 2004; in final form February 28, 2005
|
REFERENCES |
|---|
Bolsover J, Parker C, Vaughan N. (2003) Workplace protection factors of air-fed visors. HSL Internal Report PE/01/19.
BS 4275 (1997) Guide to implementing an effective respiratory device programme. London: British Standards Institute.
Groves WA, Reynolds SJ. (2003) Prototype sampling system for workplace protection factors for gases and vapours. Appl Occup Environ Hyg; 18: 394–402.[Medline]
Hayes RJ. (1988) Methods for assessing whether change depends on initial value. Stat Med; 7: 915–27.[Web of Science][Medline]
Howie RM, Johnston JBJ, Weston P et al. (1996) Workplace effectiveness of respiratory protective equipment for asbestos removal work. HSE Contract Research Report CRR 112/1996.
Johnston AR, Myers WR, Colton CE et al. (1992). Review of respirator performance testing in the workplace: Issues and concerns. Am Ind Hyg Assoc J; 53: 705–12.[Web of Science][Medline]
Miller BG. (2000) Investigation of factors affecting the performance of powered full facepiece respirators in use. HSE Contract Research Report CRR 282/2000.
Myers WR, Peach MJ, Cutright K et al. (1984) Workplace protection factor measurements on powered air-purifying respirators at a secondary lead smelter: Results and Discussion. Am Ind Hyg Assoc J; 45: 681–8.[Medline]
Myers WR, Zhuang Z. (1998) Field performance measurements of half-facepiece respirators: Steel mill operations. Am Ind Hyg Assoc J; 59: 789–95.[Medline]
Oldham PD. (1962) A note on the analysis of repeated measurements of the same subjects. J Chronic Dis; 15: 969.[CrossRef][Web of Science][Medline]
Royal Society of Chemistry (1987) Analytical Methods Committee. Analyst; 112: 199–204.[CrossRef]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  M. CLAYTON and N. VAUGHAN Fit for purpose? The role of fit testing in respiratory protection Ann. Hyg., October 1, 2005; 49(7): 545 - 548. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||