Annals of Occupational Hygiene Advance Access originally published online on June 5, 2008
Annals of Occupational Hygiene 2008 52(6):519-525; doi:10.1093/annhyg/men023
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
The Meaning of the Bias Uncertainty Measure
3904 Pocahontas Avenue, Cincinnati, OH 45227, USA
* Author to whom correspondence should be addressed. Tel: +1 513 652 4949; e-mail: dbartley{at}eos.net
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONCONTINUAL CALIBRATION WITH...INITIAL CALIBRATIONCONCLUSIONSAPPENDIX A. NON-CENTRAL STUDENT...ACKNOWLEDGEMENTSREFERENCES |
|---|
Characterization of measurement uncertainty in terms of root sums of squares of both unknown systematic as well as random error components is given meaning in the sense of prediction intervals. Both types of errors are commonly encountered with industrial hygiene air monitoring of hazardous substances. Two extreme types of measurement methods are presented for illustrating how confidence levels may be ascribed to prediction intervals defined by such uncertainty values. In the case of method calibration at each measurement, systematic error or bias may enter from a biased calibrant. At another extreme, a single initial method evaluation may leave residual bias owing to random error in the evaluation itself or to the use of a biased reference method. Analysis is simplified through new simple approximations to probabilistic limits (quantiles) on the magnitude of a non-central Student t-distributed random variable. Connection is established between traditional confidence limits, accuracy measures in the case of bias minimization and an uncertainty measure.
Keywords: bias • systematic error • uncertainty • accuracy • non-central Student's t
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONCONTINUAL CALIBRATION WITH...INITIAL CALIBRATIONCONCLUSIONSAPPENDIX A. NON-CENTRAL STUDENT...ACKNOWLEDGEMENTSREFERENCES |
|---|
‘Combined uncertainty’ uc expressed as the root sum of squares of independent ‘random uncertainty’ components (standard deviation estimates) has a simple meaning if systematic error is negligible. The result is an estimate of the standard deviation expected of the overall measurement variation. Then by defining an ‘expanded uncertainty’ U as a multiple k x uc (where k is a ‘coverage factor’), an unknown quantity M to be measured may be confidently bracketed in terms of an estimate m by intervals
 | (1) |
The more common situation encountered with workplace air monitoring concerns ‘systematic error’ or ‘bias’, unknown except for perhaps a magnitude estimate, and therefore not correctable. The ISO Guide to the Expression of Uncertainty in Measurement (Taylor and Kuyatt, 1994; ISO GUM, 1995) suggests maintaining equation (1) by pooling the bias uncertainty squared, after correction through calibration if necessary, along with the random components. The result, of course, no longer refers to a standard deviation in application measurements since the (unknown) bias does not fluctuate about zero at each measurement.
There are several ways of interpreting the ISO GUM approach. Since the proposed combined uncertainty includes and increases with bias magnitude, the quantity U can be regarded as simply an abstract rating of the measurement performance. Moreover, the uncertainty would relate to the standard deviation in an idealization as if the bias, though unknown, varied at each application measurement.
Prediction or tolerance intervals (see, e.g. Aichison and Dunsmore, 1975) provide an alternative interpretation. After correction, bias uncertainty is generally smaller than random components. [However, as an example to the contrary, estimates of aerosol concentrations according to a sampling convention (e.g. ISO 7708, 1995) may have large bias uncertainty if the aerosol size distribution is unknown (see, e.g. Bartley, 1998).] As will be shown here through two extreme examples, at a selected confidence level (e.g. 95%) regarding what is known about the method's bias, intervals can be calculated for bracketing true values at given frequency (e.g. for greater than 95% of the application measurements). If the bias magnitude is sufficiently small, the intervals are given in terms of an expanded uncertainty U by choosing an appropriate coverage factor k of uc comprised of the suggested root sum of squares of bias and random components. Conversely, with the traditional choice of k = 2, the coverage frequency can often be worked out.
First, a method is considered where calibration is done continually at each application using a standard calibration material. Measurement results may then be biased if the calibrant itself is biased. Were there no bias, this situation invokes usual confidence intervals using the Student t distribution, as touched on within ISO GUM.
With significant unknown bias, however, the ‘non-central’ Student t distribution is shown to give an expression of uncertainty. A new approximation to a symmetric quantile bracketing non-central Student t variables is developed. Over a significant range of unknown bias of small magnitude, use of the quantile as the expanded uncertainty U, with pooling of bias uncertainty and random error, is found valid for bracketing unknown measurand values at given confidence in the calibrant.
A limiting form of the non-central Student t quantile is found to equal the ‘symmetric accuracy range’ A. The function A is defined to cover method measurement results at specific frequency in terms of ‘unknown’ bias and variance of the random error. The range A has traditionally been used for many years in the capacity of rating method performance, including the case of correctable bias (Busch, 1977; Kennedy et al., 1995; NIOSH, 2003). However, here the interest is in maximally corrected bias leading to an expression of uncertainty, i.e. describing the unknown in a measurement result.
The use of A is illustrated for a method in which calibration is effected only initially by comparison to a reference method, followed by multiple subsequent applications. Residual bias may exist because of the finiteness of the calibration and also from bias of the reference method itself. In this case, the frequency of measurement coverage of measurand values does not depend on a running estimate of variation, but is specified entirely from initial calibration estimates. Again the non-central Student t quantile, in this case a confidence limit on the symmetric accuracy range A, specifies the expanded uncertainty U.
|
CONTINUAL CALIBRATION WITH BIASED CALIBRANT |
|---|
|
TOPABSTRACTINTRODUCTIONCONTINUAL CALIBRATION WITH...INITIAL CALIBRATIONCONCLUSIONSAPPENDIX A. NON-CENTRAL STUDENT...ACKNOWLEDGEMENTSREFERENCES |
|---|
Suppose a method is calibrated at each application by comparison through n measurements to a calibrant, plagued by unknown relative bias
, ‘small’ in magnitude in comparison to method imprecision. It will be shown here how to express confidence in the calibrant independent of how many method applications occur between changing calibrant. Suppose then that the estimate m in measuring unknown M may be represented as
 | (2) |
is a normally distributed random variable with variance 
2, which is estimated at each calibration and application by s2 with 
degrees of freedom. The quantity (the ‘true relative standard deviation’ sometimes denoted as TRSD) is expressed relative to M rather than a biased value for notational simplicity leading to parallel dependencies on and . Note that although the measurement variation depicted in equation (2) is very common in air quality measurements, at decreasing values of M, generally a constant variation (i.e. independent of M) becomes significant, leading to non-zero limits of quantitation and detection. (See, for example, DIS 15767, 2008.)
Now a random variable t' follows the non-central Student t distribution, peaking roughly at non-centrality parameter 
, if it is related to a variable u distributed normally about zero with unit variance and to an independent chi-square variable 
as
 | (3) |
 | (4) |
is defined (in terms of unknown quantities) as
 | (5) |
disappears from the numerator of equation (4). Suppose that is as likely positive as negative, in fact that E[] calibrant to calibrant is zero. Then, bounds of t' symmetric about zero are of interest. In particular, t'is bounded by the symmetric quantile sym
defined so that
 | (6) |
Using equations (4) and (5)
, equation (6) translates to a bound on m:
 | (7) |
The quantile symcan be calculated numerically. However, as shown in the Appendix A, this function is accurately approximated in terms of ‘central’ Student t quantiles and chi-square quantiles easily found for 
> 50% from statistical routines as
 | (8) |
| regime, which relates directly to the ISO GUM procedure. In fact, suppose
 | (9) |
2 is O[2/n] or smaller so that fluctuation of the order of 2/1/2 may be neglected in equation (9). Larger bias magnitude can be handled using a similar calculation as that following equation (A11), but leads away from the ISO GUM formalism.
Then equations (7) and (8) translate to:
 | (10) |
 | (11) |
We are now in a position to specify what is known about the bias. Though itself is not known, suppose that an accurate estimate of the variance 
of calibrant to calibrant is supplied by the calibrant vendor. Then, since E[] = 0,
 | (12) |
 | (13) |
Then (similar to the procedure of Smith, 1936; Welch, 1938; Satterthwaite, 1946), a confidence limit may be established on the running accuracy measure by approximating
 | (14) |
eff of degrees of freedom by setting the variance of the left-hand side of equation (14) equal to 2eff, the variance of the right-hand side, using the propagation of errors procedure, remembering that s2 is fixed here; the only variable is 2 in the denominator. Often the needed estimate of the variance of 2 (the kurtosis of ) may be approximated by the normal distribution value 
. In this case the effective number of degrees of freedom is given by
 | (15) |
The 95% confidence limit A95% is then given by
 | (16) |
 | (17) |
For 95% of the provided calibrants,
 | (18) |
 | (19) |
 | (20) |
. Thus, proportionality to the root sum of squares including the bias uncertainty is found to make sense.
Note that the usual confidence intervals involving the central Student t distribution are obtained in the limit that 
, with 
, with k simplifying to
 | (21) |
Furthermore, these simplified limits are also obtained with ‘non-zero’ for use in equation (20) for providing ‘mean’ (rather than 95%) confidence in the calibrant. This follows from the tacitly approximate equality of the mean of both sides of equation (14).
Symmetric accuracy range A[, ]
The tightest bracketing of unknown M in equation (7) may be found by letting 
, i.e. by effecting an impractical calibration. In this case, it is simple to see that sym in equation (8) approaches
 | (22) |
)/2 and 
are normal quantiles, e.g. 1.960 and 1.645, respectively, at = 95%. Correspondingly, the running symmetric accuracy range 
approaches:
 | (23) |
is given by
 | (24) |
The quantity defined in terms of unknown and is the symmetric accuracy range, which has been used for many years in judging method performance (Busch, 1977; Kennedy et al., 1995; NIOSH, 2003). The approximation of equation (24) was derived (Bartley, 2001) from a different point of view. [At the point of going to press, the author has been notified by Thomas Mathew, Department of Mathematics and Statistics, University of Maryland, of the discovery of the following exact expression for the symmetric accuracy range A in terms of the non-central chi-square quantile with one degree of freedom and with the above parameter (
)]:
 |
|
INITIAL CALIBRATION |
|---|
|
TOPABSTRACTINTRODUCTIONCONTINUAL CALIBRATION WITH...INITIAL CALIBRATIONCONCLUSIONSAPPENDIX A. NON-CENTRAL STUDENT...ACKNOWLEDGEMENTSREFERENCES |
|---|
Calibration-induced bias
Often it is inconvenient to recalibrate a measurement method at each application. For example, diffusive samplers may be evaluated initially by a vendor followed by many applications without re-evaluation (see ISO 16107, 2007 or ASTM D 6246, 2008). Furthermore, the spatiotemporal variations in air quality characteristics are generally so large (Vaughan et al., 1990; DIS 15767, 2008) as to preclude evaluation of a method during application through the use of replicate measurements. In these cases, often an initial single-method evaluation is undertaken with the purpose of determining uncertainty present in subsequent applications of the method.
Details are presented here on determining or setting a limit of confidence in an initial method calibration with two independent sources of bias. Suppose a method is initially calibrated by a finite comparison to a possibly biased but otherwise accurate reference-method measurement mref of an unknown constant Mcal
 | (25) |
ref « 1. During calibration, the method, biased by unknown raw, yields a number n of uncorrected (raw) estimates mraw so that
 | (26) |
The random variable 
raw is normally distributed about 0 with unknown variance 
, but is estimated by 
(with n – 1 degrees of freedom). Therefore, an estimate for raw is given in terms of the n-sample mean 
by
 | (27) |
ref (but not in raw, which is not necessarily small relative to 1.0). Suppose that all that is known about the reference method bias ref is that its magnitude is bounded by max and that ref may be taken as uniformly distributed (calibration-to-calibration) between its limits (as suggested in ISO GUM, 1995). Then, the expected value of 
, the (intercalibration) variance 
of ref, is given by
 | (28) |
Following an initial calibration suppose the method is applied without recalibration, but with bias partially eliminated by correcting raw measurements using a factor 
equal to 
. Then in measuring unknown M, the corrected measurement value m is given by
 | (29) |
and true relative standard deviation are given by
 | (30) |
 | (31) |
Then, the small-bias limit (equation (24)) for the symmetric accuracy range A is given by
 | (32) |
It should be remembered that the two independent quantities, ref and n, refer to the actual, though unknown biases. What is known is that the latter is normally distributed about zero with variance 2/n and that the former is bounded leading to equation (28). Equations (28) and (32) indicate that A may be estimated from
 | (33) |
 | (34) |
The effective number eff of degrees of freedom is calculated via propagation of errors by requiring that the variances of both sides of equation 34 agree, noting that both numerator and denominator (requiring estimates of the kurtoses of ref and n) are variable in this case. The result is
 | (35) |
Finally, since equation (18) holds and may be approximated as equation (19), the expanded uncertainty U is again given with the sense of equation (1) by A95% as
 | (36) |
 | (37) |
|
CONCLUSIONS |
|---|
|
TOPABSTRACTINTRODUCTIONCONTINUAL CALIBRATION WITH...INITIAL CALIBRATIONCONCLUSIONSAPPENDIX A. NON-CENTRAL STUDENT...ACKNOWLEDGEMENTSREFERENCES |
|---|
Applying properties of symmetric quantiles of the non-central Student t distribution, uncertainty characterization according to ISO GUM is found to be applicable to a wide range of measurement situations. In the limit of zero bias, of course, commonly applied confidence limits are obtained. With residual bias remaining after method correction, different limits of the quantiles generally cover measurement procedures, ranging from continual to simply initial characterization as to uncertainty.
In the cases of finite yet controlled bias, procedures for analyzing uncertainty are understandable. Treating bias uncertainty as equivalent to random error does make sense. The concept of prediction lends meaning to the covering intervals obtained. Extension to the situation of large unknown bias results in an abstract method rating. Alternatively, ISO GUM methods can be extended in this case by applying linear aspects of the symmetric non-central Student t quantiles.
Within entirely diverse measurement situations, the ISO GUM approach to uncertainty results in a simple meaningful analysis into root sums of squared components, whether random or systematic. The recommended procedures are therefore both extremely simple to apply and also can be ascribed specific meaning when needed. Great foresight clearly went into the original development of these procedures.
|
APPENDIX A. NON-CENTRAL STUDENT t QUANTILE APPROXIMATIONS |
|---|
|
TOPABSTRACTINTRODUCTIONCONTINUAL CALIBRATION WITH...INITIAL CALIBRATIONCONCLUSIONSAPPENDIX A. NON-CENTRAL STUDENT...ACKNOWLEDGEMENTSREFERENCES |
|---|
Single-sided quantile
The non-central Student t quantile can be approximated meaningfully and quite accurately from treating the Student t variable as closely normally distributed over either tail of its distribution. First of all, suppose that f is a sum of normally distributed independent components
, each with vanishing expected value, but with variance 
 | (A1) |
Then, the variance of f is given by
 | (A2) |
at probability > 50% is given in term of the component quantiles 
as the root sum of squares value
 | (A3) |
Now the non-central Student t variable 
is defined in terms of non-centrality parameter and number of degrees of freedom by
 | (A4) |
is an independent chi-square variable. Since a central Student t variable t is defined as
 | (A5) |
Equation (A4) can be expressed as
 | (A6) |
The two terms on the right-hand side of equation (A6) are uncorrelated, and the expected value of the first is zero and of the second is zero in approximation. Then, equation (A3) indicates that the quantile at > 0 may be calculated as
 | (A7) |
This expression (for > 50%) gives exact results in the limit 
0, also in the limit 
at fixed , as well as in the limit at fixed
 | (A8) |
 | (A9) |
),
 | (A10) |
That the limits and are not commutative reflects the diverse characters of the non-central Student t distribution ranging from limiting forms, chi-square to central Student t. The quantile for < 0 can be found similarly or directly from
 | (A11) |
= 0). Also, quantiles for < 50% take the negative root in equation (A7).
With the approximation pinned down at these limits, it is no wonder that the approximation of equation (A7) is globally accurate. For example, Fig. A1 shows a comparison of the approximation of equation (A7) to the exact quantile at = 5 over the region of maximum discrepancy.
|
Equation (A7) is more accurate than the approximations described in the literature (see, e.g. Johnson et al., 1995). Of course, equation (A7) requires a lookup for the central Student t and chi-square quantiles at given number of degrees of freedom, rather than simply the normal distribution quantile.
Symmetric quantile
Small || region.
Approximation to the symmetric quantile of the non-central Student t distribution is derived here in the ||
1 limit. The asymptotic limit ( » 1) can be derived by expanding equation (A7). However, as the result is neither as simple nor as accurate as the exact result, a direct approach is taken here.
The symmetric quantile symt[,]is defined so that
 | (A12) |
With t' defined by equation (A4), equation (A12) can be expressed explicitly as
 | (A13) |
Differentiating both sides of equation (A13) with respect to , we have
 | (A14) |
 | (A15) |
Now symt[,] is expanded in to O[2] as
 | (A16) |
(+1)/2 is the central quantile, and the constant b2 is to be determined. Equation (A16) implies that, ignoring terms of O[2],
 | (A17) |
Thus, to lowest order in , equation (A14) implies
 | (A18) |
 | (A19) |
2]
 | (A20) |
Figure A2 shows the approximation of equation (A20) in comparison to an exact result obtained numerically.
|
Arbitrary |
|.As the magnitude |
| of the non-centrality parameter increases beyond unity, the symmetric and single-sided quantiles rapidly approach each other. Therefore, equations (A7) and (A20) result in the following, generally insignificantly discontinuous, approximation for > 1/2.
 | (A21) |
Equation (A21) is illustrated in Fig. A3.
|
|
ACKNOWLEDGEMENTS |
|---|
|
TOPABSTRACTINTRODUCTIONCONTINUAL CALIBRATION WITH...INITIAL CALIBRATIONCONCLUSIONSAPPENDIX A. NON-CENTRAL STUDENT...ACKNOWLEDGEMENTSREFERENCES |
|---|
I would like to thank Stanley Shulman and Göran Lidén for introducing me, years ago, to the concepts of the non-central Student t distribution and tolerance intervals; Harry Rook, for grasping the significance of uncertainty and Paul Schlecht, for often intensely delving into my interests.
Received February 24, 2008; in final form April 14, 2008
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONCONTINUAL CALIBRATION WITH...INITIAL CALIBRATIONCONCLUSIONSAPPENDIX A. NON-CENTRAL STUDENT...ACKNOWLEDGEMENTSREFERENCES |
|---|
Aichison J, Dunsmore IR. Statistical prediction analysis (1975) Cambridge, England: Cambridge University Press.
ASTM D 6246. Standard practice for evaluating the performance of diffusive samplers (2008) West Conshohocken, PA: ASTM International.
Bartley DL. Inhalable aerosol samplers. Appl Occup Environ Hyg (1998) 13:274–78.
Bartley DL. Definition and assessment of sampling and analytical accuracy. Ann occup Hyg (2001) 45:357–364.
Busch KA. Standards completion program statistical protocol. In: Documentation of the NIOSH validation tests—Taylor DG, Kupel RE, Bryant JM, eds. (1977) Cincinnati, OH: U.S. Department of Health, Education, and Welfare (DHEW), Centers for Disease Control (CDC), National Institute for Occupational Safety and Health (NIOSH). [DHEW (NIOSH) Publ. No. 77–185]. p. 7–12.
ISO GUM. Guide to the expression of uncertainty in measurement, ISO Guide 98 (1995) Geneva: International Organization for Standardization.
ISO 7708. Air quality—particle size fraction definitions for health-related sampling (1995) Geneva: International Organization for Standardization.
DIS 15767. Workplace atmospheres—controlling and characterizing errors in weighing collected aerosol (2008) Geneva: International Organization for Standardization.
ISO 16107. Workplace atmospheres—protocol for evaluating the performance of diffusive samplers (2007) Geneva: International Organization for Standardization.
Johnson NL, Kotz S, Balakrishnan N. Continuous univariate distributions (1995) 2, 2nd. New York: John Wiley and Sons, Inc.
Kennedy ER, Fischbach TJ, Song R, et al. Guidelines for air sampling and analytical method development and evaluation (1995) Cincinnati, OH: U.S. Department of Health and Human Services (DHHS), CDC/NIOSH. [DHHS Publ. No. 95–117.].
NIOSH. Measurement uncertainty and NIOSH method accuracy range, Chapter P. (2003) NIOSH manual of analytical methods. Schlecht PC, and O'Connor PF, 4th edn. (Suppl 3), DHHS (NIOSH) Publication No. 2003-154, and references within. Washington, DC: US Government Printing Office. p. 208–228.
Satterthwaite FE. An approximate distribution of estimates of variance components. Biometrics (1946) 2:110.[CrossRef][Web of Science]
Smith HF. The problem of comparing the results of two experiments with unequal errors. J Council Sci Ind Res (1936) 9:211.
Taylor BN, Kuyatt CE. Guidelines for evaluating and expressing the uncertainty of NIST measurement results (1994) NIST Technical Note 1297. Washington, DC: US Government Printing Office.
Vaughan NP, Chalmers CP, Botham RA. Field comparison of personal samplers for inhalable dust. Ann occup Hyg (1990) 34:553.
Welch BL. (1938) Biometrika; 29: 350–62.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  D. Bartley and G. Liden Measurement Uncertainty Ann. Hyg., August 1, 2008; 52(6): 413 - 417. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||