Ann. occup. Hyg., Vol. 47, No. 1, pp. 31-36, 2003
© 2003 British Occupational Hygiene Society
Published by Oxford University Press
Evaluation of Five Extraction Protocols for Quantification of Endotoxin in Metalworking Fluid Aerosol
College of Public Health, Department of Occupational and Environmental Health, 100 Oakdale Campus, IREH, The University of Iowa, Iowa City, IA 52242-5000, USA
Received 16 July 2002; in final form 18 September 2002
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Objectives: Occupational exposures to endotoxin-contaminated, water-based metalworking fluids (MWFs) are thought to contribute to cases of respiratory illness. Before occupational exposure limits for endotoxin can be proposed, accuracy and reproducibility of laboratory measurements must be established. The method most commonly used to quantify endotoxin is the Limulus amebocyte lysate (LAL) assay and this is the basis for the American Society for Testing and Materials (ASTM) method E2144-01. This study was conducted to generate multiple samples with similar mass and endotoxin loading in order to compare four alternative extraction methods with the ASTM method. Methods: Using an exposure chamber system that provides a uniform distribution of MWF mist, aerosols with three concentrations of endotoxins (4.5, 350 and 1141 EU/m3) were collected simultaneously on multiple filter samples. The filters were examined for endotoxin concentration using five different extraction protocols: extraction with 1 h shaking at 25°C in 30 ml pyrogen-free water (PFW) (protocol 1) or in PFW with 0.05% Tween-20 (protocol 2); or shaking at 68°C in 30 ml PFW (protocol 3) or PFW with Tween-20 (protocol 4); or extraction into 20 ml PFW with sonication at 25°C and pH adjustment to 7.5 (ASTM protocol). Results: The uniformity of the aerosol mass yielded coefficients of variation of 12.7, 7.7 and 1.4% for the low, medium and high exposure groups, respectively. The variance in the endotoxin extraction protocols was highest for the ASTM method for the low, medium and high concentration trials. Low, medium and high endotoxin groups were statistically different (P < 0.001), but there were no statistical differences between extraction protocols within these exposure levels. Conclusions: ASTM method E2144-01 yielded comparable estimations of MWF endotoxin aerosol concentrations but with higher variability than the four other extraction methods. This study shows that extraction into PFW at 25°C with or without Tween-20 was an improvement over the ASTM method in that the estimation was more precise and the method is simpler.
Keywords: endotoxin; exposure assessment; Limulus amebocyte lysate assay; ASTM; lipopolysaccharide; machining fluids; metalworking fluids; metal removal fluids
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
The National Institute of Occupational Safety and Health (NIOSH) has recognized the association of exposure to metalworking fluids (MWFs) with occupational lung disease by recommending an exposure limit (REL) of 0.4 mg/m3 (NIOSH, 1998a,b). There are four categories of MWFs (neat oil, soluble oil, semi-synthetic or synthetic), each with unique characteristics that determine its use within a wide variety of manufacturing industries (Sprince et al., 1994). Among the contaminants found in water-soluble MWFs, products of microbial growth are thought to be related to a number of occupational lung diseases (Bernstein et al., 1995; Woskie et al., 1996; Robins et al., 1997; Sprince et al., 1997; Fox et al., 1999; Laitinen et al., 1999; Abrams et al., 2000; Thorne, 2000; Hodgson et al., 2001). Endotoxin, a component of the cell membrane of Gram-negative bacteria, has been shown to produce an inflammatory response in human and animal subjects (Castellan et al., 1987; Gordon, 1992; Thorne and DeKoster, 1996; Rylander, 1997; Thorne, 2000).
Exposure limits have not yet been adopted for the endotoxin component of MWF exposure, in part due to the inter- and intra-laboratory variability of endotoxin measurements. However, in answer to the pressing need for a standardized protocol for the collection and analysis of endotoxin from the workplace air, the American Society for Testing and Materials (ASTM) recently accepted the recommendation of subcommittee E-34.50 on Health and Safety Standards for Metal Working Fluids. Standard E2144-01 was adopted in 2001 and is a standard method for the collection and analysis of airborne endotoxin in MWF aerosol using the Limulus amebocyte lysate (LAL) assay (ASTM, 2001). This is the first report comparing the ASTM protocol to methods used in previous studies examining endotoxin in MWFs (Gordon, 1992; Gordon et al., 1992; Woskie et al., 1996; Sprince et al., 1997; Brown et al., 2000).
The extraction protocols used in this study were chosen to systematically examine the efficacy of the solutes, the extraction temperature and the vigor of agitation (Olenchock et al., 1989; Douwes et al., 1995; Thorne et al., 1997) on endotoxin recovery from a set of matched MWF aerosol samples.
|
MATERIALS AND METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Aerosol
Test aerosols of soluble oil MWF (Cimperial HD-90, Milacron, Cincinnati, OH) were generated using a Collison six-jet nebulizer (CH Technologies, Inc., Westwood, NJ) in an exposure chamber with a volume of 1 m3 (Thorne, 2000). The chamber was supplied with HEPA-filtered outdoor air that was maintained at 22 ± 2°C and controlled to ±5% relative humidity. Three test atmospheres were generated to create a wide range of endotoxin concentrations; low (4.5 EU/m3), medium (350 EU/m3) and high (1141 EU/m3) (specified as overall geometric means). A bulk sample of in-use MWF collected from an automotive engine plant was used to create the medium and high endotoxin exposure groups by generating MWF aerosols at two gravimetric concentrations (31.8 and 60.8 mg/m3). To create the low endotoxin atmosphere, unused MWF of the same type was diluted to the in-use concentration (5%) and aerosolized at 19.95 mg/m3.
A manifold with 12 sampling ports was placed inside the exposure chamber and attached to a vacuum line. The flow through each sampling port was calibrated to a flow rate of 1.97 ± 0.01 l/min. Filter holders were placed in the sample ports and fitted with 37 mm glassfiber filters (EPM-2000; Whatman Int., Ann Arbor, MI). Each test atmosphere was generated for 
60 min for three consecutive sample runs. Two filters from each sample run were randomly assigned to an extraction protocol, which resulted in a total of six filters per protocol per exposure group. Fifteen unexposed filters were used as experimental blanks. Filters were conditioned in a temperature- and relative humidity-controlled weighing room prior to taking the pre-weight and again after sampling for the post-weight gravimetric measurements (MT5 Microbalance; Mettler-Toledo, Inc., Columbus, OH). The limit of detection of the gravimetric measurements was 0.006 mg.
Endotoxin assays
All glassware was depyrogenated at 200°C overnight prior to use, and all plasticware was pyrogen-free. The five extraction protocols are listed in Table 1. The extraction solute used was either pyrogen-free water (PFW; BioWhittaker, Inc., Walkersville, MD) (protocols 1, 3 and 5) or PFW plus 0.05% Tween-20 (Sigma, St Louis, MO) (protocols 2 and 4). Filters for protocol 5 were sonicated at 210 W for 60 min using a bath-type ultrasonic cleaner (Mettler-Toledo). Pyrogen-free hydrochloric acid 1.0 N or sodium hydroxide 1.0 N (Sigma, St Louis, MO) was used to adjust the pH of the extract (protocol 5) to 7.5.
|
Filters were weighed and placed into sterile 50 ml centrifuge tubes (Corning, Inc., Acton, MA). The extracts processed using protocol 5 were adjusted from about pH 9.0 to pH 7.5 prior to endotoxin analysis. After the agitation step (shaking or sonication) the tubes were centrifuged for 10 min at 1000 g. Samples were assayed for endotoxin on the same day they were extracted.
Endotoxin concentration was determined by a kinetic, chromogenic Limulus amebocyte lysate (LAL) assay (Kinetic-QCL; BioWhittaker, Inc.) as previously described (Thorne, 2000). All reagents used for the endotoxin analysis were from the same lot number. For each assay, standard curves were generated over the concentration range 0.049–100 EU/ml using a reference endotoxin (Escherichia coli O55:B5; BioWhittaker, Inc.). Aerosol samples that were extracted in PFW–0.05% Tween-20 were assayed using an endotoxin standard diluted in PFW–0.05% Tween-20. Likewise, aerosol samples extracted in PFW were assayed using an endotoxin standard diluted in PFW. Aliquots (100 µl) of endotoxin standards, dilutions of MWF aerosol extracts or PFW blanks were pipetted in duplicate into the wells of pyrogen-free microtiter plates (Costar no. 3596; Corning, Inc.). The samples were preincubated for 15 min at 37°C after which time 100 µl of LAL reagent was added to each well with an eight-channel micropipettor. The assay plate was placed in a SpectraMax microplate reader (Molecular Devices, Sunnyville, CA), agitated to mix the lysate and sample, and the assay was allowed to proceed at 37°C for 1.5 h. Spectrophotometric measurements at 405 nm were taken every 30 s. Data were analyzed using SoftMaxPro software (Molecular Devices). Maximum reaction rate values for the standards were fitted to a four-parameter curve, and sample concentrations computed. The minimum acceptable r2 value of the standard curves was 0.998.
Data analysis
Data were analyzed using SPSS version 10.0 (SPSS, Inc., Chicago, IL) and were examined for normal distribution by computing Lillefor’s probabilities for Kolmogorov–Smirnov one-sample tests against a normal distribution. Because MWF concentrations were so tightly controlled within the three exposure groups, the gravimetric data were normally distributed. Endotoxin data approximated log normality and were transformed to the natural logarithm for parametric analysis.
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
In order to generate sufficient numbers of filter samples to test extraction procedures, each test atmosphere was generated on three occasions for each exposure group (10 filters used from each of 3 runs = 30 filters per exposure group). The means, confidence intervals and coefficients of variation (CV) of the gravimetric measurements for the three MWF aerosol exposure groups are presented in Table 2. The highest CV (12.7%) was observed for the filters in the lowest exposure group. As expected, the mean values of the three exposure groups were significantly different (P < 0.001) and the uniformity increased with increasing exposure concentrations.
|
During the course of the experiments, 15 blank filters were collected and three were randomly assigned to each extraction protocol and analyzed similarly to the MWF aerosol filters. The net weight change of all experimental blank filters was below the limit of detection for the gravimetric measurement (<0.006 mg). As shown in Table 3, the ASTM method was the only extraction protocol for which all three blanks were detectable and the mean concentration of endotoxin in the filter eluate was 10- to 50-fold higher than for extraction protocols 1–4. This difference may arise from the process of measuring and adjusting the pH of the solution prior to analysis, a procedure that is unique to the ASTM protocol.
|
Results of the endotoxin analysis of the MWF aerosol for the five extraction protocols are presented in Table 4 along with the gravimetric aerosol concentrations. For the low, medium and high exposure groups, the arithmetic mean particulate concentrations are provided for the sets of six filters submitted for analysis using the five extraction protocols. The geometric mean endotoxin concentrations determined by these protocols and their geometric standard deviations are also shown. As intended, endotoxin concentrations in the low exposure group were detectable but extremely low, and concentrations in the high group were at the upper end of what is normally observed in machining plants. There were no significant differences in endotoxin recovery between the five extraction protocols. The endotoxin measurements from all extraction protocols were highly correlated, with all bivariate comparisons yielding r values exceeding 0.8 (P < 0.001).
|
The coefficients of variation were calculated for each extraction protocol and are shown in Fig. 1. The protocol with the highest within-group variance was protocol 5, the ASTM method. The mean coefficient of variation for the ASTM protocol was 122%, while the mean coefficient of variation ranged from 35 to 57% for the other four protocols.
|
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
The present study demonstrates that aerosols of soluble oil MWFs collected onto air-sampling filters may be extracted for endotoxin analysis using different protocols with equal efficiency. Brown et al. (2000) examined endotoxin assay of bulk liquid machining fluids using PFW or PFW–0.05% Tween-20 with or without sonication. In the bulk soluble oil MWFs there were no significant differences in endotoxin recovery by extraction protocol. However, contrasting differences in endotoxin recovery for semi-synthetic and synthetic MWF were found. In the bulk fluids, extraction with PFW–Tween-20 with sonication returned the highest endotoxin recovery for the semi-synthetic MWF, while PFW–Tween-20 without sonication was significantly higher for the synthetic MWF samples. The geometric standard deviations for endotoxin concentration in the bulk MWFs in that study were extremely high (range 5.6–38, median 16) compared to those for the MWF aerosol samples in the present study (range 1.3–6.0, median 1.7).
The endotoxin extraction protocol specified in ASTM E2144-01 is the same as protocol 5 examined in this study. Although none of the extraction protocols were significantly different for endotoxin recovery over the range of exposures examined in this study, protocol 5 had the highest detectable endotoxin in the filter blanks and the largest confidence intervals of the extraction protocols tested with each exposure group. This may be due to the inclusion of pH adjustment in the protocol or the use of sonication.
The challenge faced in this, and any study reporting endotoxin concentrations is the inherent variability within the assay itself, particularly for occupational and environmental samples. Direct comparisons between studies reporting endotoxin concentrations are problematic due to the numerous protocols used within the research environment. For example, the recovery of endotoxin from aerosol samples is highly dependent on the filter medium and the matrix (Gordon et al., 1992; Hollander et al., 1993; Douwes et al., 1995; Thorne et al., 1997; Sarantila et al., 1999). Studies have examined the effect of the detergents such as Tween-20 and have shown both increased (Douwes et al., 1995; Brown et al., 2000) and decreased (Olenchock et al., 1989) efficiencies of endotoxin recovery. The vigor with which the extraction is conducted has been shown to increase endotoxin recovery (Jay and Margitic, 1979; Brown et al., 2000). All commercially available LAL tests are required to meet regulated performance standards against reference standard endotoxin (RSE). However, different forms of the lysate have been shown to be more efficient for high volumes of samples with greater precision, e.g. the kinetic chromogenic assay compared to the classic gel-clot assay (Górny et al., 1999). In order to monitor potential inhibition or enhancement of endotoxin, the rate of the reaction in kinetic assays may be compared directly with the rate for the standards (Hollander et al., 1993; Walters et al., 1994; Milton et al., 1997; Thorne et al., 1997).
The present study was initiated to contribute to the efforts being made to find a standard protocol that may be used to reliably measure exposures to endotoxin in occupational environments. Here, the filter medium, the lot number of the lysate and the LAL assay protocol remained constant, while the effects of extraction solutes, temperature of extraction, use of pH adjustment and vigor of extraction were varied. Under these treatments, the apparent endotoxin concentrations for each exposure group varied from 2- to 10-fold between extraction protocols, with differences most pronounced in the low exposure group. Similar ranges of concentrations have been found in controlled inter- and intra-laboratory studies on animal barn or cotton dust particulate on filters (Thorne et al., 1997; Chun et al., 2000). If we consider only extraction protocols 1–4, the apparent endotoxin concentration for the medium and high concentrations varied 2-fold.
Only soluble oil MWF aerosol was examined in this study, and more work is required to determine if semi-synthetic or synthetic MWF aerosols would behave differently. The use of an exposure chamber is recommended for the generation of multiple filters for testing, as control of the aerosol loading was demonstrated by the low geometric standard deviations of the endotoxin measurements for each extraction protocol.
In conclusion, a range of endotoxin concentrations was extracted from filters exposed to test atmospheres of soluble oil MWF aerosol. Although the aerosol loading between filters from similar exposure conditions was well controlled, extracted endotoxin concentrations varied within exposure groups by 2- to 4-fold for medium and high exposure groups and over 10-fold for the low exposure groups.
Recommendations
Airborne endotoxin in MWF may be measured using currently available technology and protocols. Use of higher extraction temperature or inclusion of Tween-20 did not demonstrate a significant effect. However, using the ASTM Standard E2144-01 protocol produced less precise exposure estimates and higher determinations in filter blanks, and thus is not recommended.
Acknowledgements—The authors acknowledge UAW-DaimlerChysler National Training Center, NIEHS P30 ES05605, and NIEHS T32 ES07314 for supporting this project. K.H.B. was the recipient of a postdoctoral fellowship from the Canadian Institutes of Health Research.
|
FOOTNOTES |
|---|
* Author to whom correspondence should be addressed. Tel: +1-319-335-4216; fax: +1-319-335-4006; e-mail: peter-thorne{at}uiowa.edu
Current address: School of Occupational and Environmental Hygiene, University of British Columbia, Vancouver, Canada
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Abrams L, Seixas N, Robins T et al. (2000) Characterization of metalworking fluid exposures indices for a study of acute respiratory effects. Appl Occup Environ Hyg; 15: 492–502.[CrossRef][Medline]
ASTM. (2001) Standard practice for personal sampling and analysis of endotoxin in metalworking fluid aerosols in workplace atmospheres. Standard E-2144-01. West Conshohocken, PA: American Society for Testing and Materials.
Bernstein DI, Lummus ZL, Santilli G et al. (1995) Machine operator’s lung: a hypersensitivity pneumonitis disorder associated with exposure to metalworking fluid aerosols. Chest; 108: 636–41.
Brown ME, White EM, Feng, A. (2000) Effects of various treatments on the quantitative recovery of endotoxin from water-soluble metalworking fluids. Am Ind Hyg Assoc J; 61: 517–20.
Castellan RM, Olenchock SA, Kinsley KB et al. (1987) Inhaled endotoxin and decreased spirometric values. N Engl J Med; 317: 605–10.[Abstract]
Chun DTW, Chew V, Bartlett K et al. (2000) Preliminary report on the results of the second phase of a round-robin endotoxin study using cotton dust. Appl Occup Environ Hyg; 15: 152–7.[CrossRef][Medline]
Douwes J, Versloot P, Hollander A et al. (1995) Influence of various dust sampling and extraction methods on the measurement of airborne endotoxin. Appl Environ Microbiol; 61: 1763–9.[Abstract]
Fox J, Anderson H, Moen T et al. (1999) Metal working fluid-associated hypersensitivity pneumonitis: an outbreak investigation and case-control study. Am J Ind Med; 35: 58–67.[CrossRef][Web of Science][Medline]
Gordon T. (1992) Acute respiratory effects of endotoxin-contaminated machining fluid aerosols in guinea pigs. Fundam Appl Toxicol; 19: 117–23.[CrossRef][Web of Science][Medline]
Gordon T, Galdanes K, Brosseau L. (1992) Comparison of sampling media for endotoxin-contaminated aerosols. Appl Occup Environ Hyg; 7: 472–7.
Górny R, Douwes J, Versloot P et al. (1999) Application of the classic Limulus test and the quantitative kinetic chromogenic LAL method for evaluation of endotoxin concentration in indoor air. Ann Agric Environ Med; 6: 45–51.[Medline]
Hodgson MJ, Bracker A, Yang C et al. (2001) Hypersensitivity pneumonitis in a metal-working environment. Am J Ind Med; 39: 616–28.[CrossRef][Web of Science][Medline]
Hollander A, Heederik D, Versloot P et al. (1993) Inhibition and enhancement in the analysis of airborne endotoxin levels in various occupational environments. Am Ind Hyg Assoc J; 54: 647–53.[Web of Science][Medline]
Jay JM, Margitic S. (1979) Comparison of homogenizing, shaking, and blending on the recovery of microorganisms and endotoxins from fresh and frozen ground beef as assessed by plate counts and the Limulus amoebocyte lysate. Appl Environ Microbiol; 38: 879–84.
Laitinen S, Linnainmaa M, Laitinen J et al. (1999) Endotoxins and IgG antibodies as indicators of occupational exposure to the microbial contaminants of metal-working fluids. Int Arch Occup Environ Health; 72: 443–50.[CrossRef][Web of Science][Medline]
Milton DK, Johnson DK, Park JH. (1997) Environmental endotoxin measurement: interference and sources of variation in the Limulus assay of house dust. Am Indus Hyg Assoc J; 58: 861–7.[Web of Science][Medline]
NIOSH. (1998a) Criteria for a recommended standard: occupational exposure to metalworking fluids. DHHS (NIOSH) pub. 98-102. Cincinnati, OH: National Institute for Occupational Safety and Health.
NIOSH. (1998b) What you need to know about occupational exposure to metalworking fluid. DHHS (NIOSH) pub. 98-116; Cincinnati, OH: National Institute for Occupational Safety and Health.
Olenchock SA, Lewis DM, Mull JC. (1989) Effects of different extraction protocols on endotoxin analyses of airborne grain dusts. Scand J Work Environ Health; 15: 430–5.[Web of Science][Medline]
Robins T, Seixas N, Franzblau A et al. (1997) Acute respiratory effects on workers exposed to metalworking fluid aerosols in an automotive transmission plant. Am J Ind Med; 31: 510–24.[CrossRef][Web of Science][Medline]
Rylander R. (1997) Evaluation of the risks of endotoxin exposures. Int J Occup Environ Health; 3 (suppl. 1): S32–6.
Sarantila R, Kangas J, Savolainen H. (1999) Interlaboratory comparison of endotoxin analyses in occupational exposure. Bull Environ Contam Toxicol; 62: 25–9.[CrossRef][Web of Science][Medline]
Sprince N, Thorne PS, Cullen MR. (1994) Oils and related petroleum derivatives. In Rosenstock L, Cullen MR, editors. Textbook of clinical occupational and environmental medicine. Orlando, FL: WB Saunders. pp. 814-25. ISBN 0721634826.
Sprince NL, Thorne PS, Popendorf W et al. (1997) Respiratory symptoms and lung function abnormalities among machine operators in automobile production. Am J Ind Med; 31: 403–13.[CrossRef][Web of Science][Medline]
Thorne PS. (2000) Inhalation toxicology models of endotoxin- and bioaerosol-induced inflammation. Toxicology; 152; 13–23.
Thorne PS, DeKoster JA. (1996) Pulmonary effects of machining fluids in guinea pigs and mice. Am Ind Hyg Assoc J; 57: 1168–72.[Web of Science][Medline]
Thorne PS, Reynolds SJ, Milton DK et al. (1997) Field evaluation of endotoxin air sampling assay methods. Am Ind Hyg Assoc J; 58: 792–9.[Web of Science][Medline]
Walters M, Milton D, Larsson L et al. (1994) Airborne environmental endotoxin: a cross-validation of sampling and analysis techniques. Appl Environ Microbiol; 60: 996–1005.
Woskie SR, Virji MA, Kriebel D et al. (1996) Exposure assessment for a field investigation of the acute respiratory effects of metalworking fluids. I. Summary of findings. Am Ind Hyg Assoc J; 57: 1154–62.[Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  S. Spaan, G. Doekes, D. Heederik, P. S. Thorne, and I. M. Wouters Effect of Extraction and Assay Media on Analysis of Airborne Endotoxin Appl. Envir. Microbiol., June 15, 2008; 74(12): 3804 - 3811. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Spaan, D. J. J. Heederik, P. S. Thorne, and I. M. Wouters Optimization of Airborne Endotoxin Exposure Assessment: Effects of Filter Type, Transport Conditions, Extraction Solutions, and Storage of Samples and Extracts Appl. Envir. Microbiol., October 1, 2007; 73(19): 6134 - 6143. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. S. Thorne, A. Adamcakova-Dodd, K. M. Kelly, M. E. O'Neill, and C. Duchaine Metalworking Fluid with Mycobacteria and Endotoxin Induces Hypersensitivity Pneumonitis in Mice Am. J. Respir. Crit. Care Med., April 1, 2006; 173(7): 759 - 768. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. DOUWES, P. THORNE, N. PEARCE, and D. HEEDERIK Bioaerosol Health Effects and Exposure Assessment: Progress and Prospects Ann. Hyg., April 1, 2003; 47(3): 187 - 200. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||