Annals of Occupational Hygiene Advance Access originally published online on September 22, 2004
Annals of Occupational Hygiene 2004 48(7):623-633; doi:10.1093/annhyg/meh055
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
© British Occupational Hygiene Society Published by Oxford University Press;
An Exploratory Study of Changing Occupational Injury Risk during the Closure of Industrial Operations
1 Department of Environmental Health Sciences, School of Public Health, University of Michigan, 109 S. Observatory, Ann Arbor, MI 48109, USA; 2 Department of Biostatistics School of Public Health, University of Michigan, Ann Arbor, MI 48109, USA
* Author to whom correspondence should be addressed. Tel: +1-734-936-0703; fax: +1-734-936-7283; e-mail: jhv{at}umich.edu
Received 3 February 2004; in final form 6 May 2004
| ABSTRACT |
|---|
|
|
|---|
Companies go out of business on a regular basis, and industrial plants close as a result. Various social, economic and interpersonal dynamics accompany such events. Environmental health and safety (EH&S) aspects are not exempt from those dynamics during a programmed closing of a manufacturing operation. How does this condition fit the industrial hygiene paradigm of anticipation, recognition, evaluation and control? This paper describes an exploratory enquiry into the nature of risk to human health and safety associated with the planned closure of a manufacturing operation. As a starting point, the recognition of risk was explored using a qualitative opinion survey of a group of independent EH&S professionals. The survey, reflecting what is often referred to as the art of industrial hygiene, gathered professional impressions of change in potential risk of injury during a hypothetical plant-closing. The results suggested that a plant-closing might be considered a real issue by most EH&S professionals, and pointed specifically to management of personnel behavior, employee satisfaction and outside influences as primary potential sources of increased risk of injury during this final phase of a plant's life. Next, a specific real-world example was considered to evaluate actual change in injury statistics. For this, compensation claim records were examined for a malt beverage manufacturer before and after an official announcement of closure for four geographically distinct North American plants. Risk was evaluated first in terms of the one-monthly and three-monthly rates of injury incidence and then in terms of severity (expressed as cost), based on data derived from compensation claim records as of the month of plant closing. The rate of injury incidence appeared to have increased during plant-closing for one of the plants, but the results were equivocal for the other three plants. On the other hand, when the results were transformed so that the injury incidences were weighted with respect to the cost, the effective impact on the injury rate appeared to increase for three of the plants. Most importantly overall, this paper identifiesfor the first timethat there is the potential for increased injury risk during a plant closure phase, and so poses an alert for EH&S professionals and management to implement possible preventative control options in such situations.
Keywords: cost measurement environmental health and safety injury risk plant closure
| INTRODUCTION |
|---|
|
|
|---|
The economic end of a business is marked by the closure of manufacturing plants and the laying-off of employees. During plant-closing, lines of communication within the organization become strained. It is therefore not unreasonable to expect that anticipated loss of employment or general job insecurity may not only undermine the sense of well-being among workers, but may also affect potential exposure to safety and health risks. However, there is little evidence to support this. Probst and Brubaker (2001)
Clearly there is potential for instability attending anticipated employee layoff, starting at the time that the approaching shutdown becomes known or suspected, continuing through the final production runs, systems shutdown and process decommissioning; and culminating in the lay-off of personnel. All these operations represent departures from normal procedures, and are likely to create their own risk conditions that may be different from normal. In turn, employees may become at increased risk, due to, for example, chemical system closure, hazardous and non-hazardous materials disposal, powerhouse equipment shutdown, dismantling and removal of production machinery and equipment, de-energizing electrical equipment and substations, temporary increase in heavy truck traffic activity, and partial or full building demolition. This paper poses the question: are these stresses and environmental factors clearly anticipated, recognized, evaluated and controlled sufficiently?
The aim of this study was to explore possible changes in the risk of injury and the way in which that risk might change during the end-phase of an industrial business life cycle. Specifically, what is the nature of risk to human health and safety during the planned closure of a manufacturing operation? Is there a measurable change in risk of injury during such closure? What are the implications to the current practice of occupational hygiene in terms of management procedures?
The study was conducted in two parts. The first involved a survey of environmental health and safety (EH&S) professionals, to gauge their level of anticipation and recognition of this potential problem, the second a study of a specific case of the closure of plants owned by a malt beverage manufacturer.
| ANTICIPATION AND RECOGNITION |
|---|
|
|
|---|
The US Department of Labor publishes monthly news bulletins that provide information on employment statistics, including lay-offs. Such data have shown recent increases in the rate of mass lay-offs and worker applications for unemployment insurance benefits. Indeed, the February 2001 statistics show that the number of layoff events and initial claims for unemployment insurance were the highest for any February since the series began in April 1995. More generally, for each affected facility, there will have been EH&S issueschemical, biological, physical or ergonomicbeyond those pertaining to normal operating conditions. The facility EH&S professionals would have been charged with managing these issues and conditions under standard operating procedures. However, they may not have been prepared to respond to the rapid changes that could occur during a plant closure. So, in such situations, there is the potential to not fully anticipate the nature and magnitude of those changes. The question therefore is: may the EH&S professional expect to encounter a greater or lesser risk of employee injury during a plant closure?
To obtain a first impression of such expectations, a survey was carried- out of 22 persons educated and experienced in industrial hygiene and/or safety. This may be regarded as an informal exercise in expert or professional judgment. Each participant was given a brief written scenario of a beverage manufacturing operation that is about to close (see below), and asked to evaluate 28 familiar EH&S hazards and rank the injury risk for each hazard as it might be affected by the plant-closing. The median EH&S experience for the survey participants was 11 years (range 228 years).
Of the 22 respondents, 10 reported having worked as EH&S professionals during an actual plant closing. A separate subgroup of four reported experience specifically in malt beverage manufacturing. The survey database was also examined for three subgroups: participants who had and had not (n = 10 and 8, respectively) held EH&S responsibilities at a facility that was closing, and participants who had experience specifically in the malt beverage industry (n = 4). The survey data showed that 47% of responses for those with no experience during plant closure, 51% of responses for experience during plant closure and 50% of responses for those with malt beverage experience ascribed an increase in risk of injury associated with the hypothetical plant-closing. Since these subgroup results do not differ substantially from the composite results, the EH&S professionals from different backgrounds all appear to consider plant closure a significant potential EH&S issue.
Experience had some influence in the survey data outcome, as shown in Fig. 1, where the percentage of responses that anticipated increased injury risk is plotted as a function of the number of years of EH&S experience. Here, a positive trend is revealed. Not surprisingly, variability in opinion was greatest for those with <10 years experience. Anticipation of increased injury risk, however, generally increased with years of experience.
|
A summary of the survey results is presented in Table 1. Overall, 49% of the responses anticipated an increase in risk of injury associated with the hypothetical plant-closing, with 48% anticipating no change and only 3% anticipating less risk of injury. The importance of these results is not so much in their magnitude, but that there were a number of hazards that were consistently identified with an increase in injury risk associated with plant closure. The five most prevalent factors that came up in the survey as ones that would increase the risk of injury associated with plant-closing were:
- workplace violence (100% of respondents)
- loss of qualified personneltechnical staff (100%)
- loss of qualified personnelEH&S (95%)
- contractorsplant decommissioning, equipment removal, demolition (91%)
- hazardous wasteaccumulation, storage, disposal (73%)
|
| EVALUATION: A CASE STUDY |
|---|
|
|
|---|
We examined the case of an actual manufacturer that went out of businessa small, privately owned malt beverage manufacturing enterprise. For this company, acquisition and merger with other malt beverage producers afforded survival in a flat-to-declining growth industry. Nonetheless, annual sales declined each year, untilin 1999the doors were closed.
Incidence of injury and illness
For the Food and Kindred Products standard industry classification (SIC 20), the range of incidence of non-fatal injury across the industry in 1999 was 7.612.3 (median = 9.4) injuries per equivalent 100 employees. Within this SIC 20 classification, the beverage sector reported at the higher end. Indeed, the reported beverage sector injury incidence rate per 100 employees ranged from 15.1 in 1995 to 11.0 in 1999. By contrast, the sub-sector malt beverage industry non-fatal injury incidence ranged from 12.1 in 1995 to 6.5 in 1999. Moreover, the beverage sector also experienced a number of fatal occupational injuries, some of which occurred in the malt beverage sub-sector. Clearly, therefore, the beverage-manufacturing sector is an appropriate example for this discussion.
Occupational Safety and Health Administration (OSHA) accident reports
Data from OSHA accident investigation reports for the malt beverage industry over the period from 1984 to 1995 show that the types of injuries sustained included fatality (oxygen deficient tank, electric shock, tank explosion and struck-by/caught-between incidents), amputated body parts, crushed hand, fractures, falls and slips. In general, these OSHA records showed that, even though non-fatal injury incidence for the malt beverage worker is lower than the beverage sector in general, during normal operations there is risk to health and safety of employees from a variety of sources. A closer look at a typical operation helps to identify the injury risk that might be associated with plant-closing.
Operations and hazards at a typical malt beverage manufacturing operation
The production of malt beverages involves a complex sequence of primary and ancillary processes. Primary processes include delivery at the plant of grain and other required products by railcar, conveyance to storage silos, transfer to the mill, mixing with water to form the mash, brewing, fermentation, filtration, packaging andfinallytransfer to the warehouse for distribution. Ancillary processes include steam generation, refrigeration, cleaning and disinfection of equipment, production and storage of secondary products (e.g. carbon dioxide), water sterilization, pasteurizing of product, and solid waste disposal.
The major processes in the hazard evaluation for the industry in question are the brewing operations themselves, filling and packaging operations, warehousing operations, utility operations, maintenance operations, laboratory operations and administrative office operations. Most occupational hazards can be managed during routine production periods through well-defined programs that are professionally implemented with accountability, audit, and continuous improvement. But the routine changes when an announcement of plant-closing is made.
Closure of business: timeline of events
In 1999, the malt beverage manufacturer that was used for this case study, with four plantsNORTH, SOUTH, EAST and WEST, in different parts of the USA as indicated by their pseudonymswas forced to close business. Each facility was independent, even though it belonged to the same company. Production operations were basically the same, but types of products and packages varied somewhat by facility. Acquisition and merger had previously afforded company survival. SOUTH and EAST belonged to a 1982 acquisition, while NORTH and WEST belonged to a 1996 acquisition. As a result, each facility had a separate culture.
A condition of closure was the transfer of viable production as an asset to another company that had suffered unused production capacity but otherwise remained economically sound. This condition made it necessary for the company to continue production even after asset transfer, so that the incoming company could increase materials procurement and specific operations in order to effectively accept the additional production. The timeline of events during closure was as follows:
- October 1998: the news media broke an unconfirmed story that talks of business merger were in progress; there was no confirmation by company officials, but employees became concerned.
- February 1999: the company announced that a sale of assets to another company had been agreed, and issued its Worker Adjustment and Retraining Notification Act 60 day advanced notice of plant-closing to employees.
- March 1999: plans to continue production for the new company were set in place.
- April 1999: the company transferred assets to the new company but continued to operate its breweries according to the schedule:
- SOUTH ceased brewing in May, and ended packaging activities in June.
- NORTH ceased brewing in June, and ended packaging activities in August.
- EAST ceased brewing in July, and ended packaging activities also in July.
- WEST ceased brewing in July, and ended packaging activities in August.
- SOUTH ceased brewing in May, and ended packaging activities in June.
- Within 10 days after production activities had ended, manufacturing areas were cleaned, tanks and vessels emptied, and wastes were accumulated for disposal. At this time, all employees were terminated with the exception of those needed to decommission utility systems.
- Within 30 days after production activities had ended, radioactive sealed sources were removed and secured by general licensee, and chemical systems (refrigeration, CO2, cleaning/sanitizing bulk and day tank systems, fuel oil) were decommissioned by utility maintenance workers.
- Within 3 months after production activities had ended, all waste materials were characterized and removed from the premises by contract services.
- By February 2000, equipment and machinery were sold and removed from the premises through a variety of brokers.
- By May 2000, the vacant industrial properties underwent environmental assessment testing by independent contractors and were transferred to new owners.
| CASE STUDY DATA AND ANALYSIS |
|---|
|
|
|---|
Compensation claim records were available for all four plants for the period from 1997 until the closures of the individual plants. The analysis was complicated by several factors. First, injury incidence may have been seasonal due to routine greater production efforts in the Spring through Fall months, and a resultant transient seasonal work force. Secondly, the company instituted an incentive program in January 1998 to improve EH&S performance with the goal of reducing injury incidence. Thirdly, the exact timing of the announcement of closing was blurred by media reports of impending business transactions. With these factors in mind, we defined two break points in time to examine before-and-after conditions of injury incidence: one after December 1997 to account for pre versus post-incentive related injury rates, and one after December 1998 to describe pre and post-announcement related injury rates. The three study periods are thus 1997 pre-incentive, 1998 pre-announcement and 1999 post-announcement.
Risk based on injury incidence
For this study, injury is defined as a compensable claim, and categories include sprains and strains; bruises, contusions and abrasions; tears, cuts and lacerations; fractures; foreign body in eye or ear; heat, burn and scald; repetitive motion-related stress; chemical contact; multiple injury; hearing impairment; and amputation. Injury incidence was calculated as the monthly rate of compensable claims per 100 employees. It is noted that this is not directly comparable to the corresponding OSHA injury incidence rate because actual work hours are not incorporated. Time plots for the four plants for the period of interest are shown in Figures 2![]()
5. Also included in these graphs are the three-monthly average injury rates. From initial inspection of the plotted data, it appears qualitatively that injury incidence at SOUTH may have declined during the incentive period but increased after the announcement of impending closure. For NORTH and EAST, the injury incidence declined continuously through the incentive and announcement periods. For WEST, the incentive did not appear to have much effect, and the injury incidence appears to have declined after the announcement.
|
|
|
|
For closer inspection, statistical analyses of the data were performed using Minitab 12 (Minitab Inc., State College, PA). Having initially standardized the data for each plant in order to make them consistent, the first analyses involved examination of the time dependence of injury incidence by the method of auto-regressive integrated moving averages (ARIMA). ARIMA analyses for each location were performed with a first-order autoregressive model (AR1). Next the monthly injury incidence data for each plant were analyzed using multiple linear regression to compare linear coefficients and population means of the three study periods. The independent variables included three contrived sets of variables: a sequential series of numbers to model the time trend, a dummy variable that assigns all periods 1 and 3 values at 0 and period 2 values at 1, and another dummy variable that assigns all periods 1 and 2 values at 0 and period 3 values at 1. In this way, period 1 values would be a reference period and the linear coefficients of the two dummy variables from the multiple regression could be used to interpret comparative significance of period 2 versus 1 or period 3 versus 1. In addition, injury rate for the previous month was also included to model first-order serial correlation as determined to be needed from the ARIMA analysis. The models were not identical across the facilities, and the choice of the model for each facility was based on examination of the time plot and the higher adjusted R2. A summary of the results of the full statistical treatment is given in Table 2.
|
For NORTH (see Fig. 2), ARIMA showed no first-order serial correlation. Although the period means were not different, there was a slight decreasing trend over time, but it was not significant (P = 0.10). For SOUTH (see Fig. 3), ARIMA showed marginally significant serial correlation with the estimated first-order autocorrelation coefficient of 0.39 (P = 0.07). The multiple regression model therefore included the previous month's incidence rate to adjust for it. This indicated that there was no significant trend over time. Instead, the final model showed that period 2 was significantly different from period 1 (P < 0.01), while period 3 was not significantly different from period 1 (P = 0.18). Furthermore, a two-sample t-test assuming unequal variances showed that the means of periods 2 and 3 were significantly different (P = 0.01). For EAST (see Fig. 4), ARIMA showed serial correlation with the first-order autocorrelation coefficient estimate of 0.35 (P = 0.05), and thus the regression model included the previous month's incidence rate to adjust for it. The incidence rate showed a decreasing trend over time (P = 0.03), showing that this was a better model. Finally, for WEST (see Fig. 5), ARIMA showed that there was no serial correlation. There was no significant trend over time, nor any period effects.
Risk based on severity of injury (as described in terms of cost)
In general, the assessment of risk requires (i) the identification of a hazard (i.e. a dangerous substance or action that can cause harm); (ii) the anticipation that an event (or events) may occur to create the possibility of harm; and (iii) a statistical estimate of the probability that harm will occur (Cohrssen and Covello, 1989
). For the last part, in the present study we have identified the incidence of injury, and this formed the basis of our analysis. However, the American Chemical Society (1996)
suggests that such assessment also requires consideration of the severity of the harm, and we define this for present purposes as the cost of the injury-related compensation claims, under the broad assumption that medical cost is proportional to clinically diagnosed severity. In turn, therefore, the impact of the injury may be definedagain for present purposesas the product of incidence and severity.
With the preceding in mind, Table 3 shows values for the annual averaged injury incidences, the weighted average cost of claims and the resultant impact for each of the plants for each of the years 1997, 1998 and 1999. Here, it is recalled that the announcement of the planned plant closures was in February 1999 and the actual closures took place during the next 8 months. When portrayed in this way, it would appear from qualitative inspection of the data that the overall impact of the injuries increased for three of the four plants, quite sharply for two of them (NORTH and SOUTH). Thus, it may be argued, the presentation of the injury-related data in this form may have amplified somewhat the plant closure effect.
|
These observations beg the question of whether or not the nature, or category, of injury changes, thus resulting in a greater risk of injury. To answer this question, we looked at the top three categories of injury impact (incidence x severity) during each period. These data are summarized in Table 4. In general, the primary risk is from sprains and strains. Observations other than sprains and strains as a primary risk might point to extraordinary circumstances, such as fractures in 1998 at SOUTH, or tears, cuts and lacerations in 1998 at WEST. One interesting observation at both NORTH and EAST was the occurrence of hearing impairment as a secondary risk but only in 1999, the post-announcement period. This occurrence might be a special artifact of plant-closing. In general, whereas the nature of injury does not change substantially at any location, the magnitude of risk appears to change substantially at three locations.
|
| DISCUSSION AND CONCLUSIONS |
|---|
|
|
|---|
This study set out to investigate the change in risk of injury and the manner in which it might change during the end-phase of an industrial business life cycle. It posed the questions: (i) what is the nature of risk to human health, safety and the environment during the planned closure of a manufacturing operation; (ii) is there a measurable change in injury incidence and risk during the planned closure of a manufacturing operation; and (ii) what are the implications to the current practice of industrial hygiene in terms of management procedures? The responses to the survey of EH&S professionals to a hypothetical situation provided an important glimpse into the anticipated nature of risk in injury associated with a plant-closing. From this survey, it was clear that EH&S professionalswhen confronted with the questiondo think that there is potential increase in risk of injury associated with plant-closing. In addition, there was substantial agreement that management of personnel behavior, employee satisfaction, independent outside influences and environmental issues are primary sources of increased risk of injury. This tendency is supported by the work of Probst and Brubaker (2001)
From our analysis of available data from an actual plant-closing, we could find no consistent pattern of change in risk emerging from inspection of injury incidence by itself. Certainly, SOUTH exhibits a significantly lower injury incidence during period 2 (post-incentive) as compared with either the pre-incentive or post-announcement periods. The increase in injury incidence during period 3 rises to a level that is similar to that before the program incentives were offered. But the results for NORTH, EAST and WEST do not exhibit any significant increase in injury incidence from before to after the announcement of plant-closing. Rather, EAST actually showed a decreasing trend in injury incidence over time. However, when the severity of injury (cost) is considered as a component of risk, the picture changes. NORTH, SOUTH and WEST plants point to a noticeable increase, or increasing trend, for risk of injury during the plant-closing period; EAST plant appears to decline in risk of injury, both in incidence and severity. But these evaluations are based on four facilities of the same company. Any conclusion from this case study alone about the possible change in the risk of injury during plant-closing is therefore inevitably weak. It is likely that, in a single case study like the one reported here, there are insufficient data to confirm the anticipation reflected in the survey (see Table 1). The insights gained during the survey therefore become additionally important. Management-related issues (e.g. personnel behavior, employee satisfaction, independent outside influences) as primary sources of increased risk of injury, rather than the traditional health and safety injury sources (e.g. ergonomics, hazardous energy, machine guarding, physical stress, noise, heat, confined space, fall protection, etc.) are clearly seen as priorities, and this is entirely plausible. It follows, therefore, that attention to continuation of effective management of EH&S programs and personnel management programs is especially important during plant closing.
More generally, in the real world, actual changes in injury risk may or may not be observed during a plant-closing. The data in the case chosen here as an example were ultimately insufficient to allow that important conclusion to be made outright. Injury incidence appeared to increase at SOUTH plant after the plant-closing was announced. But the observed final incidence rate was similar to a rate prior to a safety incentive program 1 year earlier. So, was safety motivation affected by job security? Or was injury under-reported during the incentive program for reward? The picture is clouded by the fact that, at three comparable plants under the same circumstances, but in other parts of the country, no change in injury incidence could be observed from a potential change inducing situation. From this, it would appear that change in injury incidence cannot be predicted based on a condition of plant-closing alone. When the data were re-examined in the form of the impact that includes both injury incidence and its severity (as reflected in the compensation-related cost), the outcome appears to have been amplified. Now the impact of injury at two of the plantsSOUTH and NORTHmore clearly increased after announcement of plant-closing, regardless of the trend in injury incidence itself (which was more equivocal). However, for EAST, all the indices decreased over the whole 3-year period. So the inconsistency remains. Of course, we may not have yet identified the most appropriate index by which to describe the problem, and this may be the subject of a future enquiry based on a larger and more diverse data set. One possibly useful component to add to injury-related data of the type already examined might be, for example, the incidence of near misses (if recordedor indeed recordable).
Geller (1998)
categorized factors contributing to work injury into three areas: environmental factors, personal factors, and behavioral factors. But, according to Geller, Only two of (the) three major factors . . . can be managed effectivelyenvironmental/equipment factors and work behaviors. Attitudes, expectancies, perceptions, and personality characteristics of people cannot be measured, managed, or controlled reliably; these dramatically influence the risk of personal injury. Human factors contributing to injury are complex. Not all injuries can be prevented. Geller explains the complexity and uncontrollability of human nature in seven human dimensions: behavior, attitude, sensation, imagery, cognition, interpersonal and drugs. Certainly environmental health and management programs need the elements of acting safe (behavior) and thinking safe (attitude). Perhaps these are the key elements in whether or not risk of injury will change during a scheduled plant shutdown. People's sense of personal and self-worth is closely tied to what they do at a job or during leisure time afforded by a job. A plant-closing can generate stress and tension to the point of anger and hostility. Special considerations therefore need to be given during a plant-closing occurrence. Insurance carriers provide valuable guidance on safety, security, and risk management during such a circumstance. Some general considerations for safety, security, and risk management relevant during a plant-closing scenario are summarized in Table 5. Importantly, it needs to be recognizedby both management and workersthat the EH&S professional's responsibilities must inevitably change during a plant shutdown. Special emphasis must be given to employees with high risk of injury. Supervision responsibilities must increase. Reports on injury, absenteeism and return to work all become important documents for post-closure legal investigations. Awareness is key to a safe plant closure. Overall, the strength of an EH&S program may make the difference in how employees act and think safely during a plant closure sequence. In general, it is suggested that EH&S professionals need training in the management of human psychology to round out their technical knowledge and management of environmental factors and work behaviors. For these professionals, the ability to anticipate risk is an essential work component.
|
In conclusion, although EH&S professionals perceive that a plant-closing circumstance increases the potential risk of injury, a plant-closing circumstance may not alone be a direct predictor of risk. However, this does not mean that the potential for risk of injury does not exist. Awareness of potential problems along the lines indicated is most important. Perhaps the most important conclusion from this work is that such situations need special consideration within the industrial hygiene paradigm, and in turn in the education and training of EH&S professionals.
| REFERENCES |
|---|
|
|
|---|
American Chemical Society. (1996) Chemical risk: a primer, information pamphlet. Washington, DC: American Chemical Society.
Biddle EA, Blanciforti LA. (1999) Impact of a changing U.S. workforce on the occupational injury and illness experience. Am J Ind Med; 36: 710.[CrossRef]
Cohrssen JJ, Covello VT. (1989) Risk analysis: a guide to principles and methods for analyzing health and environmental risks. US Department of Commerce, National Technical Information Service (NTIS), Washington, D.C., PB 89-137772.
Geller, E.S. (1998) Working safe: how to help people actively care for health and safety. CRC Press LLC, Becu: Raton, FL.
Probst TM, Brubaker TL. (2001) The effects of job security on employee safety outcomes: cross-sectional and longitudinal explorations. J Occup Health Psychol; 6: 139159.[CrossRef][Medline]
Santana VS, Loomis D. (2004) Informal jobs and non-fatal occupational injuries. Ann Occup Hyg; 48: 14757.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||




