Annals of Occupational Hygiene Advance Access originally published online on September 7, 2009
Annals of Occupational Hygiene 2009 53(7):651-655; doi:10.1093/annhyg/mep065
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
EDITOR'S CHOICE
Data Sharing, Federal Rule of Evidence 702, and the Lions in the Undergrowth
Editor-in-Chief, Annals of Occupational Hygiene, British Occupational Hygiene Society, Pride Park, Derby DE24 8LZ, UK
* Author to whom correspondence should be addressed. Tel: +44-1332-298-101; e-mail: editor{at}ogs.org.uk
ABSTRACT
Many concerns would find it useful to have a publicly available database recording exposures to particular substances, with contextual information. The European Union's Registration, Evaluations, Authorisation, and restriction of Chemicals regulation has increased interest in this. It has been suggested that journals should require detailed publication of exposure data with papers. But there are problems for researchers, for whom the data are a valuable resource which has been obtained with effort and often ingenuity. The publication could also raise problems of confidentiality and liability, and those who have to put the effort in publication are not those who benefit. Also, there are the problems of hostile critics misusing the information—for example, industry reanalysing data to counter any regulatory implications of a study—and this raises serious wider issues of editorial policy. Two books have recently given examples of industry misuse of science, ‘Doubt is their product’ by David Michaels and ‘Defending the indefensible’ by McCullogh and Tweedale. Michaels gives examples of hostile data reanalysis, and among other things, he discusses the impact on journals of US Federal Rule of Evidence 702, which encourages expert witnesses to try to get their testimony material into peer-reviewed journals. This certainly lies behind some submissions to this journal, and Michaels says that it has led to the creation of peer-reviewed journals which have strong industry influence. On the other hand, work funded by industry is not always wrong, papers from other sources are not always free from bias, and the problem for journals is to continue to apply consistent scientific standards in a sea of conflicting interests. It does not seem feasible or desirable for journals to insist on the publication of all the underlying data, although researchers might be encouraged to form consortia to share data.
Keywords: data sharing • journals • publication ethics
WHY CAN'T WE SHARE OUR DATA?
Pressures on journals do not decrease. A recent suggestion that this journal should do more to encourage sharing of exposure data led to a discussion in the Editorial Board on data misuse by industry and others. Two recent books have also discussed this. A simple suggestion on data sharing turns out to raise many complicated issues with implications for what we publish and how.
It is an old idea that occupational exposure data should be shared. If someone has already determined that compound A produces a certain exposure when used at a rate B with a control regime C, and if this information was available in a standard form on a publicly accessible database, it could save a lot of work. It would not only help those with an immediate problem with the same substances, but also the consolidated information could be interrogated for purposes such as generating practicable exposure limits, for specifying efficient risk management measures, or for epidemiology.
The European Union's Registration, Evaluations, Authorisation, and restriction of Chemicals regulation (REACH) has given this new importance because manufacturers or importers of chemicals have to consider likely exposures, effects, and necessary risk management measures before chemicals are placed on the market (Marquart et al., 2007). If exposure measurements and contextual information were publicly available in a standard form, effort need not be duplicated. Also, REACH has stimulated work on tools to predict exposure, without or to supplement measurement, but the tools need real data for their calibration. Examples are tools such as Stoffenmanager (Marquart et al., 2008; Tielemans et al., 2008a) and the Advanced REACH Tool (ART) (Tielemans et al., 2008b). The British Occupational Hygiene Society (BOHS, 2009) is involved in a project to gather suitable calibration data for ART.
The Dutch Health Council has looked in-depth at sharing of health data, and among other things has recommended establishment of independent data broker for The Netherlands, whose role would be to put data holders and data users in touch with one another (Raad voor Gezondheidsonderzoek, 2008). Could such a system be applied to exposure data? The report also says that, at least in The Netherlands, a researcher cannot claim ownership of data.
Last year this journal was approached with the proposal that we should help by requiring that, when we publish a paper which includes substance exposure measurements, the authors must include the basic data—individual measurements and relevant contextual information—as supplementary material in the online edition. A special session at the BOHS conference in April explored the broader problem of sharing exposure data to improve REACH assessments. The discussions have revealed several problems.
- (1) The researcher puts a lot of effort into getting good exposure data and may have plans for their further use; also access to the unpublished data can be an asset in getting further grants.
- (2) It takes time and effort to prepare data for publication, and in the short term the people who do this to make their data available are not the ones who benefit by their availability.
- (3) There may be problems with confidentiality and liability for the workplaces where the measurements were obtained.
- (4) The data may be misused; in particular, they may be reinterpreted by those with a commercial interest in undermining the conclusions drawn by the original researchers.
- (2) It takes time and effort to prepare data for publication, and in the short term the people who do this to make their data available are not the ones who benefit by their availability.
This last point can be a special case of industry's misrepresentation of science for its own interest. This has been discussed in two recent books, which are in different fields, but which are both about misuse of science. To a certain extent they leave an impression of honest researchers as innocent prey, with industry and campaigning interests like lions crouching in the undergrowth, waiting to pick them off. But in some cases, the journals and authors are seen as guilty too, and one of the books identifies this journal with the villains.
What initially looks like a fairly straightforward suggestion therefore turns out to raise far-reaching issues. This commentary considers what warnings the two books carry for our editorial policy generally, and for the publication of exposure data, and then considers what a journal's policy should be.
‘DOUBT IS THEIR PRODUCT’
The connecting theme of ‘Doubt is their Product’ (Michaels, 2008) is the attempts of industry to delay or stymie regulation by casting doubt on the science which seems to show health or environmental hazard. The author, David Michaels, was an Assistant Secretary for Energy under the Clinton administration, and in that job he was responsible for the health and safety of workers, communities, and the environment in the neighbourhood of nuclear weapons facilities, and he was responsible for the agreement to compensate workers made ill by working on nuclear weapons in the past. He has recently been nominated as head of the US Occupational Safety and Health Administration, but at the time of writing this awaits confirmation by the Senate. The title of his book comes from an internal memo by a cigarette executive in the 1960s, later made public as a result of court action. This argued that the industry's best approach in defending hazardous products is to cast doubt in the public mind about the truth of the scientific consensus. Michaels illustrates many attempts at this with much detail in a wide range of industry, mainly USA.
There are several important lessons for journals in this book, but many originate with Federal Rule of Evidence 702 (Cornell, 2008a). This stems from a law case, Daubert v. Merrill Dow Pharmaceuticals, which reached the US Supreme Court in June 1993 and to which Michaels devotes a chapter. The notes to the Rule (Cornell, 2008b) have a checklist for courts to apply to expert evidence, including whether the expert's ‘technique or theory has been subject to peer review and publication’. For some years, we have been getting submissions from consultants in the USA which are clearly attempts to get a ‘technique or theory’—usually results and an interpretation of the results—into this peer-reviewed journal. Some of these attempts are pretty crude, some seem technically sound but are of too narrow interest to be worth our publishing, and some fully meet the usual criteria, and we publish them. Even though industry seems much more aggressive and confrontational in product defence in the USA than in Europe, the international nature of scientific publication gives this Rule of Evidence worldwide significance.
Of course, publication in a peer-reviewed journal does not guarantee that something is right, only that it has no obvious errors and that the interpretation seems to be supported by the data. As Richard Smith, former editor of the British Medical Journal, puts it, ‘Science deals in provisional truths’ (Smith, 2006; p. 61). Publication exposes the interpretation to a wider critical audience, which brings in another item in the Federal Rule checklist: ‘whether the technique or theory has been generally accepted in the scientific community’. Recent illustrations of how this can work are the post-publication correspondence of Roggli et al. (2009) and Finkelstein (2008, 2009) and of Conolly et al. (2009) and Crump et al. (2008, 2009).
Michaels names authors and consultancy firms which he regards as suspect in this business, and even identifies journals which, he says, are either paid for by industry or dominated by industry trade groups and consultants. This has been made more credible by the recent case in which the publisher Elsevier has admitted that its Australia office launched six publications between 2000 and 2005 which had the appearance of peer-reviewed journals, but were in fact paid for by the pharmaceutical industry (Grant, 2009). The head of Elsevier's Health Sciences Division has said that ‘This was an unacceptable practice, and we regret that it took place ... I understand this issue has troubled our communities of authors, editors, customers and employees’ (Elsevier, 2009). It will certainly trouble many well outside those groups: this sort of thing undermines the credibility of the whole process. Looking at the past of this journal, in 1994 we used 
250 pages to publish the results of a workshop on chrysotile (38: pp. 397–646), and this publication was stated to be paid for by Canadian and Quebec government interests (p. 400). There is no reason to believe that influence was exerted, but today we would require a more prominent declaration of interest. The following year, we published the proceedings of a workshop on man-made mineral fibres (39: pp. 633–779). It is unclear how this publication was paid for.
Financial or other interest may influence even honest authors. Smith (2006, Chapter 11) discusses the effect of financial conflict of interest, citing in particular Bekelman et al. (2003), who found that industry sponsorship was significantly associated with pro-industry conclusions with an odds ratio of 3.6. Although the obvious interpretation is that industry is biased towards publications that did not show ill-effect of its products, a more important reason may be bias of non-industry researchers in publishing positive studies, which attract more interest, are probably more cited, and may therefore be more likely to be accepted by journals than negative studies.
Problem 4 in the previous section was industry reanalysis of data to support its own conclusions. Michaels alleges three examples of this, on environmental tobacco smoke, on chromium-contaminated groundwater, and on carbon black. The last example is said to have possibly influenced the International Agency for Research on Cancer (IARC) classification. However, industry is not always wrong, and campaigners can overlook this because it is easier to identify the paymaster than judge the science. Also reanalysis has benefits if done properly; we expect duplicate sampling and confirmation of the chemical analysis when appropriate, but we do not demand independent statistical analysis to test if this and the interpretation are correct. However, at present, reviewers often require refinement of the analysis or a different approach, and the Conolly–Crump and Roggli–Finkelstein correspondence cited above illustrates that statistical analysis can be fruitfully challenged after publication.
ASBESTOS AGAIN
Asbestos provides examples of most kinds of malpractice. One occurred earlier this year. As this journal reported before, a report on the risks of chrysotile, which was commissioned by the Canadian government and which had nothing very surprising to say, was suppressed by the government for a year, apparently to protect the Canadian mining industry (Ogden, 2008, 2009). It was eventually released under access to information legislation, but only on the Thursday afternoon before the Easter holiday began, and then only to members of the public who wrote and asked for it. On the Saturday, while the report was still inaccessible to all but a handful of people, a federal minister was quoted by a Quebec newspaper, saying that the experts had not reached a firm position, that they had irreconcilable positions on controlled use, and that this disagreement meant that there was no reason to change policy on chrysotile (Bussières, 2009). As chairman of the expert panel which wrote the report, I do not think that any of these statements is a fair analysis. Among Western governments, Canada is probably alone in its defence of asbestos, although in a debate in the UK parliament, a main opposition spokesman did describe chrysotile with the memorable phrase ‘soft, silky and biodegradable’ (Hansard, 2002)! If authorities in Western democracies promote such messages, it is not surprising that people distrust the statements of industry.
McCullogh and Tweedale (2008) certainly do. Geoffrey Tweedale (2000), who is Reader in Business History at Manchester Metropolitan University, wrote an exposé of the manufacturer Turner and Newall's frequently callous treatment of its workforce and their risks, based on a mountain of documentation preserved and then released by order of US courts. He has now collaborated with Jock McCullogh, Professor of History at RMIT Melbourne, to extend this study to cover the global industry. The book analyses much information not easily available and puts it into a readable account, and it would be good to be able to recommend it as a record of the capacity of industry to manipulate and suppress information. But testing it in areas I know about, I feel that it is flawed.
An example is the authors’ treatment of the McGill studies of the Canadian miners and millers, which reported very low risks except at high dust levels or where tremolite contamination might be the culprit. I read the account in Chapter 5 of industry's machinations over this, waiting for some appraisal of whether or not the risks are indeed low in this environment, but this does not come. Whether or not it is true is surely is the most important point, for the protection of the workforce as well as for the science. It is a mistake if we think that because the industry helped pay for the study and has exploited the findings in its propaganda, the results must necessarily be wrong—life, including science, is not this simple. On this particular example, the scientific consensus is that the lung cancer risk does indeed seem lower on average in Canadian mines and mills than in some other industries, but the risk still exists and the confidence limits mean that these findings are consistent with other evidence (see e.g. Hodgson and Darnton, 2000). Corbett McDonald, the McGill worker who McCullogh and Tweedale most criticize, recognized ‘the very high risks of lung cancer among workers in the manufacture of asbestos textiles’ (McDonald, 1998). An IARC panel including McDonald and two other McGill scientists (and industry representatives) had recognized unequivocally in 1972 that chrysotile is able to cause lung cancer (IARC, 1973). It is therefore misleading to say (p. 137), ‘By the end of the twentieth century, then, the McGill scientists had presented the world with a product that was whiter than white.’
I will briefly mention some other problems with McCullogh and Tweedale's book. The authors do not like BOHS, which in the context of the 1968 chrysotile standard is characterized as ‘an unofficial and self-appointed body of industry physicians and scientists’ (p. 105). In fact, in 1971 the membership was about 47% industry and nationalized industry, 42% government and academia, and 11% other or unknown (BOHS, 1971). The 1971 membership included McCullogh and Tweedale's hero Irving Selikoff, but not their bête noire Corbett McDonald! The Annals is described as the ‘mouthpiece of BOHS’ (p. 115). The journal is editorially independent of BOHS and always has been, and in its 50 years has probably not carried more than a dozen statements on behalf of the Society. In 1997, the journal published a guest editorial attacking the critics of the McGill studies in inflammatory language (Liddell, 1997). McCullogh and Tweedale wrongly attribute (p. 148) to BOHS the then editor's statement defending the editorial, but BOHS's official repudiation of the editorial (BOHS, 1997) is not mentioned, so that the reader is left with the false impression that BOHS supported the editorial. (The repudiation is probably the most recent occasion when the Annals really did become the mouthpiece of BOHS!)
On other topics, it is said that the 2 fibres per millilitre standard was adopted as a ‘legal standard’ in Britain in 1969, but as I pointed out in my review of Geoffrey Tweedale's earlier book (Ogden, 2000), a major problem for the inspectorates in applying the standard was that it was only in guidance and was not an enforceable legal standard. It would have been better if the authors had asked why Britain did not have an enforceable standard until 1987. In discussing Britain's recent requirement to manage asbestos in premises, it is said that the government ‘has exempted certain asbestos decorated building materials from its regulations’ (p. 220), but this exemption does not apply to the regulations being discussed but to the need to use licensed contractors with this material. These errors on straightforward points are not so important in themselves, but they must leave the reader with doubt about the accuracy of other statements in the book. In my view, this could have been a much more useful book if more care had been applied to this kind of detail.
Siegel (2007) discusses how overstatement of the risks of passive smoking undermines the antismoking message. ‘The dissemination of inaccurate information by antismoking groups to the public in support of smoking bans is unfortunate because it may harm the tobacco control movement by undermining its credibility, reputation, and effectiveness.’ This applies to risks generally. Careful respect for the truth is important.
SO WHAT SO WE DO?
This is not a very encouraging picture. It looks as if we cannot trust industry, and its critics are not very reliable either. Although the two books mentioned pick on the bad cases, there are a lot of honest scientists around and quite a few of them work for industry or with industry money. It seems that an industry-sponsored study is much more likely to find results favourable to industry, but this may partly or wholly be because non-industry researchers find it harder to publish negative or inconclusive results. Scientific studies must be judged primarily on the quality of the evidence, not on who pays for them.
So consideration of whether the journal should require authors to publish their original exposure data has led to broader issues of editorial policy. We welcome comment on these policies, but meanwhile some conclusions emerge.
- Public availability of exposure data is a good thing, and we can encourage it and give authors the facility to publish detailed results in online supplementary material, but there are excellent reasons why a researcher may not want to, and we must respect this. The data must, however, be available to editors and reviewers on a confidential basis. Roel Vermeulen (personal communication) has commented that researchers should try to form large consortia to share data in a particular field, to give improved studies, and to aid in the validation of prediction tools. This does not help non-research applications by users, such as manufacturers and suppliers of products under REACH, and further ideas are needed as to how these problems can be overcome. The Dutch Health Council's idea of a data broker is interesting, but do they mean to compel researchers to release data?
- Declarations of interest in publications are essential, especially if the authors are likely to be involved in legal testimony. Failure to offer this must be treated very seriously.
- Reanalysis by a different statistical technique or with different assumptions is desirable, and reviewers and commentators should be encouraged to consider this. This would also strengthen papers against hostile reanalysis by third parties—the lions in the undergrowth!
Meanwhile, editors and reviewers—and that probably includes most people who read this—have a job to do to keep the stream of information running as pure as possible.
ACKNOWLEDGEMENTS
Declaration of interest—In view of McCullogh and Tweedale's statement that this journal is the mouthpiece of BOHS, this commentary was exposed in draft to the Editorial Board, but not to any officer of BOHS. Corbett McDonald was an assistant editor of this journal from 1998 to 2002.
FOOTNOTES
The free full text version of this article can be found in the online version of this issue.
Received June 2, 2009; in final form August 7, 2009
REFERENCES
Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. J Am Med Assoc (2003) 289:454–65.
British Occupational Hygiene Society. British Occupational Hygiene Society membership list. Ann Occup Hyg (1971) 14:361–73.
British Occupational Hygiene Society. Editor's note. Ann Occup Hyg (1997) 41(4):iii.
British Occupational Hygiene Society. Collection of exposure data to support the development of the Advanced REACH Tool (ART) (2009) Available at http://www.bohs.org/standardTemplate.aspx/Home/HotTopics/REACH/TheroleofBOHSinREACH. Accessed 25 May 2009.
Bussières I. Amiante: Paradis réitère son appui (2009) Montreal, Canada: Le Soleil. Available at http://www.cyberpresse.ca/le-soleil/actualites/sante/200904/10/01-845662-amiante-paradis-reitere-son-appui.php. Accessed 16 May 2009.
Conolly RB, Miller FJ, Kimbell JS, et al. Formaldehyde risk assessment. Ann Occup Hyg (2009) 53:181–4.
Cornell. Federal Rules of Evidence (2008a) Cornell University Law School, Legal Information Institute. Available at http://www.law.cornell.edu/rules/fre/rules.htm and http://www.law.cornell.edu/rules/fre/ACRule702.htm. Accessed 29 May 2009.
Cornell. Federal rules of evidence. Notes to the rules (2008b) Cornell University Law School, Legal Information Institute. Available at http://www.law.cornell.edu/rules/fre/ACRule702.htm. Accessed 29 May 2009.
Crump KS, Chen C, Fox JF, et al. Sensitivity analysis of biologically motivated model for formaldehyde-induced respiratory cancer in humans. Ann Occup Hyg (2008) 52:481–95.
Crump KS, Chen C, Fox JF, et al. Reply. Ann Occup Hyg (2009) 53:184–9.
Elsevier. Statement from Michael Hansen, CEO Of Elsevier's Health Sciences Division, regarding Australia based sponsored journal practices between 2000 and 2005 (2009) Available at http://www.elsevier.com/wps/find/authored_newsitem.cws_home/companynews05_01203. Accessed 29 May 2009.
Finkelstein MM. Asbestos fibre concentrations in the lungs of brake workers: another look. Ann Occup Hyg (2008) 52:455–61.
Finkelstein MM. Reply. Ann Occup Hyg (2009) 53:192–3.
Grant B. (2009) Elsevier published 6 fake journals. TheScientist.com, 7 May 2009. http://www.the-scientist.com/blog/display/55679/. Accessed 29 May 2009.
Hansard. House of Commons daily debates, 24 October 2002: column 486 (2002) London: United Kingdom Parliament. Available at http://www.publications.parliament.uk/pa/cm200102/cmhansrd/vo021024/debtext/21024-28.htm. Accessed 16 May 2009.
Hodgson JT, Darnton AJ. The quantitative risks of mesothelioma and lung cancer in relation to asbestos exposure. Ann Occup Hyg (2000) 44:565–601.
International Agency for Research on Cancer. Biological effects of asbestos. Report of the advisory committee on asbestos cancers to the director of the International Agency for Research on Cancer. Ann Occup Hyg (1973) 16:9–17.
Liddell FDK. Magic, menace, myth and malice. Ann Occup Hyg (1997) 41:3–12.
Marquart H, Heussen H, Le Feber M, et al. ‘Stoffenmanager’, a web-based control banding tool using an exposure process model. Ann Occup Hyg (2008) 52:429–41.
Marquart H, Northage C, Money C. Exposure scenarios for workers. J Expo Sci Environ Epidemiol (2007) 17:S16–25.[CrossRef][Web of Science][Medline]
McCullogh J, Tweedale G. Defending the indefensible: the global asbestos industry and its fight for survival (2008) Oxford: Oxford University Press.
McDonald JC. Unfinished business: the asbestos textiles mystery. Ann Occup Hyg (1998) 42:3–5.
Michaels D. Doubt is their product: How industry's assault on science threatens your health (2008) New York: Oxford University Press.
Ogden T. Book review: magic mineral to killer dust. Ann Occup Hyg (2000) 44:654–5.
Ogden T. Canada, chrysotile, and the search for truth. Ann Occup Hyg (2008) 52:673–4.
Ogden TL. Canadian chrysotile report released—at last. Ann Occup Hyg (2009) 53:307–309.
Raad voor Gezondheidsonderzoek (RGO). Van gegevends versekerd (2008) The Hague, The Netherlands: Health Council of the Netherlands. RGO report no. 58 (in Dutch; executive summary in English).
Roggli VL, Sporn TA, Case BW, et al. Comments on asbestos fibre concentrations in the lungs of brake workers: another look. Ann Occup Hyg (2009) 53:191.
Siegel M. Is the tobacco control movement misrepresenting the acute cardiovascular health effects of secondhand smoke exposure? An analysis of the scientific evidence and commentary on the implications for tobacco control and public health practice (2007) Epidemiologic Perspectives Innovations; 4: 12; doi:10.1186/1742-5573-4-12.
Smith R. The trouble with medical journals (2006) London: Royal Society of Medicine Press.
Tielemans E, Noy D, Schinkel J, et al. Stoffenmanager exposure model: development of a quantitative algorithm. Ann Occup Hyg (2008a) 52:443–54.
Tielemans E, Schneider T, Goede H, et al. Conceptual model for assessment of inhalation exposure: defining modifying factors. Ann Occup Hyg (2008b) 52:577–86.
Tweedale G. Magic mineral to killer dust: Turner & Newall and the asbestos hazard (2000) Oxford: Oxford University Press.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||