The Ups and Downs of Journal Impact Factors
1 Editor-in-Chief, Annals of Occupational Hygiene, British Occupational Hygiene Society, 5/6 Melbourne Business Park, Pride Park, Derby DE24 8LZ, UK
2 Assistant Editor, Annals of Occupational Hygiene, 3904 Pocahontas Avenue, Cincinnati, OH 45227, USA
* Author to whom correspondence should be addressed. Tel: +44 1235 534380; e-mail: ogden{at}ogs.org.uk
ABSTRACT
The journal impact factor (JIF) for The Annals of Occupational Hygiene rose 68% between 2005 and 2006. JIFs are widely publicized and may influence subscriptions and where authors submit papers, so they are much discussed in the publishing world. But although they tell us something about a journal's citation performance, their shortcomings mean that they are poor general indicators of journal quality, and worse guides to the quality of authors and their institutions. The shortcomings include the following. (i) The two cited years may completely misrepresent the total current citation rate for the journal. (ii) The short citation period (1 year) results in many papers not contributing to the JIF, and usually two-thirds or more of the JIF depend on the most-cited 25% of papers. (iii) The JIF of the journal where a paper is published is therefore a very poor guide to the paper's citation performance or the success of the author. Citation counts more specific to the author are much better. (iv) The JIF depends strongly on the subject of the journal, even within the published categories. (v) Statistical analysis shows that the relative standard deviation of year-to-year variation of a JIF for a journal with a JIF 
1.5 is likely to be between 10 and 20%, on top of any longer trend. Quotation of JIFs to three decimal places is therefore meaningless, and, for a journal like Annals of Occupational Hygiene, a single annual change of 70% could easily be due to a chance shift from a negative to positive fluctuation. (vi) The citations counted are not only of individual papers, so it is difficult to reproduce the JIF calculation. (vii) The selection of journals has been criticized, for example, the alleged emphasis on American- or English-language publications. This journal's JIF does not noticeably influence the number of papers submitted to this journal, although it may influence some important authors. JIFs in our field seem to be increasing by 5% a year, perhaps partly because of the various measures which editors can take to improve them, some of which represent genuine improvements to the journal, but some of which are unethical.
Keywords: citations • journal impact factor • journal quality • research assessment
INFORMED AND CAREFUL USE?
Back in the old days, editors only had to attract papers, get them peer reviewed, and publish the ones which they felt were worth it. Libraries bought the journals, researchers read the papers and perhaps everyone was happy. But the need to demonstrate efficiency and value for money in relation to competitors brought numerical measures into scientific publication, and prominent among these is the journal impact factor (JIF). Smith (2006), a former editor of the British Medical Journal, writes of many editors and academics being obsessed with JIFs. ‘Now editors break open a bottle of champagne if their impact factor rises a tenth of a decimal point, or burst into tears if it falls. They build their editorial strategies around increasing their impact factors. Authors, meanwhile, can quote the impact factors of the major journals, and use them when deciding where to submit their papers.' This journal is affected by this. However, in 2006, our JIF rose 68% to 1.92, by far the highest we have ever achieved, so this gives us an opportunity to comment on this ‘obsession’ insofar as it affects journals like the Annals, without being accused of having sour grapes.
JIFs were devised in the 1960s to help select journals for inclusion in the Science Citation Index (Garfield, 2005), but their use has grown vastly. They are now published annually by Thomson Scientific, who caution:
Informed and careful use of these impact data is essential. Users may be tempted to jump to ill-formed conclusions based on impact factor statistics ... (Thomson Scientific, 2005).
However, the attraction of having a simple single number to judge complex issues is too great, and jumping to ill-informed conclusions based on impact factors seems to be an international sport.
An explanation of the main feature of JIFs is given by Thomson Scientific (2005). Taking Annals of Occupational Hygiene and 2006 as an example, our JIF is given by
 |
50. Inevitably, there are complications to the calculation. Thomson Scientific has to decide what a ‘paper’ is among the very wide range of items which journals publish. The definition is broader for citing items than for cited. An editorial, for example, will probably not be included in B, but the papers which the editorial refers to could be included in A for the journal where the papers appeared. The method of counting means that a citation just has to be to a journal and a year, not necessarily to an identified paper (Adam, 2002). This produces the first problem with JIFs—the calculation is not transparent, because the JIF calculation cannot usually be reproduced by adding up the number of citations of individual items for the periods in question. There is a very extensive literature on citation analysis, and new computational tools are permitting new insights into such things as what citations can tell about the relationships of research fields (e.g. Bornmann et al., 2007). At the same time, Thomson is having to develop policies to deal with the rapidly changing publishing world, with early publication on the web becoming normal.
We will consider various other problems with JIFs in turn.
SHORTCOMINGS OF JIFs
Years 1 and 2 are not an index of total citations
The philosophy behind the JIF is that the number of citations of a paper by other scientific papers is a measure of its usefulness to the scientific world as a whole, and by focusing on the most recent years we get an idea of the journal's current rather than past performance. However, for most journals, citations are spread over many years, and concentration on a small fraction of the total number of citations gives a misleading impression of the journal's citation performance, and leads to inevitable statistical fluctuation from year to year. We consider fluctuations below, but flag up now the possibility that a substantial increase or decrease in a JIF may have a large random component. In this commentary, we therefore quote JIF values to two decimal places rather than the three places in the Thomson published values.
For a journal like ours, the citation rate of a paper usually increases for 3 or 4 years after publication, and then slowly declines. Figure 1 illustrates this for 12 exceptionally well-cited papers published in the Annals between 1994 and 2003. Figure 2 shows the same effect, this time looking at the citations in 2005 and 2006 of all Annals papers, in terms of the age of the paper. In both of these figures, year 0 represents citations in the year of the paper's publication, year 1 the year following publication and so on.
|
|
As explained above, it is only years 1 and 2 which contribute to the JIF, and clearly they only tell part of the story. In 2006, the JIF years contributed only 23% of the papers cited from Annals volumes in the most recent 10 years, and in 2005 only 19%. For the papers averaged in Fig. 1, years 1 and 2 contribute
18% of the first 10 years total. Of course, there are many citations of papers >10 years old, and if these are taken into account then the JIF-year percentages are even lower. Marx and Cardona (2003) looked at the 100 most-cited papers published in Solid State Communications in 1975, and found that only 9% of the citations up to 2002 had contributed to the JIF. It would not matter that the JIF covers only a small fraction of the total papers cited if the most recent years were a consistent index of citation performance for all years, but this is not the case. As well as the JIF, Thomson Scientific reports the citation half-life. For 2006 for our journal, for example, this is the number of publication years we have to go back to include half of the total number of Annals papers cited in 2006. Figure 3 illustrates two hypothetical journals with different half-lives, using log-normal curves to simulate the sort of shape seen in Figs 1 and 2. Summing over the years shown, Journal B had 564 citations and Journal A 474. In terms of the total number of citations, therefore, Journal B has a bigger impact, and the difference is of course more marked if we include citations of older papers. However, the JIF of B is lower than A (1.73 against 2.05 if the journals both published 100 papers in years 1 and 2), so choosing the journal by its JIF would give the wrong decision if an author were interested in the long-term influence of the journal measured by its citation rate. The examples shown have half-lives of 4.3 and 8 years. This difference is fairly modest: for Thomson Scientific's Public, Environmental and Occupational Health journal category, which includes the Annals, 2006 half-lives ranged from 2.2 to >10 years. Thomson publishes not only the JIF and half-life but also the total number of citations in the year, and the number of articles published, which can be used to estimate the average number of citations per paper historically, if the journal size has not changed dramatically. Unfortunately, the tendency is to ignore all this and concentrate exclusively on the JIF, but if the longer term performance of the journal is of interest, this can give very misleading conclusions.
|
The JIF depends on a few papers
The majority of papers are not cited many times in any particular year, and so the JIF is strongly influenced by a small percentage of published papers. Figure 4 and Table 1 illustrate this for our 2005 and 2006 JIFs. Interestingly, our 2005 JIF depends more on a few highly cited papers than our higher 2006 JIF, which is much more influenced by papers cited a moderate number of times. This greater spread is healthier for the journal. However, in both cases a high proportion of papers are not cited at all. This is simply because 1 year is a short period in which to count citations; most papers will be cited in due course. [For example, 14 out of 55 papers we published in 2000 (25%) were not cited in time to contribute to a JIF, but only 3 of them (5%) remained uncited in Thomson database by the end of 2007.] The short period is obviously chosen to give immediacy to the figure, but it means that much of the journal content is disregarded, and leads to small numbers and, once again, large statistical variation in the JIF. This highly skewed distribution is apparently true for all journals. Table 1 shows that 60–80% of our JIF is produced by the most-cited 25% of papers. Very similar results were reported for three biochemical journals by Seglen (1997), and Marx and Cardona (2003) gave a similar result for Solid State Communications. In a field close to ours, Burdorf and Viikari-Juntara (2007a) gave a similar figure for citations in 2005 and 2006 (i.e. 2 years) of 2004 papers in Scandinavian Journal on Work, Environment and Health. Nature (2005), which has a JIF of
30, reported an even more highly skewed distribution for its 2004 JIF, saying that 89% was contributed by 25% of its papers.
|
|
The JIF of where a paper is published is not a good guide to that paper's value
One of the ‘ill-formed conclusions’ that is sometimes drawn from JIFs is that a paper published in a high-JIF journal represents better work than one published in a low-JIF journal. This belief has led to instructions to university staff to concentrate on high-JIF journals, and to the use of the JIF as a measure in assessment of researchers and institutions. In the article already quoted, Smith (2006) writes that ‘a major problem has arisen because an article published in a journal has been deemed by those assessing research to assume the impact factor of the journal'. Seglen (1997) illustrated the poor correlation of citation numbers for individual papers with the JIF of the journal in which they were published. This is also illustrated by Fig. 4. The change in our JIF from 1.14 in 2005 to 1.92 in 2006 certainly does not tell us anything about individual papers. If a paper's citation rate is seen as a measure of its quality or interest, then it is the paper's citation rate that should be measured, not the JIF of the journal where it is published, because this correlates so poorly with the performance of individual papers.
There is perhaps a shift away from this misuse of JIFs. The 2008 Research Assessment Exercise by the UK higher education funding councils states that ‘No panel will use JIFs as a proxy measure for assessing quality’ (RAE, 2006). Better uses of citations are being explored, notably the h-index: a researcher has an h-index of n if he or she has co-authored n papers with at least n citations (Ball, 2007; Hirsch, 2007). This index can be applied to groups as well as individual researchers—and even journals! Its strengths and weaknesses have been reviewed by Bornmann and Daniel (2007). Some universities still favour JIFs, but the sooner they adopt better and broader measures of staff performance the better.
A journal's JIF depends strongly on subject
Amin and Mabe (2000) present an analysis of subject dependence for the 1998 JIFs. Classified by journal subject, mean JIFs varied from <0.5 for Mathematics and Computer Sciences to >3 for Fundamental Life Sciences. These averages are likely to have increased since, but the differences remain.
Thomson Scientific publishes JIFs in subject categories which group together journals which tend to cite one another. The effect of subject can be seen even within categories, and Table 2 illustrates this for one of the categories which includes Annals of Occupational Hygiene. One university department which contributes good papers to this journal has apparently been told to publish three-quarters of its papers in journals in the top third of Thomson categories. The problem is that there are usually no occupational medicine or occupational hygiene journals in the top third of our category, although in the 2006 rankings Occupational and Environmental Medicine achieved this, ranking 27 out of 98 (the Annals ranked 40).
|
The subject dependence clearly causes problems if JIFs are used to compare authors or departments. This problem applies to other bibliometric measures.
There is a large random component in JIF changes
The relatively small number of papers contributing to the JIF means that there is a large random component in the variation from year to year. Unfortunately, there is no complete theoretical study of this, but there are three pieces of evidence which give us some idea of the variability.
(1) Amin and Mabe (2000) grouped 4000 journals by the number of papers they publish in a year, and looked at the change in JIF from 1997 to 1998. They found a strong dependence on size for small journals, with a mean annual change of about plus or minus 30% for a journal of the Annals’ size (60–70 papers per year). If the annual changes were normally distributed, this would correspond to a relative standard deviation (SD) of 34%. This is a useful piece of experimental evidence, but clearly the changes will include any real improvement or deterioration in the journal, so the 34% gives us an upper limit to the random variability from year to year. In grouping journals in this way, Amin and Mabe implicitly assumed that the relative SD does not depend on the JIF.
Amin and Mabe also present theoretical values, but unfortunately do not explain how they were derived, only saying that they estimated the random component by considering what would be expected for a small biased sample taken from a finite but large pool of available articles. Their theoretical value is very close to their observed variability, implying that any component due to journal improvement or deterioration is small. They do not mention any dependence of the relative SD on the JIF.
(2) In the Appendix 1, we estimate the expected variability on very simple assumptions. If we assume that there is a probability P that a paper published anywhere in 2006 cites the Annals from 2004 or 2005, the actual number of citations will follow a binomial distribution, and we can estimate the variance of the number of citations in terms of the observed value of the JIF, J. The estimated value R of the relative SD of the JIF is given by
 | (1) |
135, the predicted relative SD is 7%. Considering the assumptions, this is likely to be a low estimate, so the true random component is therefore likely to lie between this 7% and the 34% observed by Amin and Mabe (2000).
(3) Figure 5 illustrates that changes in JIFs in our field in the last 10 years or so can be approximated by fluctuations about linear trends. Details for a total of 10 journals are given in the supplementary data available at Annals of Occupational Hygiene online. Nine of these have been increasing. For eight of these, the fitted linear trends predict a 2005 to 2006 increase of between 2.3 and 7.2%; the ninth gives 15.3%. The 10th journal amalgamated with another in 2004, but before then its JIF was decreasing at 5% a year (the merger seems to have solved the problem). Calculations of the variance of the fluctuation about the linear trend are given in the supplementary data available at Annals of Occupational Hygiene online, and the SDs given in Table 3 are the square roots of these. This is divided by the average JIF for 2001–2004 to give a relative SD (these years were chosen because all the journals had JIFs for those years). It can be seen from the table that the relative SDs vary between 9 and 18%. This is probably the best estimate of the random year-to year variation. These values do not correlate significantly with those calculated from 
equation (1) (data not shown), but in every case the Table 3 value is higher, confirming that the simple approach taken in the Appendix 1 will underestimate the variance. Although we quote relative SDs, it may be that the absolute variation is a better measure, and these percentage figures should not be applied to journals with JIFs much outside the range shown in Table 3.
|
|
To sum up, the relative SD of the year-to-year fluctuations of the JIF cannot average less than that given by
equation (1), which for most journals in our field will be >5%. It will be less than the overall changes observed by Amin and Mabe, which correspond to a SD of 34% for a journal of the Annals’ size. The values in Table 3 are probably the best estimate, roughly between 10 and 20% for JIFs in the range shown.
Returning to the case of Annals of Occupational Hygiene, if our current ‘true’ JIF is 1.5 (the average of the 2005 and 2006 values), then we would expect to see fluctuations between 1 and 2 on top of any long-term trend. The 68% change between 2005 and 2006 amounts to a change from 1.6 SDs below the mean to 1.8 SDs above. As often happens, we can identify some causes of changes even though they can be statistically described. For example, the year included an overview of the Annals first 50 volumes (Ogden, 2006), which among its 144 references included 29 to the Annals in 2004 and 2005, and this will have contributed 0.21 to the JIF of 1.92. (This is a clear example of a case where the single-citation approximation in the simplified calculation given in the Appendix 1 is not valid.) However, whatever the reason, the increase is clearly within a reasonable range of statistical variation. The three decimal places to which JIFs are usually quoted are therefore meaningless—Smith (2006) refers to them as an ‘absurdity'—and even the second decimal place does not mean much.
Other problems
JIFs have been criticized because of the selection of cited journals from the world's scientific literature, with allegations of over-representation of certain publishers and English-language publications (Kaltenborn and Kuhn, 2004). This reflects but also must strengthen the dominance of English. The only epidemiology journal in the bottom class in Table 2 is the only non-English language epidemiology journal in the category.
EFFECT ON SUBMISSIONS
Editors are interested in whether changes in their journal's JIF will affect submission rate of new papers. There are at least two major problems in testing this. (i) It is notoriously risky to compare two time series, especially when both are showing long-term trends. (ii) It is not our JIF on its own which might be important, but our JIF in comparison with other journals where an author might submit the paper, and which journals are competitors varies from paper to paper. However, as a rough test to see if there might be any gross effect, we looked at the 10 years 1997–2006, and in each year calculated the difference between the JIF of the Annals and the average JIF of the three journals with the highest JIFs from a set of journals which might compete for our papers. The journals we compared ourselves with were American Industrial Hygiene Association Journal, American Journal of Industrial Medicine, International Archives of Occupational and Environmental Health, International Journal of Environmental Health, Journal of Environmental Monitoring, Journal of Exposure Science and Environmental Epidemiology, Journal of Occupational and Environmental Hygiene, Journal of Occupational and Environmental Medicine, Occupational and Environmental Medicine and Scandinavian Journal of Work, Environment and Health. Not all of these were published in every year. JIFs are published in May or June, so we compared this JIF difference with the number of submissions we received in the 12 months starting in the July after JIF publication. The results are shown in Fig. 6. Clearly, there is no noticeable effect. Some authors may have been influenced, and their papers may have been important ones, but the effect on the total number of submissions was not large enough to be noticeable.
|
HOW TO IMPROVE YOUR JIF
In our field, and perhaps in most others, JIFs are drifting upwards. The linear trend has already been mentioned. The supplementary data available at Annals of Occupational Hygiene online shows that for individual journals, the trends predict 2005–2006 increases between 2.7 and 7.2%, with outliers at –5.6% and +15.3%. Figure 7 compares our JIF with the average of the 10 other journals mentioned in the previous section (but remember that not all of these were published for the whole period), and an average change of 5% a year. The increases may mean that papers are citing more literature on average, or may arise because editors have taken steps to improve their JIFs. For the benefit of new editors who do not realize how to do this, we offer the following ideas. We have arranged them in rough order of decreasing ethicality.
- (1) Improve your publication time. This should work because most specialist journals are cited most by other papers in the same journal. The citing papers must be published in the following year or the year after that in order to affect the JIF. There is usually a year or two's delay before a new paper is absorbed by the scientific community and cited in new submissions (Figs 1 and 2), and if these are delayed further in the editorial process, they may miss publication in the first citing year or even the second. Improving publication time is obviously in everyone's interests anyway.
- (2) Commission reviews in fields where your journal has published a lot. Papers cited by the review will contribute to the JIF if they are in the right year, and reviews themselves attract more citations than ordinary research papers. However, the Committee on Publication Ethics (2004) has condemned this practice if the citations are deliberately biased to the journal in question.
- (3) If you have a submission that is likely to be well cited, publish it in the first issue of the year rather than the last—this gives it almost an extra year to come to the attention of authors and to be cited in a year that affects the JIF.
- (4) For the same reason, publish big issues near the start of the year and smaller ones near the end.
- (5) If you have a submission that cites many of your journal's papers, publish it in the year that maximizes the effect on the JIF.
- (6) Consider sympathetically submissions that cite papers which will improve your JIF, and less sympathetically papers with citations that will improve your competitors’ JIFs.
- (7) Suggest to authors that they cite particular papers which will improve your JIF. Yes, some editors have done this. The Committee on Publication Ethics (2003) condemned this practice.
- (2) Commission reviews in fields where your journal has published a lot. Papers cited by the review will contribute to the JIF if they are in the right year, and reviews themselves attract more citations than ordinary research papers. However, the Committee on Publication Ethics (2004) has condemned this practice if the citations are deliberately biased to the journal in question.
|
SO WHAT DO WE DO?
Consider this scenario. A reputable journal receives a paper proposing a quality measure for something in its field, let us say the performance of analytical laboratories. The editor gives it an initial read through, and notices that (i) some of the data used in the measure are not generally available, even to the laboratory concerned, so the calculation cannot be reproduced; (ii) the measure uses a relevant parameter of performance, but in the interests of immediacy uses less than a quarter of the relevant results; (iii) within this quarter, the measure depends very strongly on a few of the results from the laboratory; (iv) the measure depends strongly on what range of substances the laboratory is analysing; (v) there is a large random variation in the measure, giving it a relative SD of 10 to 20%; (vi) despite this, the measure is given to four significant figures, and used to distinguish laboratories by that fourth place.
What will the editor do with the paper? He or she will probably reject it immediately, as having problems which are too obvious to justify peer review. Nevertheless it is a measure like this that is being used to judge journals.
So what are we to do about JIFs? Keep on plugging the message: JIFs do tell us something (total citation rates tell us a lot more), but ‘informed and careful use’ is essential, as Thomson Scientific (2005) says. For a journal like the Annals, year-to-year changes of 35% could well be due to chance. Even this journal's 68% increase in 2006 is consistent with the long-term trend.
Of course, journals have to be marketed. Our publisher puts JIFs on its websites to four significant figures like everyone else. (Apparently the site software demands that the JIF is entered to three decimal places, but we have asked for the last one to be made zero in this journal's site.) BioMed Central issued a press release on the 2006 JIFs which said, ‘Impressive impact factors prove that BioMed Central's Open Access journals are high quality’ (BioMed Central, 2007). Prove? Perhaps this was written in ignorance, but fortunately Biomed Central's editors have a better idea of what ‘prove’ means than whoever wrote this!
Despite the pressure, we editors have to do what we can to publish papers that are scientifically sound and advance knowledge and its application, and not just go for the numbers. This journal's first priority should be publishing papers which reduce ill-health, with potential citation rate being very much a secondary consideration. One occupational medicine journal has deliberately chosen not to play the JIF game, because they have decided that their duty is to the practising occupational physicians that read the journal. That is a sound decision. A recent editorial in the Scandinavian Journal of Work, Environment and Health commented that in occupational health the ‘societal impact of research ... should be more appreciated than receiving many citations’ (Burdorf and Viikari-Juntura, 2007b). Perhaps the move away from JIFs in RAE (2006) and the adoption of other citation measures such as the h-index (Bornmann and Daniel, 2007) show a weakening of interest in JIFs. Let us hope so. Returning to Smith's (2006) picture of editors opening a bottle of champagne to celebrate their JIF increase, if you feel like joining us, drink to the decline of ill-informed and careless use of this unsatisfactory measure.
APPENDIX 1
Simple estimate of the likely random fluctuation of JIFs
Consider a particular journal which we will call A. Suppose that A published NA papers in a particular 2-year period, and that there were N papers published in the whole scientific literature in the following year, the year of the JIF. Assume that each of the N papers has a probability P of citing one of the NA papers from journal A, and ignore the possibility that it cites more than one of the NA. If this results in n papers citing one of the NA the JIF is given by
 |
With n distributed binomially, the expected number of citations
 |
And, from the properties of the binomial distribution, the variance
 |
If P << 100%, meaning that there is a low probability that the average paper in the world's scientific literature will cite a journal A paper from the candidate years, then
 |
Thus,
 |
Thus from this very simple approach we obtain
 |
Now suppose E[JIF] = 1.00, then, using a normal approximation, we would expect 95% of the estimates of JIF to lie in the range
 |
is shown in Fig. 8. Remember that NA is the number of papers published in a 2-year period, so for a journal like the Annals NA 
135. Also, for a E[JIF] of 1.5, which might better characterize the Annals than E[JIF] = 1, the expected variation range of JIF annual values would be 
times the values shown in Fig. 8, and the SD of the annual values, if all the fluctuation were due to this random variation, would be 0.105, or a relative SD of 7%. This is a much smaller than from Amin and Mabe's unexplained model.
|
FOOTNOTES
The free full text of this article can be found in the online version of this issue.
Received September 12, 2007; in final form December 31, 2007
REFERENCES
Adam D. The counting house. Nature (2002) 415:726–8.[Medline]
Amin M, Mabe M. Impact factors: use and abuse. Perspective in publishing no. 1 (2000) Elsevier Science, Oxfordshire, UK. Available at http://www1.elsevier.com/homepage/about/ita/editors/perspectives1.pdf. Accessed 29 June 2007.
Baldwin PEJ, Maynard AD. A survey of wind speeds in indoor workplaces. Ann Occup Hyg (1998) 42:303–13.
Ball P. Achievement index climbs the ranks. Nature (2007) 448:737.[CrossRef][Medline]
BioMed Central. Open access journals get impressive impact factors. Press release, 23 June 07 (2007) Available at http://www.biomedcentral.com/info/about/pr-releases?pr=20050623. Accessed 29 June 2007).
Bornmann L, Daniel H-D. What do we know about the h index? J Am Soc Inf Sci Technol (2007) 58:1381–5.[CrossRef]
Bornmann L, Leydesdorff L, Marx W. Citation environment of Angewandte Chemie. Chimia (2007) 61:104–9.[CrossRef][Web of Science]
Burdorf A, Viikari-Juntara EV. Bibliometric analysis of the journal Scandinavian Journal of Work, Environment & Health: results from the past 10 years. Scand J Work Environ Health (2007a) 33:318–9.[Web of Science]
Burdorf A, Viikari-Juntara EV. Focus and future of occupational health journals. Scand J Work Environ Health (2007b) 33:241–3.[Medline]
Burstyn I, Kromhout H, Kauppinen T, et al. Statistical modelling of the determinants of historical exposure to bitumen and polycyclic aromatic hydrocarbons among paving workers. Ann Occup Hyg (2000) 44:43–56.
Committee on Publication Ethics. Cases: manipulation of a journal's impact factor (2003) http://www.publicationethics.org.uk/reports/2003/14y.pdf/view. Accessed 13 July 2007.
Committee on Publication Ethics. Cases: using annual reviews to massage the impact factor (2004) http://www.publicationethics.org.uk/reports/2005/using-annual-reviews-to-massage-the-impact-factor-2004/. Accessed 13 July 2007.
Donaldson K, Borm PJA. The quartz hazard: a variable entity. Ann Occup Hyg (1998) 42:287–94.
Douwes J, Thorne P, Pearce N, et al. Bioaerosol health effects and exposure assessment: progress and prospects. Ann Occup Hyg (2003) 47:187–200.
Fenske RA. Dermal exposure assessment techniques. Ann Occup Hyg (1993) 37:687–706.
Fubini B. Surface chemistry and quartz hazard. Ann Occ Hyg (1998) 42:521–30.
Garfield E. The agony and the ecstasy—the history and meaning of the Journal Impact Factor. Presented at the International Congress on Peer Review and Biomedical Publication, Chicago, September 2005 (2005) Available at http://garfield.library.upenn.edu/papers/jifchicago2005.pdf. Accessed 28 June 2007.
Hirsch JE. Does the h-index have predictive power? (2007) eprinthttp://arxiv.org/PS_cache/arxiv/pdf/0708/0708.0646v2.pdf. Accessed 30 November 2007.
Hodgson JT, Darnton A. The quantitative risks of mesothelioma and lung cancer in relation to asbestos exposure. Ann Occup Hyg (2000) 44:565–601.
Jongeneelen FJ. Benchmark guideline for urinary 1-hydroxypyrene as biomarker of occupational exposure to polycyclic aromatic hydrocarbons. Ann Occup Hyg (2001) 45:3–13.
Kalterborn K-F, Kuhn K. The journal impact factor as a parameter for the evaluation of researchers and research. Rev Esp Enferm Dig (2004) 96:460–76.[Web of Science]
Kenny LC, Aitken R, Chalmers C, et al. A collaborative European study of personal inhalable aerosol sampler performance. Ann Occup Hyg (1997) 41:135–53.
Kromhout H, Swuste P, Boleij JSM. Empirical modeling of chemical exposure in the rubber-manufacturing industry. Ann Occup Hyg (1995) 38:3–22.[CrossRef][Web of Science]
Marx W, Cardona M. The impact of Solid State Communications in view of the ISI Citation data. Solid State Commun (2003) 127:323–36.[CrossRef]
Nature. Editorial: not-so-deep impact. Nature (2005) 435:1003–4.[CrossRef][Medline]
Ogden TL. Annals of Occupational Hygiene at Volume 50: many achievements, a few mistakes, and an interesting future. Ann Occup Hyg (2006) 50:751–64.
RAE. Research Assessment Exercise, 2008. Generic statement on criteria and working methods, para 32 (2006) http://www.rae.ac.uk/pubs/2006/01/docs/genstate.pdf. Accessed 29 June 2007.
Rappaport SM, Lyles RH, Kupper LL. An exposure-assessment strategy accounting for within-worker and between-worker sources of variability. Ann Occup Hyg (1995) 39:469–95.
Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ (1997) 314:497. http://www.bmj.com/cgi/content/full/314/7079/497. Accessed 30 November 2007.
Smith R. Commentary: the power of the unrelenting impact factor—is it a force for good or harm? Int J Epidemiol (2006) 35:1129–30. http://ije.oxfordjournals.org/cgi/reprint/35/5/1129.pdf. Accessed 29 June 2007.
Thomson Scientific. The Thomson Scientific Impact Factor (2005) http://scientific.thomson.com/free/essays/journalcitationreports/impactfactor/. Accessed 28 June 2007.
Tielemans E, Kupper LL, Kromhout H, Heederik D, Houba R. Individual-based and group-based occupational exposure assessment: some equations to evaluate different strategies. Ann Occ Hyg (1998) 42:115–9.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  J. A. Dempsey Impact factor and its role in academic promotion: A statement adopted by the International Respiratory Journal Editors Roundtable J Appl Physiol, October 1, 2009; 107(4): 1005 - 1005. [Full Text] [PDF]
|
|
|
|
 |
|
 D. Loomis OEM's impact factor continues moving upward Occup. Environ. Med., October 1, 2009; 66(10): 641 - 641. [Full Text] [PDF]
|
|
|
|
 |
|
 K. B. Adler Impact Factor and its Role in Academic Promotion Am. J. Respir. Cell Mol. Biol., August 1, 2009; 41(2): 127 - 127. [Full Text] [PDF]
|
|
|
|
 |
|
 A. R. Leff Impact Factor and its Role in Academic Promotion Proceedings of the ATS, August 1, 2009; 6(4): 337 - 337. [Full Text] [PDF]
|
|
|
|
 |
|
 T. Ogden Annals of Occupational Hygiene Performance Indicators, 2008 Ann. Hyg., April 1, 2009; 53(3): 199 - 199. [Full Text] [PDF]
|
|
|
|
|
|
T. Ogden Annals of Occupational Hygiene Performance Indicators, 2007 Ann. Hyg., April 1, 2008; 52(3): 151 - 151. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||