Where's the meat in clinical journals?
ACP J Club. 1994 May-June;120:87. doi:10.7326/ACPJC-1994-120-3-087b
To the Editor
Your editorial in the November/December 1993 issue implies that journals can be meaningfully ranked for amounts of clinically useful information by considering the number of their articles abstracted in ACP Journal Club (1) Where's the meat in clinical journals?. It brought to mind a story, doubtless apocryphal, about medieval scholars at Corpus Christi College in Oxford who calculated that the center of the universe was in their college quadrangle. Leg-pulling aside, I think you are making a vital contribution to the medical literature, and I am happy to endorse the ACP Journal Club as a “gold standard” for medical information, with 2 minor reservations.
The first is the fact that your audience cuts across specialties and you, understandably, seek material that may have broad relevance. General journals have the same mission, and it is, therefore, not surprising that they wind up at the top of your batting order. The result is the inevitable trade-off between specificity and sensitivity: In maximizing general appeal, how do you avoid passing over high-quality material in journals aimed at subspecialists?
Second, the articles are read with full knowledge of their source. For example, the same ACP Journal Club issue carries an abstract on a crossover trial assessing the role of soluble dietary fiber in patients who have already been tried on a diet to reduce cholesterol (2). The study was physiologically important but far from being routinely applicable to clinical practice. Had it appeared in The American Journal of Clinical Nutrition rather than New England Journal of Medicine, my guess is that either you would not have abstracted it or, as reviewer/commentator, I might not have recommended that you proceed with it. In short, to the extent that you, your coeditors, and your commentary writers believe this year's rankings, the rankings are likely to be reaffirmed next year.
C. David Naylor, MD, DPhil
University of Toronto
Toronto, Ontario, Canada
The medieval scholars were right! From their unique perspective, their turf was the center of the universe. ACP Journal Club is an unabashedly egocentric publication designed to redefine the universe of new information about medical care from the unique perspective of general internal medicine. There is, however, nothing mystical about the process we use. We publish our criteria in every issue and have run reproducibility tests from time to time that showed that the selection process is highly dependable and not affected by knowledge of the source journal or authors. If next year's list of journals and of their yield from ACP Journal Club's self-centered perspective is not more or less the same, it will be because the journals have changed their content, not because we have changed our perspective or process.
Does this mean that studies reported in these journals that do not meet our criteria are not of good quality or that they are not relevant to other specialties? Of course not! They just do not meet our dual sets of criteria for being both relevant to general internal medicine and at a sufficiently advanced stage of testing to warrant consideration for direct application in clinical practice. Also, it is not surprising that journals with target audiences similar to those of ACP Journal Club provide more articles that meet our criteria—although it is perhaps a bit unexpected that the general medical journals, which have a broader target audience than internal medicine, have so many articles of interest for internists.
Are our criteria perfect? No. In fact, they are rather lenient and pass many studies with findings that are not truly ready for clinical application. Some of these we abstract and others appear as citations in the Other Articles Noted section. For those that we abstract, we ask the commentator to provide guidance on whether the findings are ready for application in practice. We feel that identifying “false-positive” studies—those that look as if they might warrant application but do not hold up when examined closely—is as useful as highlighting the “genuine article.” Indeed, for the article in question, the commentator did exactly this. Thus, we wish to thank Dr. Naylor for his help.
Are we missing valid and applicable studies reported in subspecialty journals? Perhaps, but we have examined over 100 journals now, and, as the table in our editorial showed, the drop-off is steep for articles relevant to internal medicine that meet our criteria. Nevertheless, we welcome nominations from readers of journals that might merit ongoing review for ACP Journal Club. If readers find even one article that meets our criteria in a journal that we do not currently review regularly (list published on the Contents page in each issue), send it to us and we will consider it and review several issues of the journal of origin as well.
Finally, if you are a specialist in some field other than general internal medicine, what can you do to find articles that are relevant and sound for your own discipline? Try using our methodologic criteria with your own content criteria. Better still, lobby the editor of your favorite specialty journal to have someone do this for you and create a section in the journal with abstracts and commentaries of articles that meet the criteria for methods and content. This has already happened in a few disciplines other than general internal medicine, including pediatrics (1) and sports medicine (2).
R. Brian Haynes, MD, PhD
Hamilton, Ontario, Canada
2. Naylor CD. Commentary on “Very high levels of soluble fiber slightly reduced cholesterol levels in adults with high cholesterol levels who were on a low-fat, low-cholesterol diet.” ACP J Club 1993 Nov-Dec:68 (Ann Intern Med. vol 119, suppl 3). Comment on: Jenkins DJ, Wolever TM, Rao AV, et al. Effect on blood lipids of very high intakes of fibre in diets low in saturated fats and cholesterol. N Engl J Med. 1993;329:21-6.