Intravenous magnesium for acute myocardial infarction
ACP J Club. 1992 Sept-Oct;117:62. doi:10.7326/ACPJC-1992-117-2-062
To the Editor
In the May issue of ACP Journal Club, the commentary Intravenous Magnesium for Acute Myocardial Infarction: A Meta-analysis (1) on the overview of randomized trials of intravenous magnesium in myocardial infarction gives three arguments against believing the results, none of which is convincing.
1. Lack of mechanism. Imagine Galileo dropping pairs of balls: 20 pairs are dropped and all pairs hit the ground together. Because of his earlier intuition, however, he concludes that heavier things fall faster because there is no mechanism to make them fall at the same speed!
Causal reasoning (mechanism) should influence our belief before an experiment, but strong, unbiased evidence should eventually overwhelm this initial belief. With the magnesium trials, we must conclude that either the experiments were biased (not so: they were mostly blinded, randomized trials with a high rate of follow-up) or that the evidence here (P < 0.001) is strong enough to overcome our earlier hesitancy. At perhaps, P = 0.02 or P = 0.01, I might still be skeptical, but P < 0.001 from high-quality studies convinces me.
The meta-analysis does provide some experimental evidence of a mechanism: a reduction in ventricular arrhythmias. Although the mechanism of this mechanism is unknown, it is usual for some details of causal pathways to be missing.
2. Publication bias. This is certainly the Achilles heel of meta-analysis. However, the "file-drawer n" here is 21 (2); this is the number of negative unpublished studies of similar size needed to make the results no longer statistically significant. Thus, three fourths of all the studies on the problem would need to be unpublished and negative to negate these results. This situation seems unlikely.
3. Too effective. Reworded, the final argument says, "It works so well, I cannot believe it works at all." Would the commentator really find a less strong result more convincing? Perhaps he meant to say that, if we took into account any unpublished negative studies and also our initial skepticism, the effect, although still statistically significant, would be smaller (e.g., a 30% to 40% reduction in mortality instead of 55%).
The first two of these arguments are genuine concerns with any meta-analysis, but in this case the quality and strength of the evidence are enough to overcome them. There is a striking resemblance to the history of streptokinase here. Positive meta-analyses in 1982 and 1985 did not alter practice; the ISIS-2 and GISSI studies were required to provide results substantially the same as the meta-analyses to persuade clinicians. Fortunately, ISIS-4 should mean the delay for magnesium will be shorter.
Paul Glasziou, MBBS, PhD
The University of Queensland Herston
Therapies are most credible when supported by several lines of evidence from studies of different design, including basic pathophysiologic investigations (i.e., mechanism), epidemiologic studies, and the results of clinical trials. In a Bayesian framework, the first two types of study would provide the basis for the prior (pretrial) likelihood that the therapy works, and clinical trial results would permit revision of the prior probability. The lack of solid pathophysiologic and epidemiologic evidence about the role of magnesium in acute myocardial infarction led me to have a low prior belief in its efficacy. The evidence from the meta-analysis raised my prior probability, but my post-study belief was still not high. I also discounted the meta-analysis result as "too good to be true" because most effective therapies lower relative risk by 20% to 25% at best, and the 55% risk reduction attributed to intravenous (IV) magnesium was (literally) incredibly high. Interestingly, a randomized trial has just reported a risk reduction of 24% by IV magnesium in 2316 patients (1)—a more credible figure but one that is outside the 95% confidence limit of the meta-analysis of 1301 patients in 7 small studies.
I think meta-analysis can be valuable, but I have both practical and philosophical concerns about the technique. One problem that has been mentioned is publication bias, which is quite real. Another problem is heterogeneity among studies in design and results. Studies differ in design in many ways, most importantly in the therapeutic regimen, which may be as different as "apples and oranges." The regimens for IV magnesium varied enough to cause concern. The results of studies included in the meta-analysis may be heterogeneous, but (ironically) formal statistical tests for heterogeneity have low power! It is good practice to graph the results of all studies and allow the reader to judge their consistency. The graphical display of trials of intravenous magnesium for acute myocardial infarction did not look particularly homogeneous to my eye. Finally, the calculated level of statistical significance in a meta-analysis may be very misleading because meta-analysis is a post-hoc data analysis in which the analyst is well aware of the trends in the literature before conducting the analysis.
Does meta-analysis eliminate the need for large clinical trials with adequate power? I do not think so; too many questions about the technique still exist.
Mark A. Hlatky, MD
Stanford University School of Medicine
1. Hlatky MA. Commentary on "Intravenous Magnesium for Acute Myocardial Infarction: A Meta-analysis." ACP J Club. 1992 May-Jun:68 (Ann Intern Med. vol 116, suppl 3). Comment on: Teo KK, Yusuf S, Collins R, Held PH, Peto R. BMJ. 1991;303:1499-503.