DIGAMI 2 trial post hoc analysis: Lessons in overinterpretation
In their post hoc analysis of the DIGAMI 2 (Diabetes Mellitus Insulin-Glucose Infusion in Acute Myocardial Infarction 2) study, Mellbin and colleagues suggest that insulin therapy after myocardial infarction (MI) may be associated with increased clinical events (not mortality), whereas metformin therapy may be associated with reduced events and sulfonylurea therapy with neutral effects.1 Several factors must be considered, however, before this somewhat dogmatic conclusion regarding insulin treatment can be accepted. This commentary focuses on the risks of overinterpretation of post hoc analyses, especially when they are not prespecified analyses. It also provides a brief review of issues regarding the original DIGAMI study, shedding light on why the authors’ conclusions about insulin must be interpreted cautiously.2 Finally, a brief summary of insulin and glycemic control in post-MI patients is provided to put the authors’ review in context.3,4Guidelines for conducting post hoc analyses
What are the generally acceptable guidelines for conducting post hoc analyses of a clinical trial? First, when the primary outcome of the trial is positive (either for efficacy or safety), then analyses examining differences in subgroups are often performed for age, gender, and medication use to determine whether similar outcomes are seen in all subgroups. Overlapping confidence intervals (CIs) imply no differences among subgroups. Even when “no difference” is demonstrated, far too often there is an overinterpretation of “differences” in point estimates. An even more serious flaw occurs when looking for differences in subgroups when no differences existed in the primary outcome analyses. This is precisely what Mellbin and associates have done. Finally, use of secondary end points in post hoc analyses, especially when they were negative in the original report, is especially prone to overinterpretation. In DIGAMI 2, the primary end point was all-cause mortality.2 In the current report, there are no differences in all-cause mortality between insulin-treated and noninsulin-treated patients. All of the differences in the current report are related to the secondary end points, even though the authors of the original DIGAMI 2 study did not report any differences in these secondary end points.
Why does this approach raise such a serious concern? When a trial demonstrates overall negative results, as with DIGAMI 2, looking for differences among the variables is potentially treacherous and heavily dependent on asking a sufficient number of queries and conducting adequate “data mining.” Second, when a substantial number of patients in the clinical trial are effectively excluded from the study, especially if they were at high risk for an adverse outcome (in the current study, subjects already deceased), analyses in the remaining subjects is substantially compromised. Finally, statistical adjustment for multiple variables is markedly limited with a small number of events.
There are some interesting and instructional reports on the results of analyses using variables to predict outcomes that were unlikely related to the outcome(s) under evaluation. One of these came from the ISIS-2 (International Study of Infarct Survival 2) Collaborative Group.5,6 In their review of intravenous streptokinase and aspirin studies, the investigators performed several subgroup analyses, including one that examined the subjects’ astrological signs. They found that patients born under Gemini or Libra had a slight adverse effect on mortality with aspirin treatment compared with those born under other Zodiac signs in whom “a strikingly beneficial effect” was observed. This example very succinctly highlights the risk of performing too many analyses. In their paper, the ISIS-2 investigators state: “It is, of course, clear that the best estimate of the real size of the treatment effect in each astrological subgroup is given not by the results in that subgroup alone but by the overall results in all subgroups combined.” In a “negative” trial like DIGAMI 2, sufficient numbers of analyses are likely to result in some intervention (in this case, insulin) that may simply show “adverse” effects as a play of chance. Similar points about multiple nonprespecified analyses have been noted by Austin and colleagues, who reported that Ontarians born under Leo had a higher probability of gastrointestinal hemorrhage and those born under Sagittarius had a higher probability of humerus fractures.7 The authors noted: “Our analyses illustrate how the testing of multiple, nonprespecified hypotheses increases the likelihood of detecting implausible associations. Our findings have important implications for the analysis and interpretation of clinical studies.” These examples illustrate the importance of caution when conducting post hoc analyses.
Concerns about DIGAMI 2
A brief examination of DIGAMI 1 and DIGAMI 2 is needed to put the post hoc analysis of DIGAMI 2 in context.2,8-10 Several reports from DIGAMI 1 found a reduction in mortality in patients receiving insulin after an MI.8-10 Whether this reduction was related to insulin use or to effects on glycemic control could not be deduced; thus, DIGAMI 2 was designed, at least in part, to answer the question of whether acute or long-term insulin therapy (and measures of glycemic control) would reduce mortality after an MI. The study used 3 treatment strategies: (1) acute insulin-glucose infusion followed by insulin-based long-term glucose control; (2) insulin-glucose infusion followed by standard glucose control; and (3) routine metabolic management according to local practice. Power calculations determined that 3000 patients (1250 in the first 2 groups and 700 in the third group) were necessary to answer the research question; however, less than half (n = 1253) were actually randomized when the study was closed by the steering committee because of “slow patient recruitment.” A trial underpowered to show efficacy of its prespecified primary (mortality) and secondary (clinical events) outcomes because of a failure to achieve recruitment targets is probably remiss in reporting an adverse outcome in post hoc analyses.
Hyperglycemia and MI: Further considerations
In 2006, Pittas and colleagues conducted a careful review of in-patient insulin use and the effects on outcomes in several patient populations.4 The authors found a trend toward possible benefits of insulin after MI (relative risk, 0.89; 95% CI, 0.76-1.03). In a recently published scientific review by Deedwania and colleagues for the American Heart Association, the authors note that although hyperglycemia is clearly associated with adverse outcomes after an MI, the data supporting glucose-lowering interventions are still not robust (level C evidence).3 They also acknowledge that insulin is the quickest way to achieve glycemic control in these high-risk patients. In light of the paucity of data in these high-risk patients, the authors state “there is an urgent need for definitive large randomized trials to determine whether treatment strategies aimed at glucose control will improve patient outcomes and to define specific glucose treatment targets.” Although Mellbin and colleagues also suggest the need for future studies, investigators who read their report may be dissuaded from considering insulin interventions in such studies. This possibility is disconcerting because it could reduce the likelihood of some carefully designed studies to answer important questions regarding the relationships of glucose reduction and insulin use in post-MI patients.
Mellbin and colleagues’ post hoc analysis of DIGAMI 2 found that insulin therapy is potentially hazardous in patients who have had an MI, but this conclusion was based on a seriously flawed analysis. Future clinical trials of glycemic control in patients with acute coronary syndromes should not exclude insulin as a possible intervention based on this study. Well-designed and adequately powered studies are needed to determine the effect and role of insulin in acute coronary syndrome patients.