The FDA’s Guidance for Industry is entitled “Diabetes Mellitus – Evaluating Cardiovascular Risk in New Antidiabetic Therapies to Treat Type 2 Diabetes” (FDA 2008). Prior to submission of a New Drug Application (NDA) or Biologics License Application (BLA), sponsors are required to compare the incidence of centrally adjudicated MACE composite endpoint outcomes (recall discussions in Sect. 6.​6) occurring in the drug treatment arm with those occurring in the control treatment arm to show that the upper bound of the two-sided 95 % confidence interval for the risk ratio point estimate is less than the 1.8 threshold. This requirement translates to the prospective exclusion of excess cardiovascular risk of 80 % or greater.

This is the first step in what is usually a two-stage process. The requirement can be satisfied by performing a meta-analysis of adjudicated cardiovascular events using participant level data from the Phase II and Phase III trials conducted in the drug’s clinical development program or, if the data from the studies included in the meta-analysis will not meet this requirement, by conducting a large cardiovascular safety outcome trial that, alone or when added to the other trials, would discharge the 1.8 threshold.

If the upper bound of the two-sided 95 % confidence interval for the risk ratio point estimate is between 1.3 and 1.8, and the overall benefit–risk analysis supports approval, a postmarketing trial is required to fulfill the second requirement, i.e., definitive demonstration that the upper bound of the two-sided 95 % confidence interval for the risk ratio point estimate is then less than the 1.3 threshold. This requirement translates to the prospective exclusion of excess cardiovascular risk of 30 % or greater. The much larger sample size facilitates greater precision, which translates to a tighter (narrower) confidence interval being placed around the relative risk point estimate, and a greater likelihood, therefore, that the upper bound will be less than the 1.3 threshold. If the upper bound of the two-sided 95 % confidence limit for the risk ratio point estimate from the analysis of preapproval studies is less than 1.3, and the overall benefit–risk analysis supports approval, a postmarketing cardiovascular outcome trial may not be necessary: however, this scenario is likely to occur infrequently.

Although the confidence interval upper limit values are emphasized, the magnitude of the risk ratio point estimate itself will be considered. A detailed statistical analysis plan addressing proposed analytical strategies for the adjudicated cardiovascular events of interest must be discussed with regulators early in clinical development. If a meta-analysis is planned, it should include all placebo-controlled, add-on, and active-controlled trials performed in the drug’s development program.

The EMA’s guideline is entitled “Guideline on Clinical Investigation of Medicinal Products in the Treatment or Prevention of Diabetes Mellitus” (EMA 2012). It noted that either of the following should be submitted at the time of the marketing authorization application: an integrated safety analysis focused on adjudicated MACE composite endpoints or results from a long-term, controlled cardiovascular outcome study with at least 18–24 months of follow-up. The guideline did not define any specific thresholds that must be excluded. While emphasis was placed on MACE, MACE-plus can be used as the primary endpoint in certain circumstances. The acceptability of the data will be based on their overall quality and also on the relative risk point estimate and confidence interval obtained for the calculation of cardiovascular risk. Indications of an increased risk in certain adverse events or an unacceptable lack of precision (a wider than desirable confidence interval) may trigger a request for an additional long-term trial to exclude an unacceptable increase in cardiovascular risk.

Given that the EMA guideline does not include specific thresholds of regulatory interest (i.e., does not provide values analogous to the 1.8 and 1.3 thresholds presented in the FDA guidance), discussions in the rest of this chapter focus on satisfying the explicit FDA thresholds. Nonetheless, many aspects of the discussions are also pertinent to satisfying the EMA guideline’s requirements. The clinical implications of the FDA guidance will be discussed first, followed by commentary on statistical considerations. Discussions in this chapter are necessarily a little more complex from a statistical perspective than in other chapters, but we have endeavored to keep them as digestible as possible (recall foundational discussions in Chap. 6).

A validated approach to discharge the 1.8 and 1.3 thresholds has been to conduct a meta-analysis of MACE or MACE-plus outcomes accrued during Phase II and Phase III trials to discharge the 1.8 threshold and to conduct a cardiovascular safety outcome trial to discharge the 1.3 threshold. The outcome trial might begin during Phase III, after submission of a marketing approval request, or post-approval. If the trial is initiated during Phase III, interim data from the study may or may not be included in the meta-analysis to discharge the 1.8 pre-submission requirement.

Saxagliptin, a dipeptidyl peptidase-4 (DPP-4) inhibitor, was approved by the FDA in 2009, shortly after the FDA’s 2008 guidance was issued. Centralized adjudication of cardiovascular events had not been a requirement during the drug’s development period. Instead, to assess the cardiovascular safety of the drug, meta-analyses using reported cardiovascular Medical Dictionary for Regulatory Activities (MedDRA) event terms were conducted. Post-approval, the SAVOR-TIMI 53 trial (Scirica et al. 2014) was conducted to demonstrate that the upper bound of the two-sided 95 % confidence interval placed around the hazard ratio was less than 1.3.

A total of 16,492 participants with type 2 diabetes or at risk for cardiovascular events were randomized to saxagliptin or placebo (on a background of standard of care for diabetes and cardiovascular risk factors) and followed for a median of 2.1 years. The primary endpoint was occurrence of MACE outcomes. A primary endpoint event occurred in 613 participants in the saxagliptin group and in 609 participants in the placebo group. The statistical analysis plan prespecified that a test for noninferiority would be conducted first, followed by a test for superiority. Saxagliptin did not increase or decrease the rate of MACE outcomes (hazard ratio = 1.00, 95 % CI = 0.89–1.12: p < 0.001 for noninferiority, p = 0.99 for superiority).

Dapagliflozin, a sodium glucose co-transporter 2 (SGLT-2) inhibitor, was approved by the FDA in 2014. The sponsor planned two sequential meta-analyses to discharge the 1.8 threshold (Sponsor’s Background Document, Dapagliflozin). The first meta-analysis was to occur after a prespecified number of studies completed, and, if the 1.8 threshold was not discharged, a second meta-analysis was to be conducted after another group of studies concluded. Given the plan for sequential testing, alpha spending modifications were specified to preserve the type 1 error rate. The first meta-analysis was performed on 78 MACE-plus outcomes from 6,228 participants in 14 trials. The hazard ratio for dapagliflozin vs. comparator (placebo and active comparators combined) was 0.67 (95 % CI: 0.42, 1.08). Thus, an unacceptable increase in cardiovascular risk in participants with type 2 diabetes was ruled out, and the second meta-analysis was not necessary.

This approach provided two opportunities to discharge the 1.8 threshold, with the possibility of an earlier submission if the first meta-analysis did so successfully. The DECLARE-TIMI 58 trial was a superiority trial designed to test the hypothesis that dapagliflozin reduces the incidence of MACE events compared with placebo in individuals with type 2 diabetes at high risk for cardiovascular events. The study was also designed to definitively exclude an unacceptable cardiovascular risk from dapagliflozin in these individuals, i.e., to demonstrate a post-approval upper bound of the two-sided 95 % confidence interval placed around the hazard ratio point estimate of less than 1.3 to further support the cardiovascular safety of dapagliflozin.

Canagliflozin, a SGLT-2 inhibitor, was approved by the FDA in 2013. The strategy employed to discharge the 1.8 safety margin was to perform a meta-analysis using MACE-plus events from the Phase II and Phase III trials and interim data from the Canagliflozin Cardiovascular Assessment Study (CANVAS) (CANVAS: Canagliflozin Cardiovascular Assessment Study 2015). CANVAS was designed to demonstrate that treatment with canagliflozin would reduce cardiovascular risk (based on MACE events) compared with placebo in participants with type 2 diabetes with, or at high risk for, cardiovascular events. The trial was designed to enroll two sequential cohorts, with a decision to recruit further participants dependent upon a protocol-specified interim analysis of results from the initial cohort (Cohort A). An interim analysis by an independent data monitoring committee was planned to be conducted 4 years after trial initiation to assess study feasibility of achieving the primary hypothesis of cardiovascular benefit and to recommend that study recruitment be reopened if the interim results were positive. The data monitoring committee operated under a prespecified interim monitoring program whereby they could recommend early termination for safety (ruling out a 30 % or greater excess cardiovascular risk) or for futility. If enrollment was not reopened, participants in Cohort A would continue to be followed for long-term safety.

The plan was to perform a meta-analysis to discharge the 1.8 threshold when 201 MACE-plus outcomes had accumulated in the development program. Assuming an annualized event rate of 2.5 % in the CANVAS trial, approximately 160 MACE-plus outcomes were expected within 2 years of activating Cohort A, which would provide more than 90 % power to discharge the 1.8 threshold. These data would be included in a submission dossier prepared by a group of researchers who were independent of the team that would continue to manage the trial. Cohort A would continue to generate additional events that could be combined with events from Cohort B to discharge the 1.3 threshold if Cohort B were activated. Data from both cohorts would be combined and the trial would continue until a maximum of 1,600 MACE outcomes had accumulated, which would provide 90 % power to detect a 15 % cardiovascular risk reduction. Given that the maximum number of MACE outcomes from Cohort A and the maximum number of outcomes from combining Cohort A and Cohort B were both prespecified, and no results from either cohort would be publicly available during the course of the entire CANVAS trial, this study design was considered statistically valid and free of bias.

A group sequential approach was planned to discharge the 1.3 threshold. The significance levels for the multiple analyses were based on the Lan–DeMets spending function with an O’Brien–Fleming boundary. The first analysis to discharge it would be conducted at the time when the meta-analysis to discharge the 1.8 threshold was performed. The next planned interim meta-analysis would be conducted when approximately 500 MACE-plus events had accrued, and a final interim analysis was planned to occur after approximately 700 events had accumulated, if the 1.3 threshold had not yet been discharged. In actuality, Cohort A enrolled 4,411 participants within 15 months, and a decision was taken not to activate Cohort B.

When 201 MACE-plus events had accrued from 9,632 participants enrolled in nine trials, the meta-analysis showed that canagliflozin did not unacceptably increase MACE-plus events based on a hazard ratio point estimate of 0.91 (95 % CI: 0.68–1.22) (FDA Medical review document). However, evaluation of the individual components of the MACE-plus composite endpoint revealed that the hazard ratio point estimate for nonfatal stroke was 1.46 (95 % CI: 0.83, 2.58). Also, the CANVAS trial contributed the majority of events in the meta-analysis (nearly 80 %) due to its size and higher cardiovascular risk population. Therefore, data from CANVAS were analyzed separately from the other studies. For CANVAS, the hazard ratio point estimate was 1.0 (upper bound of the two-sided 95 % CI = 1.39) compared with 0.65 (upper bound of the two-sided 95 % CI = 1.21) in the other 8 trials. Given the relatively low number of cardiovascular events, it is not surprising that the observed hazard ratio point estimates for each analysis lay above and below 1.0 with fairly wide confidence intervals.

A single cardiovascular safety outcome trial initiated during Phase III could be used to discharge both the 1.8 and 1.3 thresholds. The trial would be designed to demonstrate a post-approval upper bound of the two-sided 95 % confidence interval placed around the hazard ratio point estimate of less than 1.3 to provide evidence of the drug’s cardiovascular safety. The primary intent would be to show that the new therapy does not increase the risk of MACE or MACE-plus events. The trial could be designed as a noninferiority trial, a superiority trial, or both using a sequential testing methodology. Interim data (MACE or MACE-plus outcomes) could be used to discharge the 1.8 threshold, and these data could be included in the submission dossier. Alternatively, interim data from the cardiovascular safety outcome trial could be combined with MACE or MACE-plus events collected in Phase II and Phase III trials in a meta-analysis to discharge the 1.3 threshold.

Alogliptin, a DPP-4 inhibitor, was approved by the FDA in 2013. For this program, a single cardiovascular safety outcome trial, Examination of Cardiovascular Outcomes with Alogliptin (EXAMINE), was used to discharge both the 1.8 and 1.3 thresholds. EXAMINE was a noninferiority trial with a prespecified noninferiority margin of 1.3, using a MACE primary endpoint (White et al. 2013). A total of 5,380 participants with type 2 diabetes and having had an acute myocardial infarction or unstable angina requiring hospitalization were randomized to either alogliptin or placebo in addition to existing antidiabetic and cardiovascular therapies and followed for 40 months. Four interim analyses of the MACE endpoint were prospectively planned using an O’Brien–Fleming alpha spending function. If, at any of these analyses, the 1.8 threshold was not discharged, the trial would be stopped for futility. If at a given analysis the 1.8 threshold was discharged, the trial would continue and an interim analysis would be performed after 550 and 650 events accrued to discharge the 1.3 threshold. If noninferiority was declared and the conditional power for superiority (with 650 events) was less than 20 % at the 550-event interim analysis, the study would be terminated.

After 83 MACE events had accrued, the first interim analysis was performed and the upper bound of the confidence interval placed around the hazard ratio point estimate was 1.51. An independent statistician performed this analysis and communicated the results to the independent data monitoring committee. Having therefore discharged the 1.8 threshold, the sponsor was able to include these interim results in the NDA. To protect trial integrity and statistical validity, the researchers involved with this analysis were not involved in subsequent trial conduct, data reviews, and trial analyses and did not communicate with those still involved in the study.

In keeping with the statistical plan, the next interim analysis of MACE events was performed after 550 events had occurred, resulting in a hazard ratio point estimate of 0.96 and an associated confidence interval upper bound of 1.17, demonstrating that alogliptin was noninferior but not superior to placebo. Since the conditional power to show superiority with 650 events was less than 20 %, the trial was terminated. Prior to data lock, an additional 71 participants had a primary endpoint event. The final analysis of MACE yielded a hazard ratio of 0.96 and an associated confidence interval upper bound of 1.16 (p < 0.001 for noninferiority; p = 0.32 for superiority), confirming that alogliptin did not unacceptably increase cardiovascular risk.

If a meta-analysis of MACE or MACE-plus events from Phase II and Phase III trials is conducted, all placebo-controlled, add-on, and active-controlled trials should be included, as previously noted. However, these trials will likely vary in size, duration, participant characteristics including baseline cardiovascular risk, and comparator therapies. These differences may lead to heterogeneity, i.e., dissimilarities in results that may or may not be due to chance. Heterogeneity has been observed in some of the cardiovascular meta-analyses submitted to the FDA (Sahlroot 2012). Utilization of similar study designs, eligibility criteria, and a uniform process to collect and adjudicate cardiovascular events may mitigate some of this risk. The quality of the meta-analysis will depend on the quality and comparability of the component trials and the methodological rigor employed in conducting the meta-analysis.

If study participants are at low to moderate cardiovascular risk, it is possible that a small number of cardiovascular events may be reported in a given trial. In this case, the precision of the hazard ratio point estimate for the new drug vs. the comparator may be low, as indicated by large confidence intervals. If the true hazard ratio is close to 1.0, it can be expected that point estimates greater and smaller than 1.0 will be observed among the trials. Therefore, it is recommended that all of the hazard ratios and their associated confidence intervals for each study included in the meta-analysis be displayed in a forest plot, accompanied by an interaction test for a common hazard ratio.

Whether or not to include trials with no cardiovascular events in a meta-analysis has been a highly debated topic. If a trial had a sufficiently long period of follow-up and no cardiovascular events were reported, these data would appear to support the hypothesis of noninferiority; however, no formal statistical method exists for estimating the cardiovascular risk from such data, and they are typically excluded from the analysis. Tian and colleagues (2009) discussed a method of obtaining an exact confidence interval for the difference in event rates at a fixed time point that permits combining data from trials that have zero events with data from other trials. This method could be utilized as a supportive sensitivity analysis.

Prior to issuance of the FDA guidance, the cardiovascular safety of many antidiabetic therapies had not been established and, for some classes of drugs, concern still exists that they may increase cardiovascular risk (Horsdal et al. 2011; Schramm et al. 2011; Phung et al. 2013). Hence, comparisons of cardiovascular event rates with the new drug vs. placebo and active control groups combined are challenging to interpret. Therefore, meta-analyses of the investigational drug vs. placebo only and then vs. active comparators only should be considered and may be requested by some regulatory authorities.

Adaptive methodologies, such as sample size or outcome re-estimations and early stopping decision rules, could be specified in the statistical analysis plans for cardiovascular safety outcome trials, potentially to reduce study duration, increase the chances of success, and facilitate earlier submissions. Many different types of adaptive designs are in use. The most common adaptation is to increase the sample size if the rate at which outcomes are accruing is slow. This modification raises no statistical issues if done in a blinded manner.

More recently, there has been interest in adaptive designs where the sample size and number of events are increased based on an unblinded look at interim analysis results. For example, a trial might be initially sized based on the number of MACE outcomes needed to demonstrate noninferiority; however, if superiority appears likely based on the conditional power available at the time of an interim analysis, the number of needed MACE outcomes and the sample size could be increased. Such adaptive designs require special statistical analyses to protect the alpha level and special provisions to minimize operational bias (FDA 2010). This approach reduces the risks to participants (and to the sponsor) associated with initiating a large superiority trial up-front, when there is limited information about the compound.

Group sequential designs permit interim monitoring of accumulating data with the possibility of stopping early for efficacy, safety concerns, or futility. Cardiovascular safety outcome trials typically require a substantial follow-up time to obtain the required number of cardiovascular events and are therefore attractive candidates for group sequential monitoring. The number of planned interim analyses and the cumulative number of cardiovascular events at which each interim analysis is performed in a group sequential design study should be specified in the study’s statistical analysis plan to promote trial integrity and ensure adequate documentation of testing procedures. At each interim analysis, a determination is made as to whether or not there is sufficient evidence to stop the trial and demonstrate noninferiority by performing an hypothesis test, or equivalently by computing a confidence interval that is adjusted for multiple looks at (analyses of) the accumulating data.

The first interim analysis should occur only after a reasonable number of events have occurred, and there has been adequate participant exposure to the drug to be able to draw a meaningful clinical conclusion. In practice, it may not be feasible to adhere strictly to the planned spacing and number of interim analyses. For example, the actual number of events at an interim analysis after the data are cleaned and the database locked may differ from that specified in the statistical analysis plan. For this reason, an alpha spending function (Lan and DeMets 1983) must be specified that controls the overall type 1 error rate while allowing flexibility with respect to the number and timing of interim analyses. Because of this flexibility, most group sequential designs utilize spending functions from families such as the O’Brien–Fleming family (O’Brien and Fleming 1979) and the Pocock family (Pocock 1977) to elicit stopping boundaries at interim looks, rather than specifying the stopping boundaries directly.

Spending functions differ in how aggressively they spend alpha relative to the timing of each interim analysis. The selection of a spending function is not prescriptive, but the criteria should include the ability to have a convincing point estimate and sufficient information to address secondary hypotheses and other safety considerations adequately if the study is terminated early to declare success. The Lan and DeMets (1983) O’Brien–Fleming-type spending function, which is often preferred, meets these criteria because very little alpha is spent in early looks. This spending function produces group sequential boundaries that are very similar, but not identical, to the O’Brien–Fleming boundaries. If the trial is stopped early, the evidence is compelling due to their higher hurdle. In addition, the penalty for the interim analysis is minor, resulting in a sample size that is approximately equivalent to that employed in a fixed study design of the same power.