Accounting for the mortality benefit of drug-eluting stents in percutaneous coronary intervention: a comparison of methods in a retrospective cohort study

Background Drug-eluting stents (DES) reduce rates of restenosis compared with bare metal stents (BMS). A number of observational studies have also found lower rates of mortality and non-fatal myocardial infarction with DES compared with BMS, findings not observed in randomized clinical trials. In order to explore reasons for this discrepancy, we compared outcomes after percutaneous coronary intervention (PCI) with DES or BMS by multiple statistical methods. Methods We compared short-term rates of all-cause mortality and myocardial infarction for patients undergoing PCI with DES or BMS using propensity-score adjustment, propensity-score matching, and a stent-era comparison in a large, integrated health system between 1998 and 2007. For the propensity-score adjustment and stent era comparisons, we used multivariable logistic regression to assess the association of stent type with outcomes. We used McNemar's Chi-square test to compare outcomes for propensity-score matching. Results Between 1998 and 2007, 35,438 PCIs with stenting were performed among health plan members (53.9% DES and 46.1% BMS). After propensity-score adjustment, DES was associated with significantly lower rates of death at 30 days (OR 0.49, 95% CI 0.39 - 0.63, P < 0.001) and one year (OR 0.58, 95% CI 0.49 - 0.68, P < 0.001), and a lower rate of myocardial infarction at one year (OR 0.72, 95% CI 0.59 - 0.87, P < 0.001). Thirty day and one year mortality were also lower with DES after propensity-score matching. However, a stent era comparison, which eliminates potential confounding by indication, showed no difference in death or myocardial infarction for DES and BMS, similar to results from randomized trials. Conclusions Although propensity-score methods suggested a mortality benefit with DES, consistent with prior observational studies, a stent era comparison failed to support this conclusion. Unobserved factors influencing stent selection in observational studies likely account for the observed mortality benefit of DES not seen in randomized clinical trials.


Background
The comparison of alternative treatments has long been a primary mission of both randomized trials and observational studies. With the commitment of $1.1 billion in support of comparative effectiveness research in the American Recovery and Reinvestment Act of 2009, the number of studies comparing different drugs, devices, techniques and systems will undoubtedly increase dramatically [1]. While randomized clinical trials are likely to remain the gold standard for comparing alternative treatments, observational studies should continue to have significant, if not leading, roles in comparative effectiveness research moving forward, particularly in light of recommendations to prioritize assessments of community-based interventions within populations traditionally underrepresented in clinical trials [2].
However, observational studies are subject to a number of limitations, foremost among them the potential for unmeasured variables that confound results. While a number of methods directed at assessing causal effects and eliminating confounding have been developed, few clinical studies describe reasons for the specific choice of method used, and fewer present multiple methods to help corroborate findings [3,4].

Methods
We conducted a retrospective dynamic cohort study within Kaiser Permanente of Northern California (KPNC), a large integrated healthcare delivery system caring for > 3.2 million individuals that are broadly representative of the local surrounding and statewide population [28]. All health plan members aged 30 years and older between January 1998 and the end of December 2007 were considered eligible. The study was reviewed by the institutional review board of the Kaiser Division of Research, and requirement for informed consent was waived due to the nature of the study. From this cohort, we identified all PCI procedures using either DES or BMS based on relevant International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) and Current Procedural Terminology (CPT) codes that occurred within health plan and non-health plan hospitals [29]. Consecutive procedures occurring within 7 days of one another were considered part of the same clinical episode. Procedures in which both DES and BMS were used were excluded from the analysis.

Patient characteristics and coexisting illnesses
Patient demographic information was obtained from health plan electronic databases. Relevant ICD-9 or CPT codes found in hospital discharge databases during the eight years before the procedure date were used to identify prior cardiovascular disease including prior myocardial infarction, prior angina, prior PCI or coronary artery bypass graft (CABG) surgery, history of ischemic stroke and peripheral arterial disease. Prior chronic heart failure was determined based on a validated algorithm using information from hospitalization, outpatient and emergency department diagnostic codes [30].
Relevant data sources were searched for cardiovascular risk factors. Diabetes mellitus was identified from a validated longitudinal Diabetes Registry relying on inpatient and outpatient diabetes diagnoses, receipt of anti-diabetic therapies, and abnormal glycosylated hemoglobin or blood glucose levels [31]. Hypertension was based on serial ambulatory diagnoses or the combination of diagnoses and receipt of anti-hypertensive medications. Chronic lung disease, systemic malignancy, and history of gastrointestinal bleeding were determined based on validated methods [29][30][31][32][33][34][35]. The presence and severity of chronic kidney disease was ascertained using the abbreviated four-variable Modification of Diet in Renal Disease (MDRD) equation for estimated glomerular filtration rate based on most recent outpatient determination of serum creatinine within the previous 24 months before the year examined [33].

Medication usage
Prior observational studies comparing outcomes for DES and BMS have not included the chronic outpatient use of cardioprotective medications, an important potential confounder [6,19,36]. Outpatient medication use within 30 days before PCI was ascertained from health plan pharmacy records for therapies known to lower cardiovascular risk, including beta-blockers [37], angiotensinconverting enzyme (ACE) inhibitors or angiotensin receptor blockers (ARBs), [38] statins [39], and thienopyridines [40]. Outpatient warfarin use was also assessed. More than 90% of patients had a drug benefit that provided strong financial incentives to obtain medications from health plan pharmacies.

Primary outcomes
Death from any cause occurring within thirty days and one year after the procedure was identified from health plan administrative databases, proxy information, Social Security Administration vital status files, and California state death certificate information [41,42]. Subsequent hospitalized myocardial infarction based on ICD-9-CM codes (primary discharge diagnosis coded as 410.x1) within one year was also identified.

Statistical analysis
Three separate methods were used to compare outcomes after DES and BMS: 1) statistical adjustment based on propensity score decile, 2) propensity-score matching, and 3) a stent era comparison, in which outcomes before and after the introduction of DES in 2003 were compared.

Method 1: propensity-score adjustment
Because patients were not eligible to receive DES until after April of 2003, only PCI procedures after this date were included in the propensity score methods. A nonparsimonious logistic regression model was used to generate propensity scores for the likelihood of receiving DES based on 23 demographic and clinical variables. Propensity score deciles were created, and logistic regression was performed modeling the association of DES with outcomes, adjusted for propensity score decile (categorical variable) as well as the complete set of patient covariates. To accommodate data from individuals who may have undergone multiple procedures during the study period, robust standard errors accounting for clustering by patient were used.

Method 2: propensity-score matching
In the second method, one-to-one nearest neighbor matching was performed to compare outcomes after DES and BMS in patients undergoing PCI after April 2003. Caliper width was set at 0.0001 using the STATA program psmatch2 (Leuven and Sianesi, 2003) [43]. Adequacy of the match was assessed by estimating the standardized differences between DES and BMS patients for all variables. Outcomes were compared between groups using the McNemar's Chi-square test for matched comparisons.

Method 3: stent era comparison
Because DES was not available before 2003 and from 2004 onward DES was used in the vast majority of cases, a comparison of outcomes of PCI prior to and after DES introduction serves as a useful surrogate for assessing the impact of DES (Figure 1) [4,36,44]. Procedures between January 1998 and March 2003 were assigned to the 'BMS era', while those between April 2003 and December 2007 were assigned to the 'DES era'. This variable would be expected to be strongly associated with treatment assignment, and less prone to unmeasured confounding by stent indication. Logistic regression was then performed to determine the association of stent era with outcomes, adjusted for all assessed covariates.
Analyses were performed using STATA (Stata Corp. 2008. Stata Statistical Software: Release 10, College Station, TX: StataCorp LP). A two-sided P value of < 0.05 was considered significant for all analyses. these, 19,103 (53.9%) utilized DES and 16,335 (46.1%) utilized BMS. Patients receiving DES were significantly different than those receiving BMS in nearly all baseline characteristics, including having significantly higher rates of hypertensions, diabetes, and congestive heart failure (Table 1). DES patients also had lower rates of presentation with ST-elevation myocardial infarction within 30 days prior to PCI, and were more likely to be receiving outpatient cardioprotective medications including statins, beta-blockers and thienopyridines. Crude-mortality was significantly higher at thirty days (2.2% versus 1.6%, P < .001) and one year (5.1% versus 4.6%, P = 0.04) after PCI with BMS compared to DES. The rate of myocardial infarction at one year was 4.7% after BMS and 4.3% after DES (P = 0.06).

Discussion
We have shown that after the introduction of drug-eluting stents, bare metal stents were reserved for patients with significant comorbidities and high mortality rates. Commonly used propensity-score methods continued to suggest a mortality benefit to DES, while a stent era comparison showed no significant differences in all outcomes for DES and BMS, similar to results from randomized clinical trials.   A number of prior observational studies comparing DES to BMS have suggested reductions in mortality and myocardial infarction associated with DES, the majority of which have utilized propensity-score methods to mitigate potential confounding [5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20]. However, data from randomized trials have not corroborated these findings [21][22][23][24][25][26][27]. In a meta-analysis of 22 randomized trials and 34 observational studies, Kirtane et al found no differences in mortality or myocardial infarction in randomized trials, but consistent reductions for both outcomes in observational studies, and attributed these differences, at least in part, to residual confounding by unmeasured differences between patients receiving BMS and DES [23]. Our results are consistent with this conclusion. In our cohort, patients who received BMS after April 2003 had significantly greater comorbidities and unadjusted death and myocardial infarction rates compared to patients who received BMS before this date, suggesting that physicians selectively reserved BMS for a sicker population of patients after DES was introduced. While propensity-score adjustment and matching both attempt to account for such differences, they only account for variables that are assessed. In the presence of such a high degree of patient selection, any residual confounding in this case is likely to lead to the appearance of improved outcomes with DES, as was seen in this study. While propensity scores have been put forward as a method which overcomes some of the limitations of traditional logistic regression, they do not address a primary threat to the validity of such studies -unmeasured confounding.
A comparison of the BMS era (100% BMS) to the DES era (88.3% DES) as a surrogate for a BMS-DES comparison eliminates the unwanted influence of patient selection [45]. Consistent with this, the stent era comparison showed no differences in mortality or myocardial infarction between BMS and DES, most closely approximating results seen in randomized trials. However, because such an analysis compares patients undergoing PCI at two different times, important secular trends may have influenced the results. For example, comparisons of stent eras in the Medicare population and in the New York PCI Registry both suggested lower rates of myocardial infarction in the DES era compared to BMS era [36,44]. However, neither study adjusted for differences in the use of outpatient medications such as statins and beta-blockers, which have increasingly been used in patients with coronary disease over time [46]. In this study, we found significantly higher rates of use of beta blockers, statins, angiotensin-converting enzyme inhibitors, and thienopyridines in the DES era. After accounting for these and other secular trends, no differences in myocardial infarction were observed between DES and BMS patients.
A number of circumstances specific to stent use allowed for the critical examination of these analytical methods. First, the rapid uptake of DES over BMS created a natural experiment allowing use of a stent era comparison. Second, because a large number of patients from randomized clinical trials comparing DES with BMS have been performed, these studies help provide an estimate of the 'truth.' However, in other examples in which the questions of interest may be less well studied, there could be much greater difficulty interpreting disparate results.

Conclusions
In summary, we have compared outcomes after PCI with either DES or BMS, using three commonly used methods of adjustment in observational studies. Large baseline differences in BMS and DES groups suggested strong treatment selection, which were incompletely adjusted for by propensity-score methods. In the presence of such treatment selection, alternative methods of analysis of observational data should be considered.