You are here

Article Text

Article menu
Original research
Combined risk modelling approach to identify the optimal carotid revascularisation approach
  1. James Francis Burke1,
  2. Lewis B Morgenstern1,
  3. Nicholas H Osborne2,
  4. Rodney A Hayward3
  1. 1 Neurology, University of Michigan Michigan Medicine, Ann Arbor, Michigan, USA
  2. 2 Vascular Surgery, Univerity of Michigan Medicine, Ann Arbor, Michigan, USA
  3. 3 Internal Medicine, University of Michigan Michigan Medicine, Ann Arbor, Michigan, USA
  1. Correspondence to Dr James Francis Burke; jamesbur{at}med.umich.edu

Abstract

Background Carotid endarterectomy (CEA) results in fewer perioperative strokes, but more myocardial infarctions (MI) than carotid artery stenting (CAS). We explored a combined modelling approach that stratifies patients by baseline stroke and MI.

Methods Baseline registry-based risk models for perioperative stroke and MI were identified via literature search. We then selected treatment risk models in the Carotid Revascularisation Stenting versus Endarterectomy (CREST) trial by serially adding covariates (baseline risk, treatment (CEA vs CAS), treatment-risk interaction and age-treatment interaction terms). Treatment risk models were externally validated using data from the Society for Vascular Surgery (SVS) Vascular Quality Initiative (VQI) CEA and carotid stenting registries and treatment models were recalibrated to the SVS-VQI population. Predicted net benefit was estimated by summing the predicted stroke and MI risk differences with CEA versus CAS.

Results Perioperative treatment models had moderate predictiveness (c-statistic 0.69 for stroke and 0.68 for MI) and reasonable calibration across the risk spectrum for both stroke and MI within CREST. On external validation in SVS-VQI, predictiveness was substantially reduced (c-statistic 0.61 for stroke and 0.54 for MI) and models substantially overpredicted risk.

Most patients (86.7%) were predicted to have net benefit from CEA in CREST (97.0% of symptomatic patients vs 75% of asymptomatic patients).

Discussion A combined modelling approach that separates risk elements has potential to inform optimal treatment. However, our current approach is not ready for clinical application. These data support guidelines that suggest that CEA should be the preferred revascularisation modality in most patients with symptomatic carotid stenosis.

  • technique
  • stroke

Data availability statement

Data are available in a public, open access repository. Data may be obtained from a third party and are not publicly available. CREST data are available from NINDS. SQS-VQI data are available via application to SVS-VQI.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/svn-2020-000558

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Two primary revascularisation strategies exist for patients with carotid artery stenosis—carotid endarterectomy (CEA) and carotid artery stenting (CAS). Most patients with CAS could be treated with either procedure and, thus, numerous trials have compared these techniques, generally slightly favouring CEA.1 The largest single trial, which also used the most rigorous proceduralist credentialing strategy, was The Carotid Revascularisation Endarterectomy versus Stenting Trial (CREST).2 3 It found that CAS was non-inferior to CEA on a composite endpoint including stroke and periprocedural myocardial infarction (MI) and that younger patients may benefit more from CAS than CEA. Reflecting this evidence, guidelines have generally suggested that while there is more evidence for CEA than CAS, that both procedures can be considered as treatment options for individual patients.4

    Yet, the question of how to select the best approach for an individual patient is unclear. In part, based on the results of the Stenting and Angioplasty with Protection in Patients at High Risk for Endarterectom (SAPPHIRE) trial,5 CAS is generally recommended for high risk patients, although the precise definition of high risk is uncertain.4 Similarly, the subgroup effect in CREST towards greater benefit with CAS in younger patients could be used to inform individual decisions.2 In addition to these strategies, the observation that CAS is generally associated with a higher perioperative risk of stroke while CEA is generally associated with a higher risk of periprocedural MI1 suggests a possible approach to selecting the optimal treatment for individual patients. To the extent that risk for stroke and MI risk can be accurately predicted and these risks can be separated, treating patients at relatively high stroke risk with CEA and patients with relatively high MI risk with CAS may be a feasible strategy. This strategy is particularly promising because prior work suggests that the risk of stroke in patients with symptomatic carotid stenosis can be effectively stratified with straight-forward clinical models.6–9

    In this study, we describe the development and external validation of a combined modelling approach to predict the optimal revascularisation technique in individuals undergoing carotid revascularisation using data from the CREST trial. The goal of this modelling approach is to inform individual-level risk with CEA and CAS by accounting for all known factors that predict outcomes with either treatment.

    Methods

    We used a 3-step approach to develop models to estimate and compare individual-level untreated and treated risk of MI and stroke among patients who are candidates for CEA versus CAS (figure 1). First, we identified baseline risk models for perioperative stroke and MI based on clinical factors measured prior to undergoing a procedure. We focused on perioperative stroke risk because most strokes occurred in the perioperative period and the stroke risk after the perioperative period is similar in CEA and CAS in CREST. (online supplemental figure 1) Second, we developed treatment models to predict stroke/MI using individual level data from the CREST trial. Third, we externally validated these models in the Society for Vascular Surgery (SVS) Vascular Quality Initiative (VQI) CEA and carotid stenting registries.

    Supplemental material

    Figure 1
    Figure 1

    Schematic overview of methods. CAS, carotid artery stenting; CEA, carotid endarterectomy; CREST, Carotid Revascularisation Stenting versus Endarterectomy; MI, myocardial infarction; SVS, Society for Vascular Surgery; VQI, Vascular Quality Initiative.

    Datasets

    The CREST randomised 2502 patients with carotid stenosis (47% asymptomatic) to receive either CAS or CEA. The primary inclusion criterion was greater than 50% carotid stenosis by angiography or 70% stenosis on non-invasive imaging. A distinguishing feature of CREST, relative to other CEA versus CAS trials, was the requirement that both surgeons and interventionalists with must be qualified via both volume and outcome criteria prior to performing CEA or CAS in the trial. This feature may have contributed to the relatively low event rates in both arms compared with other CEA versus CAS trials.10 The primary composite outcome in CREST was a combination of periprocedural stroke, death or MI or ipsilateral stroke within the next 4 years. Fifty per cent of patients were allocated to each treatment arm. Mortality was the same in both arms (0.2%), while stroke was more common in CAS (4.1%) vs CEA (2.3%) and MI was more common in CEA (2.3%) vs CAS (1.1%).2

    The SVS VQI registries contain detailed clinical self-reported data on hundreds of thousands of patients undergoing vascular surgery in more than 500 centres in the USA and Canada. The registry records preprocedure baseline clinical status for the patient, details of the indication for the procedure, procedural details, complications and outcomes. For patients at participating institutions, all individual receiving a vascular procedure at the institution are included in the registries.11 12 Short-term mortality and MI measures have been shown to correlate well between SVS VQI and other registries,13 but less is known about the validity of stroke outcomes reported in the registries.

    Identifying baseline risk models for MI and stroke

    Numerous models have been developed to predict the risk of perioperative MI and stroke in patients undergoing CEA versus CAS. For perioperative MI, we used the Revised Cardiac Risk Index14 because the data necessary to stratify patients with this widely used model were readily available in CREST and prior work has found that this model has reasonable discrimination and calibration across a wide variety of patient populations.15 For perioperative stroke, no single model has been widely used and a number of risk models have been developed.16–20 Most of these models use a series of binary risk predictors (eg, baseline demographics, vascular risk factors, comorbidities, procedural circumstances) to stratify baseline risk,16–19 while one model also included preprocedure blood pressure.20 To select a model for application in this analysis, we mapped the variables included in these models to the CREST trial and SVS registries as best as possible, (necessitating that some risk factors were omitted) and measured their discrimination, accounting for omissions. Generally, discrimination was modest (c-statistics ranging from 0.60 to 0.64) and similar across models. However, modest discrimination has been shown to be adequate to reveal clinically important heterogeneity of treatment effect.21 Consequently, we selected the Goodney et al 19 model as our Primary Baseline Stroke Model as it was the model that was most completely mappable to the CREST and SVS VQI datasets (age >70, contralateral occlusion, heart failure, qualifying event Stroke/transient ischemic attack (TIA)). In a sensitivity analysis, we applied the Rothwell and Warlow20 model as a Sensitivity Baseline Stroke Model as it included baseline blood pressure (in addition to sex and whether the qualifying event was a stroke/TIA) as it was the model most qualitatively dissimilar from the other models. For both models, we determined their discrimination in CREST by calculating c-statistics and explored their clinical utility by determining how event rates varied for CEA and CAS across baseline risk quintiles.

    Developing models to predict MI, stroke with CEA/CAS treatment in CREST

    To inform the risks of treatment for individual patients, we built models incorporating the relevant baseline risk indices (eg, Primary Stroke Treatment Model incorporates the Primary Baseline Stroke Model) to separately predict the risk of stroke and MI in CREST. Initial treatment models predicted each outcome (MI, perioperative stroke/death) separately using logistic regression models that included the baseline risk index (linear predictor of risk using each baseline risk score), symptomatic status and treatment (CEA vs CAS) as independent variables. For all treatment models multi-collinearity of variables was assessed using variance inflation factors. For all baseline risk and treatment models alpha values of 0.05 were applied, calibration was assessed by plotting observed (proportion of population with a given outcome) versus expected (mean predicted outcome probability from a given model) across quintiles of baseline risk. Analyses were executed using Stata (V.15) (StataCorp: Release 15).

    To assess the accuracy of model specification, we explored whether treatment interaction terms should be retained in the final model. This was done by exploring alternate model specifications that included interaction terms with treatment status. Interactions were added to the model based on our judgement of their prior theoretical/empirical evidence and retained in the final model if the interaction term was statistically significant or if the model with the interaction reduced the model’s Akaike information criterion.22 Interactions were explored in the order: baseline risk, age (given the presence of an age-treatment interaction in CREST), symptomatic status, and then all individual baseline risk factors included in each individual risk index. Models are summarised in online supplemental table 1.

    Supplemental material

    To assess for overfitting, models were internally validated with bootstrapping and shrinkage coefficients were estimated for each model.23 After final internally validated treatment models were developed for both stroke and MI, overall model discrimination was measured using c-statistics. Model calibration for stroke, MI and total vascular events was assessed (predicted probability of both individual models) by plotting actual event risk versus estimated event risk across event quintiles.

    To inform the overall net benefits of CEA versus CAS, we first explored the correlation between risks estimated from individual models using Pearson correlation coefficients. We then used model-based estimates to predict the net number of events for each individual in CREST under the assumption they received CEA versus CAS in the overall trial and separately in the symptomatic and asymptomatic populations.

    External validation of treatment models

    To approximate the CREST population, all patients were included in SVS-VQI external validation except those that had: (1) prior neck radiation, (2) CEA or CAS as a reoperation on a previously treated carotid, (3) unstable angina or (4) a MI within the prior 6 months. For the final stroke and MI models, we externally validated the models in SVS VQI by measuring c-statistics for the linear predictors in the combined SVS VQI CEA and CAS registries and measured calibration by comparing observed to expected event rates. As summarised in results, the primary model overpredicted risk in SVS-VQI. To account for this, we estimated models that recalibrated relative to the SVS-VQI population. Specifically, we estimated a model with a recalibrated intercept by calculating the linear predictor of the treatment model in SVS-VQI and estimating a model with a single parameter, the intercept in SVS-VQI. We also estimated a model with recalibrated slope and intercept by fitting logistic regression models in SVS-VQI using a single predictor variable with the linear predictor from the CREST-based treatment models. Finally, to estimate the potential real world utility of the combined modelling approach, we compared estimated stroke and MI rates (using intercept and slope recalibrated models) in patients assigned to optimal model assigned treatment compared with the currently assigned treatment and strategies of treating all patients with CAS or CEA in SVS-VQI. Final models and recalibrated models were integrated into an online risk calculator, available at: (https://carotid-risk-calculator.herokuapp.com/riskCalculator/).

    Results

    Baseline risk index performance in crest

    Our Primary Baseline Stroke Model predicted stroke with a c-statistic of 0.64 in CREST, while the Sensitivity Baseline Stroke Model was slightly less predictive with a c-statistic of 0.62. Conversely, the Sensitivity Baseline Stroke Model had slightly better calibration compared with the Primary Baseline Stroke Model which underestimates risk in the highest risk quintile. The Primary Baseline MI Model had a c-statistic of 0.62 with reasonable calibration across the risk spectrum. Predicted baseline MI and stroke risks were correlated. Correlation for predicted risk from the Primary Baseline Stroke Model and the Primary Baseline MI Model was slightly higher (r=0.28, variance explained=0.08) than for the Sensitivity Baseline Stroke Model and the Primary Baseline MI Model (r=0.19, variance explained=0.04). Primary and sensitivity baseline stroke risk models were moderately strongly correlated. (r=0.55, variance explained=0.30).

    Treatment model performance

    The Primary Stroke Treatment Model did not include an interaction between baseline risk and treatment status, but did retain an age-treatment status interaction (as observed in the baseline CREST data) (c-statistic 0.69). The Sensitivity Stroke Treatment Model did retain a baseline risk-treatment interaction (c-statistic 0.68). Coefficients in both treatment models are summarised in table 1. Calibration for the Primary Stroke Treatment Model was reasonable across the risk spectrum. (online supplemental figure 2A). The Primary MI Treatment Model also had moderate predictiveness (0.68) and reasonable calibration (online supplemental figure 2B). There was little evidence of overfitting, mean shrinkage coefficient for the Primary Stroke Treatment Model 1.00 (95% CI 0.99 to 1.01) and the Primary MI Treatment Model : 1.00 (95% CI 1.00 to 1.00). Predicted stroke and MI risk (estimated from the primary treatment models) were correlated, (correlation coefficient=0.28, variance explained=0.08), but the magnitude of the correlation was modest (online supplemental figure 3).

    View this table:
    Table 1

    Summary of regression coefficients for stroke/MI treatment models in crest

    Most patients (86.7%) under the Primary Stroke and MI Treatment Models were predicted to have net benefit (change in stroke risk from CEA-CAS—change in MI risk from CAS-CEA) from CEA (median net benefit 0.8% (IQR 1.8%)). The retention of an interaction term in the model in the Sensitivity Stroke Treatment Models substantially reduced this proportion such that only 54.0% were anticipated to benefit from CEA, although the magnitude of the expected benefit was larger (median net benefit 2.1% (IQR 3.1%)). The distribution of net benefit for both models is illustrated in figure 2. These proportions varied markedly by symptomatic status with 97% of symptomatic patients with predicted net benefit from CEA in the Primary Stroke and MI Treatment Models (median net benefit 2.0%, IQR 0.9%) vs 83% in the Sensitivity Stroke Treatment Model/Primary MI Treatment Model (median net benefit 2.0%, IQR 3.4%). For asymptomatic patients, 75% were predicted to have net benefit from CEA in the Primary Stroke and MI Treatment Models (median net benefit 0.2%, IQR 0.6%) vs 22% in the Sensitivity Stroke Treatment Model/Primary MI Treatment Model (median net benefit −0.8%, IQR 1.3%).

    Figure 2
    Figure 2

    Distribution of predicted net benefit for each individual in crest with treatment on Primary Stroke and MI Treatment Models (A) and the Sensitivity Stroke and Primary MI Treatment Models (B). The y-axis represents the net change in the risk of MI with CEA vs CAS (higher values indicate greater risk with CEA compared with CAS) calculated by taking the predicted probability with CEA on the Primary MI Treatment Model and subtracting the predicted probability with CAS. The x-axis represents the change in stroke risk with CEA versus CAS (higher values indicate greater risk with CAS compared with CEA), calculated by taking the predicted probability with CAS on the primary stroke (A) or sensitivity stroke (B) and subtracting the predicted probability with CEA on the same model. The red line has a slope of 1 and intercept of 0, representing the break-even point where stroke and MI risks are comparable. To the left of the line, the net benefit is greater with CAS and to the right, net benefit is greater with CEA. CAS, carotid artery stenting; CEA, carotid endarterectomy; MI, myocardial infarction.

    External validation in SVS-VQI

    In the combined SVS-VQI dataset, (n=76 269), 14.4% of patients received CAS. The rate of perioperative stroke or death was 1.4% and the rate of perioperative MI was 0.7%, the SVS-VQI population is described in online supplemental table 2. Both Primary Stroke and MI Treatment Models had reduced discrimination in SVS VQI compared with CREST (c-statistic 0.60 for stroke model and 0.54 for MI model). Both models overpredicted risk compared with self-reported event rates in SVS VQI and the magnitude of over prediction was more extreme at the high end of the risk spectrum for both. Recalibrating the model intercept in SVS-VQI attenuated this over prediction, but some over prediction persisted among high-risk patients for both stroke and MI. Models that recalibrated both the slope and intercept had good calibration across the risk spectrum (figure 3).

    Figure 3
    Figure 3

    Calibration of baseline treatment models in SVS-VQI. (A) displays the predicted stroke risk from the CREST treatment model (red bars), the actual stroke risk measured in SVS-VQI (blue bars) and the estimated risk in a model that recalibrated the intercept of the CREST model (green bars) as well as a model that estimated both the slope and intercept of the CREST model in SVS-VQI (yellow bars) . (B) is analogous to (A), but the models are for MI prediction. CREST, Carotid Revascularisation Stenting versus Endarterectomy; MI, myocardial infarction; SVS, Society for Vascular Surgery; VQI, Vascular Quality Initiative.

    The optimal predicted treatment was CEA for 89% vs 11% for CAS in the SVS-VQI sample. The median age among patients for whom CAS was predicted optimal treatment is 53 (IQR 7) vs 72 (IQR 8) for CEA. CEA was the predicted optimal treatment for 84% of patients receiving CAS. Conversely, CAS was the predicted optimal treatment for 10% of patients currently receiving CEA.

    The model-based optimal treatment strategy resulted in the lowest overall event rates (2.04%), but the difference between the as-treated population (2.12%) was quite small and was almost identical to the predicted overall event rate if all patients were treated with CEA (2.05%) . Conversely, the predicted overall event rate was higher for a strategy of treating all patients with CAS (2.63%) (table 2).

    View this table:
    Table 2

    Predicted optimal event rates applying MI and stroke treatment models (with recalibrated slopes and intercepts) in the SVS-VQI dataset

    Discussion

    We found that a combined modelling strategy that stratifies stroke and MI risks while accounting for other factors that predict response to treatment and outcomes has promise to identify the best carotid treatment modality for individual patients. However, for this approach to result in meaningful practical gains, more discriminating and better calibrated tools would be needed to justify the increased complexity.

    In terms of selecting the optimal carotid revascularisation technique our analyses, based on limited imperfect baseline risk models, support current guidelines that generally favour CEA over CAS as initial treatment for most symptomatic patients. Concordant with prior work, we found that CEA results in consistently lower stroke rates and while it is associated with higher MI rates, the magnitude of the effect is small and that across both baseline risk models that more patients would benefit from CEA. Moreover, for the subset where CAS was favoured among symptomatic patients, the magnitude of projected benefit for CAS was quite small both in CREST and SVS-VQI. As a consequence, we found that a treatment policy that offered CEA as a first line treatment to all patients in SVS-VQI would be likely to result in similar overall event rates (and slightly lower stroke rates) as applying the optimal model based prediction. The fact that a simple strategy such as CEA for all would perform almost as well as a policy of is in many ways the strongest argument against applying our current complex model-based strategy clinically. At the same time, our data suggest that for most symptomatic patients that the magnitude of benefit with CEA over CAS is not large—with a risk reduction of about 2 percentage points for CEA for CAS. Consequently, if clinical factors are present that would substantially increase the risk of CEA (but not CAS) by more than this margin, CAS could be considered as an alternative treatment. Given that most existing risk prediction models include factors that likely increase the risk of both CEA and CAS among patient with no anatomic contraindications to open surgery (eg, increased age, medications, qualifying event as a stroke), it is likely that such patients are relatively rare. However, it is not implausible that models with better discrimination might identify a meaningful subset of such patients.

    Our data provide less insight into the optimal treatment for asymptomatic carotid stenosis. Markedly different optimal treatment decisions emerged depending on which baseline risk model was applied. When applying the primary baseline risk model—75% were predicted to benefit from CEA, whereas under the secondary/blood pressure model, 78% were expected to benefit from CAS. Although, under both baseline risk models, the absolute difference in the predicted benefit of CEA versus CAS was fairly small—less than one percentage point for most patients under both baseline risk models. The ongoing CREST2 trial should provide strong data not only on which is the optimal revascularisation modality in this population, but about which patients, if any, should receive revascularisation at all compared with modern medical management.24

    Our experience with these models highlights two issues which may be relevant to future treatment decision modelling: 1. selecting the optimal baseline risk model and 2. challenges in calibration and external validation. Numerous different models have been previously published to predict the baseline risk of perioperative stroke—most based on reasonably high-quality data and with reasonable modelling strategies. When building a treatment model, then, which baseline risk model should be chosen? Our approach to model selection was to pick the model that fit our data best and, practically, could best be applied to the available data. Theoretical considerations led us to also explore a sensitivity analysis model—a trial-derived model that included a continuous physiologic measure, baseline blood pressure, as opposed to the types of binary risk factors included in the other mostly registry-based models. Yet, as described, treatment models based on our primary and secondary baseline risk models arrived at divergent conclusions for asymptomatic patients, even though the baseline risk models were moderately correlated. This divergence is largely due to the retention of a baseline risk-treatment interaction term in the secondary treatment model, leading to a lower predicted stroke risk in low baseline risk patients in the secondary model compared with the primary model. In retrospect, we do not see an obviously superior strategy for selecting a baseline risk model. Instead, we would generally argue that the best approach would be to apply the model that predicts most accurately in the target population. In this case, unfortunately, the absence of a baseline blood pressure measures in SVS-VQI, precludes us from comparing model performance in our target population. More predictive baseline risk models, incorporating both physiologic, anatomic (eg, plaque morphology, high risk features) and risk factor data, are likely needed to improve the accuracy and consistency of predictions of the optimal carotid revascularisation strategy.

    The problem of adequately calibrating risk models relative to their target population is common. As a particularly salient example, the atherosclerotic cardiovascular disease risk score that is guideline-supported to target statin therapy in primary prevention has varying calibration depending on the study population.25 In this analysis, we found that our trial-based treatment risk models predict considerably higher risk than what was observed in the SVS-VQI registry. While this may occur for many reasons, the two most plausible reasons are that the CREST population was higher risk than the SVS-VQI population or that outcomes are under-reported in SVS-VQI. It is unlikely that the CREST population was at markedly higher baseline risk than the SVS-VQI population. First, CREST did not have particularly restrictive inclusion criteria, but to the extent that it did, it likely biased the population to a lower risk population (ie, excluding patient with prior disabling stroke).26 Moreover, clinical trial populations generally select for somewhat lower risk populations than in the general population,27 28 and in prior carotid trials, those selected into the trials likely had lower risk than the treated populations at those centres.29 Second, optimal medical management is likely an important factor in limiting stroke risk in this population and was explicitly attended to in the CREST trial.2 Conversely, it is quite plausible that the self-reported outcomes in SVS-VQI under-report some events. While CREST’s evaluation of stroke risk included a systematic examination and screening instrument applied to all patients, no specific evaluation strategy is specified in SVS-VQI. What is reported, then, is likely a reflection only of events that are clinically diagnosed and likely underdiagnoses some mild strokes. Regardless, neither interpretation is necessarily a fundamental barrier to the application of our carotid models, nor of risk models more generally. Based on the features of one’s practice environment, one could reasonably choose to apply either the CREST-based or SVS-VQI based risk estimates. The optimal approach would likely be to recalibrate trial-derived models in the target population, to account for differences between the trial and target populations and intervention characteristics, in an approach that accounts for measurement error within the target-population ascertained outcomes.30

    Conclusion

    Our data suggest that decomposing individual elements of risk (eg, stroke and MI) could potentially be a valuable approach for individualising the CEA versus CAS treatment decision, but would require risk prediction tools that are somewhat more discriminant and locally calibrated. For selection of carotid revascularisation, our data support guideline-based recommendations to favour CEA over CAS, and strongly support this idea for symptomatic patients.

    Data availability statement

    Data are available in a public, open access repository. Data may be obtained from a third party and are not publicly available. CREST data are available from NINDS. SQS-VQI data are available via application to SVS-VQI.

    Ethics statements

    Patient consent for publication

    Ethics approval

    The study protocol was reviewed by the University of Michigan Institutional Review Board and consider exempt as it relied on deidentified or limited datasets.

    References

    1. ↵
      2. Sardar P ,
      3. Chatterjee S ,
      4. Aronow HD , et al
      . Carotid artery stenting versus endarterectomy for stroke prevention: a meta-analysis of clinical trials. J Am Coll Cardiol 2017;69:2266–75.doi:10.1016/j.jacc.2017.02.053 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28473130
      OpenUrlFREE Full Text
    2. ↵
      2. Brott TG ,
      3. Hobson RW ,
      4. Howard G , et al
      . Stenting versus endarterectomy for treatment of carotid-artery stenosis. N Engl J Med 2010;363:11–23.doi:10.1056/NEJMoa0912321 pmid:http://www.ncbi.nlm.nih.gov/pubmed/20505173
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      2. Brott TG ,
      3. Howard G ,
      4. Roubin GS , et al
      . Long-Term results of stenting versus endarterectomy for carotid-artery stenosis. N Engl J Med Overseas Ed 2016;374:1021–31.doi:10.1056/NEJMoa1505215
      OpenUrl
    4. ↵
      2. Brott TG ,
      3. Halperin JL
      . 2011 ASA/ACCF/AHA/AANN/AANS/ACR/ASNR/CNS/SAIP/SCAI/SIR/SNIS/SVM/SVS guideline on the management of patients with extracranial carotid and vertebral artery disease: Executive summary. A report of the American College of cardiology Foundation/American heart association Task force on practice guidelines, and the American stroke association, American association of neuroscience nurses, American association of neurological Surgeons, American College of radiology, American Society of Neuroradiology, Congress of neurological Surgeons, society of atherosclerosis imaging and prevention, Society for cardiovascular angiography and interventions, society of interventional radiology, society of NeuroInterventional surgery, Society for vascular medicine, and Society for vascular surgery. Circulation 2011;124.doi:10.1161/CIR.0b013e31820d8d78
    5. ↵
      2. Yadav JS ,
      3. Wholey MH ,
      4. Kuntz RE , et al
      . Protected carotid-artery stenting versus endarterectomy in high-risk patients. N Engl J Med 2004;351:1493–501.doi:10.1056/NEJMoa040127 pmid:http://www.ncbi.nlm.nih.gov/pubmed/15470212
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      2. Rothwell PM ,
      3. Eliasziw M ,
      4. Gutnikov SA , et al
      . Analysis of pooled data from the randomised controlled trials of endarterectomy for symptomatic carotid stenosis. Lancet 2003;361:107–16.doi:10.1016/S0140-6736(03)12228-3 pmid:http://www.ncbi.nlm.nih.gov/pubmed/12531577
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      2. Rothwell PM ,
      3. Slattery J ,
      4. Warlow CP
      . Clinical and angiographic predictors of stroke and death from carotid endarterectomy: systematic review. BMJ 1997;315:1571–7.doi:10.1136/bmj.315.7122.1571 pmid:http://www.ncbi.nlm.nih.gov/pubmed/9437274
      OpenUrlAbstract/FREE Full Text
    8. ↵
      2. Rothwell PM
      . Can overall results of clinical trials be applied to all patients? The Lancet 1995;345:1616–9.doi:10.1016/S0140-6736(95)90120-5
      OpenUrl
    9. ↵
      2. Rothwell PM ,
      3. Eliasziw M ,
      4. Gutnikov SA , et al
      . Endarterectomy for symptomatic carotid stenosis in relation to clinical subgroups and timing of surgery. Lancet 2004;363:915–24.doi:10.1016/S0140-6736(04)15785-1 pmid:http://www.ncbi.nlm.nih.gov/pubmed/15043958
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      2. Hobson RW ,
      3. Howard VJ ,
      4. Roubin GS , et al
      . Carotid artery stenting is associated with increased complications in octogenarians: 30-day stroke and death rates in the crest lead-in phase. J Vasc Surg 2004;40:1106–11.doi:10.1016/j.jvs.2004.10.022 pmid:http://www.ncbi.nlm.nih.gov/pubmed/15622363
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      2. Cronenwett JL ,
      3. Kraiss LW ,
      4. Cambria RP
      . The Society for vascular surgery vascular quality initiative. J Vasc Surg 2012;55:1529–37.doi:10.1016/j.jvs.2012.03.016 pmid:http://www.ncbi.nlm.nih.gov/pubmed/22542349
      OpenUrlCrossRefPubMed
    12. ↵
      2. Desai SS ,
      3. Kaji AH ,
      4. Upchurch G
      . Practical guide to surgical data sets: Society for vascular surgery vascular quality initiative (SVS VQI). JAMA Surg 2018;153:957–8.doi:10.1001/jamasurg.2018.0498 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29617537
      OpenUrlPubMed
    13. ↵
      2. Aiello FA ,
      3. Shue B ,
      4. Kini N , et al
      . Outcomes reported by the vascular quality initiative and the National surgical quality improvement program are not comparable. J Vasc Surg 2014;60:152–9.doi:10.1016/j.jvs.2014.01.046
      OpenUrlCrossRefPubMed
    14. ↵
      2. Lee TH ,
      3. Marcantonio ER ,
      4. Mangione CM , et al
      . Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation 1999;100:1043–9.doi:10.1161/01.CIR.100.10.1043 pmid:http://www.ncbi.nlm.nih.gov/pubmed/10477528
      OpenUrlAbstract/FREE Full Text
    15. ↵
      2. Ford MK ,
      3. Beattie WS ,
      4. Wijeysundera DN
      . Systematic review: prediction of perioperative cardiac complications and mortality by the revised cardiac risk index. Ann Intern Med 2010;152:26–35.doi:10.7326/0003-4819-152-1-201001050-00007 pmid:http://www.ncbi.nlm.nih.gov/pubmed/20048269
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      2. Nolan BW ,
      3. De Martino RR ,
      4. Goodney PP , et al
      . Comparison of carotid endarterectomy and stenting in real world practice using a regional quality improvement registry. J Vasc Surg 2012;56:990–6.doi:10.1016/j.jvs.2012.03.009 pmid:http://www.ncbi.nlm.nih.gov/pubmed/22579135
      OpenUrlCrossRefPubMed
    17. ↵
      2. Calvillo-King L ,
      3. Xuan L ,
      4. Zhang S , et al
      . Predicting risk of perioperative death and stroke after carotid endarterectomy in asymptomatic patients. Stroke 2010;41:2786–94.doi:10.1161/STROKEAHA.110.599019
      OpenUrlAbstract/FREE Full Text
    18. ↵
      2. Gupta PK ,
      3. Ramanan B ,
      4. Mactaggart JN , et al
      . Risk index for predicting perioperative stroke, myocardial infarction, or death risk in asymptomatic patients undergoing carotid endarterectomy. J Vasc Surg 2013;57:318–26.doi:10.1016/j.jvs.2012.08.116 pmid:http://www.ncbi.nlm.nih.gov/pubmed/23159474
      OpenUrlCrossRefPubMed
    19. ↵
      2. Goodney PP ,
      3. Likosky DS ,
      4. Cronenwett JL , et al
      . Factors associated with stroke or death after carotid endarterectomy in northern new England. J Vasc Surg 2008;48:1139–45.doi:10.1016/j.jvs.2008.05.013 pmid:http://www.ncbi.nlm.nih.gov/pubmed/18586446
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      2. Rothwell PM ,
      3. Warlow CP
      . Prediction of benefit from carotid endarterectomy in individual patients: a risk-modelling study. European carotid surgery Trialists' Collaborative group. Lancet 1999;353:2105–10.doi:10.1016/s0140-6736(98)11415-0 pmid:http://www.ncbi.nlm.nih.gov/pubmed/10382694
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      2. Hayward RA ,
      3. Kent DM ,
      4. Vijan S , et al
      . Multivariable risk prediction can greatly enhance the statistical power of clinical trial subgroup analysis. BMC Med Res Methodol 2006;6:18. doi:10.1186/1471-2288-6-18 pmid:http://www.ncbi.nlm.nih.gov/pubmed/16613605
      OpenUrlCrossRefPubMed
    22. ↵
      2. Posada D ,
      3. Buckley TR
      . Model selection and model averaging in phylogenetics: advantages of akaike information criterion and Bayesian approaches over likelihood ratio tests. Syst Biol 2004;53:793–808.doi:10.1080/10635150490522304 pmid:http://www.ncbi.nlm.nih.gov/pubmed/15545256
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      2. Steyerberg EW ,
      3. Harrell FE ,
      4. Borsboom GJ , et al
      . Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001;54:774–81.doi:10.1016/s0895-4356(01)00341-9 pmid:http://www.ncbi.nlm.nih.gov/pubmed/11470385
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      2. Howard VJ ,
      3. Meschia JF ,
      4. Lal BK , et al
      . Carotid revascularization and medical management for asymptomatic carotid stenosis: protocol of the CREST-2 clinical trials. Int J Stroke 2017;12:770–8.doi:10.1177/1747493017706238 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28462683
      OpenUrlPubMed
    25. ↵
      2. Ridker PM ,
      3. Cook NR
      . Statins: new American guidelines for prevention of cardiovascular disease. The Lancet 2013;382:1762–5.doi:10.1016/S0140-6736(13)62388-0
      OpenUrl
    26. ↵
      2. Sheffet AJ ,
      3. Roubin G ,
      4. Howard G , et al
      . Design of the carotid revascularization endarterectomy vs. stenting trial (crest). Int J Stroke 2010;5:40–6.doi:10.1111/j.1747-4949.2009.00405.x pmid:http://www.ncbi.nlm.nih.gov/pubmed/20088993
      OpenUrlCrossRefPubMedWeb of Science
    27. ↵
      2. Rothwell PM
      . Factors that can affect the external validity of randomised controlled trials. PLoS Clin Trials 2006;1:e9. doi:10.1371/journal.pctr.0010009 pmid:http://www.ncbi.nlm.nih.gov/pubmed/16871331
      OpenUrlCrossRefPubMed
    28. ↵
      2. Rothwell PM
      . External validity of randomised controlled trials: "to whom do the results of this trial apply?". Lancet 2005;365:82–93.doi:10.1016/S0140-6736(04)17670-8 pmid:http://www.ncbi.nlm.nih.gov/pubmed/15639683
      OpenUrlCrossRefPubMedWeb of Science
    29. ↵
      2. Wennberg DE ,
      3. Lucas FL ,
      4. Birkmeyer JD , et al
      . Variation in carotid endarterectomy mortality in the Medicare population. JAMA 1998;279:1278–81.doi:10.1001/jama.279.16.1278
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      2. Sussman JB ,
      3. Wiitala WL ,
      4. Zawistowski M , et al
      . The Veterans Affairs cardiac risk score: recalibrating the atherosclerotic cardiovascular disease score for applied use. Med Care 2017;55:864–70.doi:10.1097/MLR.0000000000000781 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28763374
      OpenUrlPubMed

    Supplementary material

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Contributors JFB conceived the research, acquired data, executed analyses and drafted the manuscript. RAH and LBM conceived the research, supervised and provided detailed feedback on the manuscript and analyses. NHO participated in data acquisition, study design and provided detailed feedback on the manuscript.

    • Funding This work was funded by the National Institute of Health (NINDS, K08 NS082597).

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.