Background and purpose The Trial of Org 10 172 in Acute Stroke Treatment (TOAST) system is the most widely used aetiological categorisation system in clinical practice and research. Limited studies have validated the accuracy of routine aetiological diagnosis of patients with ischaemic stroke according to the TOAST criteria when the reported subtype is assumed to be correct. We investigated the agreement between centralised and non-centralised (site-reported, at discharge) stroke subtypes in the Third China National Stroke Registry (CNSR-III), and analysed the influence of classification consistency on evaluation during hospitalisation and for secondary prevention strategy.
Methods All patients with ischaemic stroke from the CNSR-III study with complete diffusion-weighted imaging data were included. We used multivariable Cox proportional-hazard regression models to evaluate the factors associated with consistency between centralised and non-centralised stroke subtypes. Sensitivity analyses were conducted of the subgroup of patients with complete information.
Results This study included 12 180 patients (mean age, 62.3 years; and women, 31.4%). Agreement between centralised and non-centralised subtype was the highest for the large-artery atherosclerosis subtype stroke (77.4% of centralised patients), followed by the small-vessel occlusion subtype (40.6% of centralised patients). Agreements for cardioembolism and stroke of other determined aetiology subtypes were 38.7% and 12.2%, respectively. Patient-level and hospital-level factors were associated with the inconsistency between centralised/non-centralised aetiological subtyping. This inconsistency was related to differences in secondary prevention strategies. Only 15.3% of the newly diagnosed patients with cardioembolism underwent centralised subtyping with indications to receive oral anticoagulants at discharge. In comparison, 51.3% of the consistent cardioembolism group and 42.0% of the centrally reassigned cardioembolism group with anticoagulation indications were prescribed oral anticoagulants.
Conclusions Substantial inconsistency exists between centralised and non-centralised subtyping in China. Inaccurate aetiological subtyping could lead to inadequate secondary prevention, especially in patients with cardioembolic stroke.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Stroke has become the second leading cause of death worldwide1 and the leading cause of mortality in China.2 Ischaemic stroke accounts for 69.6% of all strokes and the incidence of stroke has continued to steeply rise over the last three decades.3 4 Although gradual improvements in stroke care have been achieved in recent years, regional differences still exist in the quality of diagnostic measures and stroke recurrence.5–7
Accurate classification of ischaemic stroke aetiology is indispensable for patient management and research because stroke prognosis8 and strategies for secondary prevention of stroke differ by stroke subtype.9–11 There are several aetiological subtyping systems12–19; however, despite the limited discriminatory ability and prognostic value,8 the Trial of Org 10 172 in Acute Stroke Treatment (TOAST) system12 remains the most widely used aetiological categorising system in clinical research and practice.20 21
In clinical practice and most clinical trials, aetiological diagnosis is based on the judgement of a local neurologist or site investigator. Previous studies have reported only moderate inter-rater reliability (κ: 0.42–0.64) with the TOAST system.22 23 Both standardised medical record review24 and centralised adjudication25 have been shown to improve this situation. However, the accuracy of the routine aetiological diagnosis of ischaemic stroke according to TOAST criteria when the reported subtype is assumed to be correct has not been validated yet.
To the best of our knowledge, no large-scale study has compared the centralised versus non-centralised aetiological diagnosis of ischaemic stroke. The potential for heterogeneity in patient treatment following a centralised versus non-centralised diagnosis also warrants discussion. We, therefore, investigated the consistency between site-reported stroke subtypes at discharge according to centralised and non-centralised TOAST criteria for stroke diagnosis in the Third China National Stroke Registry (CNSR-III), its influence on aetiological evaluation during hospitalisation and its impact on secondary prevention strategy.
This study is an analysis of patients within the CNSR-III, a prospective patient cohort with ischaemic stroke or transient ischaemic attack (TIA) in China. Patients in our study were recruited between August 2015 and March 2018 from 201 study sites covering 26 provinces and municipalities in China.26 Detailed inclusion and exclusion criteria are described in the CNSR-III study protocol.
Study population and standardised diagnoses
The present analysis included all patients with ischaemic stroke with complete diffusion-weighted imaging (DWI) data as well as a complete aetiological evaluation and a non-centralised TOAST subtype designated by a site investigator at discharge. Site investigators and evaluators made decisions based on the 1993 version of the TOAST standard,12 according to a previously published study protocol.26 The TOAST system stratifies patients with ischaemic stroke into five subtypes: large artery atherosclerosis (LAA), cardioembolism (CE), small-vessel occlusion (SVO), stroke of other aetiology (OE) and stroke of undetermined cause (UE). A detailed description of the training process is included in the online supplemental materials (see online supplemental methods).
Centralised and non-centralised subtyping
To ensure that the diagnosis standard was consistent, we adopted the use of just one version of the TOAST subtype system12 for the centralised decision algorithm. A detailed methodology of the centralised and non-centralised subtyping is described in the supplementary methods (see online supplemental methods, online supplemental materials-screening report form and online supplemental materials-decision algorithm).
A χ2 test or a Fisher’s exact test was used to compare categorical variables. Continuous variables were compared using either one-way analysis of variance or the Kruskal-Wallis test, where appropriate. The heterogeneity of centralised versus non-centralised subtype consistency was assessed using the χ2 test. To analyse the potential causes of differences between centralised and non-centralised subtypes, inpatient department, hospital level and geographic location were also compared. Patients were further stratified into three categories per subtype based on centralised/non-centralised subtype consistency: consistent, centrally reassigned and newly diagnosed. For each specific subtype, the centrally reassigned group was defined as the group of patients diagnosed with the subtype at discharge but reassigned to a different subtype after centralised diagnosis. The newly diagnosed group for a given subtype included patients who were initially diagnosed with a subtype at discharge and then newly diagnosed with the given subtype after centralised diagnosis. For example, a patient originally diagnosed as OE and then centrally reassigned as CE would be included in the centrally reassigned group for OE and the newly diagnosed group for CE. We used multivariate Cox proportional hazards regression to evaluate the factors associated with the inconsistency between centralised and non-centralised subtypes. Variables showing statistical differences in univariate analysis at baseline were modelled as confounding factors. We imputed missing values of the baseline variables used to adjust the regression models with mean or mode, as appropriate. No outcomes were imputed. In-hospital and discharge medication prescriptions were displayed for each category in every subtype. Indications of secondary prevention strategies were determined in accordance with established guidelines.10 27 We performed sensitivity analyses using only the patient subgroup with complete information, that is, having DWI imaging, ≥1 extracranial artery evaluation, ≥1 intracranial artery evaluation, ≥1 cardiac rhythm evaluation (ECG (EKG) or 24-hour Holter) and ≥1 cardiac structure evaluation (transthoracic echocardiography (TTE) or transoesophageal echocardiography).
A two-sided p value <0.05 was considered to be statistically significant. All statistical analyses were performed using SAS V.9.4 (SAS Institute, Cary, North Carolina, USA).
Participant distribution and baseline characteristics
This study included a total of 12 180 patients with acute ischaemic stroke after excluding 1020 patients with TIA and 1966 patients with incomplete MRI data (figure 1). The included patients were older and had a lower rate of TIA history, myocardial infarction history and atrial fibrillation history; however, a higher percentage of patients had dyslipidaemia, were current smokers and or were diagnosed with cerebral infarction (see online supplemental table I).
Baseline characteristics of the study population are listed in online supplemental table II). The average age of the study population was 62.3 years (SD: 11.2 years), and 31.4% of all the included patients were women. A total of 11 586 (95.1%) patients underwent intracranial artery evaluation and 11 454 (94.0%) patients underwent a complete extracranial artery test. TTE data were available for 94.7% of the patients, EKG data for 94.3% and Holter data for 83.6%.
Consistency between centralised and non-centralised subtype distributions
Differences between the distributions of aetiological subtypes based on discharge and those based on the centralised subtyping system are presented in figure 2. Substantial differences were observed between the centralised and non-centralised subtypes. Only 4316 patients (35.4%) were assigned to the same aetiological category after the centralised subtyping procedure performed by the site investigator at discharge. Agreement between the centralised and non-centralised subtypes was highest for the LAA subtype (77.4% of patients centrally diagnosed as LAA), followed by the SVO type (40.6% of patients centrally diagnosed as SVO). The agreement of CE and OE subtypes was 38.7% and 12.2%, respectively. Similar results were observed in the subgroup of patients with complete information.
To identify the potential reasons for the inconsistency between discharge and centralised subtype diagnosis, we compared both patient-level (demographic characteristics, baseline National Institutes of Health Stroke Scale (NIHSS) and so on) and hospital level (inpatient department, hospital type and geographic location or area of the hospital) factors (see online supplemental table III).
In multivariate logistic regression analysis, a history of ischaemic stroke (adjusted OR 0.841; 95% CI 0.749 to 0.943), TIA (adjusted OR 0.606; 95% CI 0.451 to 0.815) and dyslipidaemia (adjusted OR, 0.822; 95% CI 0.689 to 0.980) were each associated with fewer patients who were centrally reassigned from the LAA subtype. Additionally, hospitals in the urban area (adjusted OR 0.792; 95% CI 0.687 to 0.913) and tertiary hospitals (adjusted OR 0.748; 95% CI 0.627 to 0.891) were each associated with fewer patients centrally reassigned from the LAA subtype. Furthermore, we observed a significant interaction (p=0.005) of hospital area with hospital type in the consistency of LAA diagnosis. Non-stroke unit inpatient departments were associated with more patients being centrally reassigned from the LAA subtype (adjusted OR 1.140; 95% CI 1.012 to 1.284). Hospitals in the middle (adjusted OR 0.445; 95% CI 0.330 to 0.600) and the eastern (adjusted OR 0.476; 95% CI 0.359 to 0.631) regions of China were both associated with less newly diagnosed patients, indicating a more consistent LAA diagnosis.
A history of atrial fibrillation or flutter was associated with a more consistent diagnosis of the CE subtype (centrally reassigned CE group: adjusted OR 0.122; 95% CI 0.070 to 0.213; newly diagnosed CE group: adjusted OR 0.086; 95% CI 0.053 to 0.140). Evaluation of cardiac rhythm via 24-hour Holter was related to a more consistent CE subtype diagnosis (newly diagnosed CE group: adjusted OR 0.567; 95% CI 0.350 to 0.919).
A higher baseline NIHSS score was associated with more centrally reassigned patients with SVO (adjusted OR 1.360; 95% CI 1.164 to 1.588) and more newly diagnosed patients with SVO (adjusted OR 1.452; 95% CI 1.236 to 1.707). Additionally, tertiary hospitals were associated with more newly diagnosed patients with SVO (adjusted OR 1.410; 95% CI 1.140 to 1.743), while hospitals in urban areas were associated with fewer newly diagnosed patients with SVO (adjusted OR 0.446; 95% CI 0.360 to 0.552). An interaction was observed between hospital type, location/area and consistency of SVO diagnosis (p=0.037).
Older age was associated with a high proportion of being centrally reassigned from the OE subtype (adjusted OR 1.062; 95% CI 1.022 to 1.102) and being newly diagnosed with OE (adjusted OR 1.062; 95% CI 1.016 to 1.109). Several patient-level and hospital-level factors were associated with agreement between centralised and non-centralised aetiological subtyping (table 1).
Aetiological diagnosis and patient management
Fewer patients with newly diagnosed CE underwent 24-hour Holter (82.1%) than the patients in both the consistent CE (87.8%) and centrally reassigned CE groups (87.7%; see online supplemental table III). The prescription rate of oral anticoagulants for secondary prevention was lower in newly diagnosed patients with CE (10% vs 50.2% for the consistent CE group and 27.6% for the centrally reassigned CE group, p<0.001). The prescription rate of antihypertensive agents was the highest in the consistent SVO group (48.7% vs 42.7% for the centrally reassigned SVO group and 44.3% for the newly diagnosed SVO group). Prescription rates for guideline-recommended treatments other than antithrombotic agents (such as lipid-lowering, antidiabetic and antihypertensive treatments) were comparable among the different groups based on the consistency of centralised and non-centralised subtyping in patients with LAA (table 2). Similar results were observed in the patient subgroup with complete information (see online supplemental table IV).
Of the 449 newly diagnosed patients with CE stroke, 431 (95.9%) had high-risk sources of CE. Among the high-risk sources, the largest proportion was newly diagnosed atrial fibrillation (58.47%), followed by a history of atrial fibrillation (28.54%; figure 3).
Oral anticoagulant prescription rates were significantly different among the three groups of patients with CE categorised by agreement between centralised and non-centralised subtyping. Among patients with indications for anticoagulation therapy, the prescription rate of oral anticoagulants at discharge in the newly diagnosed CE group was surprisingly lower (15.3%) than that in the consistent CE (51.3%) and the centrally reassigned CE groups (42.0%; table 3). Similar results were also observed in the patient subgroup with complete information (see online supplemental table V).
In the current study, we found substantial differences between centralised and non-centralised aetiological classifications of stroke subtypes in China. Factors related to the heterogeneity between centralised and non-centralised subtyping included the hospital level, geographical region/location and area of the admitting hospital; the admitting department and stroke severity. Incomplete aetiological investigation during hospitalisation and inadequate secondary prevention strategies were observed in patients with inconsistent classification by site evaluators (non-centralised) using centralised adjudication. Furthermore, we found that both a high baseline NIHSS score and admission to a stroke unit were each associated with a more consistent aetiological diagnosis between centralised and non-centralised stroke subtyping. Thus, differences in evaluation and care quality among patients with different severities might lead to inaccurate aetiological diagnoses in patients with mild neurological deficits.
The proportion of distribution of each subtype after centralised and non-centralised TOAST classification was similar to those reported in a previous registry,28 with LAA accounting for a large proportion of ischaemic stroke and CE accounting for a small proportion. The particularly high prevalence of intracranial artery stenosis in Chinese patients with stroke29 might explain the high-ranking proportion of the LAA subtype relative to the other subtypes. However, the proportion of LAA subtype among all patients classified by the non-centralised or centralised designations was different (60.8% vs 26.7%). A strict application of the TOAST classification criteria can lead to the designation of a significant number of strokes as an undetermined cause category.14 A neurologist’s ‘clinical opinion’, based on experience, might help to assign a high degree of confidence to one specific stroke cause in a particular patient when facing competing evidence of different aetiologies. Recent subtyping systems, such as the Chinese Ischaemic Stroke Subclassification (CISS) criteria, emphasise underlying pathological mechanisms and contain revised standards for categorising the LAA subtype.19 According to CISS criteria, patients with infarct in the territory of an isolated penetrating artery and evidence of atherosclerotic plaque or any degree of stenosis in the parent artery are categorised into the LAA subtype, regardless of the degree of luminal stenosis of the clinically relevant artery. With the continuously increasing development of novel techniques for plaque detection and vessel wall imaging, mild intracranial artery stenosis (<50% luminal stenosis) or non-stenotic atherosclerotic plaques are being recognised in >50% of patients with ischaemic symptoms.30 These new subtyping criteria have had a profound influence on Chinese neurologists and might bias their ‘clinical opinions’ towards the LAA subtype.
Unlike the considerably high incidence of the CE subtype in the Caucasian population,31 CE stroke accounts for only 5.7% of ischaemic strokes among Chinese patients, even with a complete aetiological investigation. Disagreements between non-centralised and centralised aetiological diagnosis of the CE subtype were mainly caused by the detection of atrial fibrillation (AF) after ischaemic stroke. AF is a well-established predictor of stroke recurrence and has a validated association with cardiogenic embolism.32 The AF-SCREEN International Collaboration recommends 72-hour electrocardiographic monitoring for AF detection.33 Because the CNSR-III protocol was written before publication of the white paper,33 a stepwise approach was used to evaluate AF using resting EKG followed by 24-hour Holter monitoring. The current diagnostic approach detected AF-related stroke in 252 patients and reassigned them to the CE category. Another reason for subtyping inconsistency might be the availability of complementary information from the centralised data collection. Newly observed high-risk or medium-risk cardiac embolism sources within centralised data contributed to 62.3% of cardioembolic stroke classifications.
Inconsistency between non-centralised and centralised classifications of the CE subtype was associated with inadequate guideline-recommended secondary prevention strategy, especially with regard to indications for treatment with oral anticoagulants. Among patients with ischaemic stroke with non-valvular atrial fibrillation (NVAF) in the CNSR without contraindications to anticoagulation, only 16.2% received warfarin therapy as secondary prevention.34 In the second CNSR (CNSR-II), the prescription rate of warfarin for patients with stroke with NVAF remained low (19.4%).35 Although an overall increase was observed in the use of oral anticoagulants in the current analysis, we found that the prescription rate in patients with unrecognised CE with indications of anticoagulation treatment (15.3%) was even lower than that reported in previous studies.34 35 Inaccurate diagnosis of CE aetiology could cause suboptimal anticoagulation treatment in patients with CE indicated for anticoagulant therapy. Enhancing the accuracy and reliability of aetiological diagnosis is an essential step to improve adherence to guideline-recommended secondary prevention treatment.
Our study has several strengths. We used an algorithm for centralised adjudication of aetiology categories, which integrated all necessary data elements collected and processed centrally by trained specialists. Furthermore, this study was based on a cohort in which >94% of all included patients underwent complete aetiological examinations. The centralised aetiology classification was reproducible and stable because it eliminated any disagreements between evaluators in the judgement step.
However, this study had certain limitations. First, the algorithm was designed based on previously published rules.12 New perspectives to stratify the sources of cardiac embolism risk are continually being updated.36 Efficient treatments for different potential embolic sources continue to be developed, such as percutaneous patent foramen ovale (PFO) closure for PFO-related stroke.37–39 For future use, the algorithm employed in the current study needs to be updated with the most recent knowledge. Second, many patients in the current study were assigned to the undetermined causes category with ≥1 competing mechanism after completing the diagnostic workup. It is thus essential to refine the hierarchical competing standard among subtype elements. Third, the standard evaluation process in the current study lacked cardiac MRI, 72-hour EKG monitoring and specific biomarkers for cardiac function. Fourth, the evaluators at each study site were trained in the use of videotape. The intra-rater and inter-rater reliabilities were not centrally assessed before aetiological diagnosis was performed. However, this reflects a real-world challenge of evaluating interobserver and intraobserver agreements of aetiological diagnoses owing to the large scale of neurological departments and constantly rotating physicians in the many hospitals in China. Efforts to improve the quality of centralised training processes and constant interactive feedback are needed to narrow the gap between non-centralised and centralised subtyping in future multicentre studies.
We report substantial differences between discharge and centralised diagnosis of stroke aetiology. The factor most associated with these differences was the inhomogeneity among admitting departments and hospitals in their ability to identify potential stroke mechanisms. An inadequate secondary prevention strategy was observed in patients with inconsistent classification by site evaluators (non-centralised) and centralised adjudicators.
Contributors Y-JW had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Y-JW, JJ and YS designed the protocol and framework of this study. YS, JJ, XM, YJ and YP collected the data. Huan Liu and HY conducted statistical analysis. YS drafted the manuscript. ZL, YJ, XY, LL, XZ, Y-LW and Hao Li reviewed, edited and approved the final version.
Funding This study was funded by National Science and Technology Major Project National Key R&D Program of China Beijing Municipal Science & Technology Commission (D171100003017002).
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available on reasonable request.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.