Article Text
PDF
Abstract
Introduction Vascular diseases, such as stroke and heart failure (HF), are associated with cognitive decline. Vascular cognitive impairment (CI) is commonly found in patients who had a stroke and with HF, ranging from mild CI to dementia. Early detection of CI is crucial for effective management and rehabilitation. This study aimed to develop the VasCog Screen test, a screening tool to detect CI in patients who had a stroke and with HF.
Method 427 patients who had a stroke and with HF were assessed using cognitive measures including Mini-Mental State Examination (MMSE), Montreal Cognitive Assessment (MoCA) and a formal neuropsychological battery. The short-MoCA was derived and combined with Symbol Digit Modalities Test (SDMT) to create the VasCog Screen. The discriminatory ability of different tests for CI was compared, establishing optimal cut-off points. Variants of short-MoCA including the SDMT were also evaluated.
Results Similar prevalence rates of CI were found in stroke and HF cohorts. The most prevalent neuropsychological impairment was visuomotor speed, followed by visual memory and visuoconstruction. More than half of the patients were found to have CI. The VasCog Screen outperformed MMSE, MoCA and short-MoCA in detecting CI. The addition of SDMT to variants of the short-MoCA increased diagnostic accuracy.
Conclusion The VasCog Screen test offers a cognitive screening tool, which is sensitive to cognitive deficits characteristically found in patients who had a stroke and with HF. It was found to have good sensitivity, specificity and classification accuracy. It is easy to administer in busy clinics, enabling early detection of CI and facilitating appropriate interventions.
- Stroke
- Cerebrovascular Disorders
- Cognitive Dysfunction
- Brain
Data availability statement
No data are available.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Existing cognitive screening tools have limitations in terms of sensitivity, administration time and the lack of assessing processing speed which is commonly impacted in vascular cognitive impairment. A sensitive screening tool which includes the assessment of processing speed needs to be developed for use in this population.
WHAT THIS STUDY ADDS
This study shows that the VasCog Screen test outperformed existing cognitive screening tools in detecting cognitive impairment in patients who had a stroke and with heart failure. The test has shown good sensitivity, specificity and classification accuracy, making it suitable for early detection of cognitive impairment and facilitating timely interventions.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
The study findings support the use of the VasCog Screen test as an effective screening tool for cognitive impairment in patients who had a stroke and with heart failure, which addresses the limitations of using existing screening tests in this population.
Introduction
Vascular diseases, such as stroke and heart failure (HF), are a major public health burden in Asia1 and related to accelerated cognitive decline in individuals.2 Cognitive impairment (CI) related to vascular diseases (ie, HF, stroke) is referred to as vascular CI, which ranges from mild CI (MCI) to dementia. Vascular CI has been reported in more than 33% of stroke survivors3 4 and 25–80% of patients with HF.5 6
The profile of CI after stroke is inherently intricate due to the heterogenous nature of stroke (ie, type of stroke, location of lesion, vascular territories implicated) and its impact on cerebral function.7 The two most common cognitive deficits seen after stroke are aphasia and hemispatial neglect, particularly from stroke occurring in a localised area.7 8 Apraxia is also commonly seen after a left hemisphere stroke.8 Studies have reported heterogeneous neuropsychological patterns of poststroke CI, with an emphasis on deficits in processing speed and frontal–executive function and relative sparing of episodic memory.9 10 CI has also been suggested to be underdiagnosed in patients with HF, with research suggesting a strong relationship between the two.11 A meta-analysis and systematic review of 33 articles examining the neuropsychological functioning of patients with HF found the greatest impairments in global cognition, processing speed, executive functioning and verbal memory.12
In a paper describing two systematic meta-analytic reviews, 27 studies evaluating neuropsychological profiles between vascular CI and healthy controls, and 20 studies evaluating neuropsychological profiles between vascular CI and non-vascular MCI were included.13 Individuals with vascular CI were found to show greatest impairment in processing speed when compared with healthy controls, while they were found to show greater deficits in processing speed and executive functioning when compared with individuals with non-vascular MCI.13 The authors suggest that the cognitive deficits seen are resultant of the disruption to subcortical white matter tracts, commonly seen after cerebral small vessel disease. Subcortical changes in white matter are also thought to occur after HF due to lowered cardiac output and decreased oxygenation to the brain and body. Both cardiovascular and cerebrovascular diseases lead to increased subcortical white matter hyperintensities which can explain a similar neurocognitive profile.14
With this knowledge, cognitive testing after stroke or HF is integral to patient care, as it can allow providers to better plan for rehabilitation. Formal neuropsychological assessments are time, manpower and resource intensive, as it can take several hours to complete. Undergoing a brief screening test first, and only conducting formal neuropsychological assessments for patients who screen positive for CI would save time and costs. It has been proposed that screening tests for vascular CI should cover the following domains: (1) attention and processing speed; (2) frontal–executive function; (3) learning and memory; (4) language; (5) visuoconstructional–perceptual ability; (6) praxis–gnosis–body schema; and (7) social cognition.15 Screening tests should be sensitive to neuropsychological features characteristic of vascular CI.15
Existing brief screening tests include the Mini-Mental State Examination (MMSE)16 and the Montreal Cognitive Assessment (MoCA).17 Of these, the MoCA has been shown to have greater sensitivity than the MMSE in detecting CI in patients with HF.18 Similarly, when compared with the existing Neurological Disorders and Stroke–Canadian Stroke Network (CSN) Vascular Cognitive Impairment Battery for patients who had a stroke, the MoCA displayed good sensitivity and specificity, while the MMSE displayed ceiling effects.10 The MoCA was officially recommended by the National Institute of Neurological Disorders and Stroke (NINDS Common Data Elements) for cognitive screening in all research for stroke.19 However, a notable limitation of the use of MoCA and MMSE in patients who had a stroke is that both are not suitable for use in individuals with aphasia, apraxia and neglect due to a substantial number of items which have verbal or physical requirements. Performance on these tests can also be affected by these conditions. This suggests that a significant proportion of the stroke population may either be scoring poorer due to reasons unrelated to cognition or entirely excluded from cognitive screening.
In addition, the MoCA does not assess processing speed, a domain frequently impaired in patients with HF and who had a stroke,5 10 which was found to be independently predictive of dependency.20 At cut-offs of <25 and <26, the MoCA was found to overlook nine and five cases out of 19 cases of non-amnestic MCI, respectively, most of which were single domain. A lack of measure of processing speed was proposed to explain this relative insensitivity of the MoCA to detect non-amnestic single-domain MCI.10 Given that over 50% of patients were found to have impairment in processing speed 3–6 months after mild ischaemic stroke and transient ischaemic attack,21 a processing speed test would improve the detection of vascular CI. One commonly used processing speed task is the Symbol Digit Modalities Test (SDMT).22 The SDMT has been widely used and validated in diverse clinical populations including stroke, with significant correlation with stroke severity.7 23 24
In this study, we aimed to develop the VasCog Screen test to better detect CI in patients who had a stroke or with HF. To our knowledge, two versions of a paper-and-pen short-MoCA have been created in the context of vascular disease.25 26 However, as cultural norms and cut-offs corresponding to the CI present in the target population are essential,26 we derived and established the cut-off for a short-MoCA specific to patients who had a stroke and with HF in Singapore. This short-MoCA was then combined with the SDMT to form the VasCog Screen test. This study also sought to compare our version of the short-MoCA and VasCog Screen test with Bocti et al’s short-MoCA27 and the NINDS-CSN short-MoCA.25
Methods
Sample characteristics
This study included patients who had a stroke (n=327) and patients with HF (n=100), recruited from two existing stroke and HF studies. The methodology of both studies has been reported previously.24 28
For the stroke study, 400 clinically stable non-aphasic patients (≥21 years) with a recent acute ischaemic stroke or transient ischaemic attack (≤14 days) were recruited during inpatient admission at the National University Health System of Singapore. Demographic characteristics, MoCA and MMSE, stroke severity and disability measures (National Institutes of Health Stroke Scale (NIHSS)29 and the modified Rankin Scale (mRS)30) were collected at the subacute stroke phase. Stroke subtypes included small artery occlusion, large artery atherosclerosis, cardioembolism, undetermined aetiology, stroke of other determined cause and transient ischaemic attack. Patients were excluded if they had major disability (mRS>4), significant aphasia or dysarthria (best language (aphasia) and dysarthria score >1) on the NIHSS that impeded cognitive assessment. Patients were also excluded if they had major and active psychiatric illness and pre-existing dementia, or a score of >3.38 on the Informant Questionnaire on Cognitive Decline in the Elderly.31 The Delirium Rating Scale-Revised 98 was used to exclude patients with acute delirium.32 At the 3–6 months follow-up, 327 patients remained.
For the HF study, 100 patients with HF were recruited from the Singapore Heart Failure Outcomes and Phenotypes study initiated in 2012, as described previously.28 Patients (>18 years) with a primary diagnosis of HF based on the European Society of Cardiology criteria (New York Heart Association Class I, II, III, IV) were recruited from hospital or outpatient clinics in Singapore. Patients were excluded if they had severe valve disease as the primary cause of HF, primary diagnosis of acute coronary syndrome causing transient pulmonary oedema, end-stage renal failure or receiving renal replacement therapy, specific subgroups of HF (including constrictive pericarditis, complex adult congenital heart disease, hypertrophic cardiomyopathy, eosinophilic myocarditis, cardiac amyloid, acute chemotherapy-induced cardiomyopathy), isolated right-sided HF, life-threatening comorbidity with life expectancy of <1 year and concurrent participation in a clinical trial of new pharmacotherapy.
Standard protocol approvals and patient consent
Written informed consent was obtained from all participants or their legally acceptable representatives.
Procedures
Baseline characteristics
Demographic information (age, gender, ethnicity, years of education), cognitive status (MMSE, MoCA, formal neuropsychological assessment), clinical information, vascular risk factors (hypertension, diabetes mellitus, hyperlipidaemia) and neurological status were collected for both studies.
Cognitive measures
All patients from both studies (total n=427) were assessed with the MMSE,16 MoCA17 and a formal neuropsychological battery locally validated for Singaporeans at 3–6 months after stroke.3 33 Translation and back-translation of cognitive tests (MoCA, MMSE and the formal neuropsychological battery) were undertaken by bilingual research psychologists and three equivalent versions (English, Chinese and Malay) of these cognitive tests were established, with the translation described previously.18 34 All cognitive testing was conducted in patients’ language of choice to ensure test fairness.
MoCA (Singapore) was modified from the original MoCA for the Singaporean population and its validation has been described previously.18 21 The formal neuropsychological battery used widely in previous local studies was used to determine CI.20 21 24 It covered the domains of executive function, attention, language, visuoconstruction, visuomotor speed, verbal and visual memory, and was administered by trained research psychologists blinded to the MMSE and MoCA scores. Age and education-adjusted cut-offs of 1.5 SDs below the established norms were used on individual tests. Failing at least half the tests in a domain constituted failure in that domain.
The SDMT is one of three tests in the visuomotor processing speed domain of the formal neuropsychological battery. The remaining two tests in this domain included a digit cancellation test and a maze task. Although processing speed is assessed in the digit cancellation test and maze task, the former is designed for screening neglect, while the latter is primarily considered an executive functioning measure.35 As such, these two tests were not chosen to represent visuomotor processing speed functioning. The SDMT was chosen as it is a purer visuomotor processing speed test, which takes around 2 min to administer.35
Statistical analysis
Demographic frequencies were analysed in IBM Statistical Package for the Social Sciences (SPSS) V.22. For the derivation of a short-MoCA, a Graded Response Model (GRM) was performed using the statistical package Stata V.14.0 for Windows. We used the GRM as this Item Response Theory approach allowed for the inclusion of both dichotomous and polytomous items on the same test. Items in the MoCA are scored dichotomously, with the exception of three items: clock drawing (3 points), serial subtraction (3 points) and delayed recall (5 points). The full sample of patients (n=427) was used to derive the short-MoCA.
Derivation of the VasCog Screen and discriminatory ability
A receiver operating characteristic (ROC) curve analysis with area under the curve (AUC) was used to compare the discriminatory ability of the MMSE, MoCA, short-MoCA, SDMT and VasCog Screen for CI. The AUCs of the MMSE, MoCA, short-MoCA, SDMT and VasCog Screen were compared using established inferential statistical methods.36 Optimal cut-off points for discrimination were established based on the largest Youden index criterion (calculated by sensitivity+specificity–1).
Comparison between versions of the short-MoCA±SDMT
The short-MoCA was compared with Bocti et al’s short-MoCA27 and the NINDS-CSN short-MoCA,25 as well as when they were combined with the SDMT. The AUCs of the short-MoCA, VasCog Screen, Bocti et al’s short-MoCA, Bocti et al’s short-MoCA+SDMT, NINDS-CSN short-MoCA and NINDS-CSN short-MoCA+SDMT were compared using established inferential statistical methods.36 ROCs were compared using DeLong’s test based on the R package ‘pROC’ V.4.3.2.
Results
CI was defined by the local formal neuropsychological test. Similar prevalence rates of CI were found in the stroke (n=187, 57.2%) and HF cohorts (n=44, 44.0%). Of the 187 patients with CI in the stroke cohort, visuomotor speed (n=116, 62.0%) was the most prevalent neuropsychological impairment, followed by visual memory (n=110, 58.8%) and visuoconstruction (n=106, 56.7%). Of the 44 cognitively impaired patients in the HF cohort, neuropsychological impairment in visuomotor speed was most prevalent (n=27, 61.4%), followed by visual memory (n=21, 47.7%) and visuoconstruction (n=21, 47.7%).
As the stroke and HF cohorts had similar profiles of neuropsychological impairment, data from both were pooled for the analysis. The pooled sample comprised mostly of male patients (n=311, 73.0%) who were Chinese (n=297, 69.7%). More than half the patients were found to have CI (n=230, 54.0%). Patients with CI were older (64.3±11.1 years vs 53.9±8.5 years, p<0.001) and had fewer years of education (6.6±3.8 vs 9.8±4.1, p<0.001) compared with those with no CI (NCI). Patients with CI also had significantly lower MMSE (24.4±3.6 vs 27.8±1.8, p<0.001), MoCA (19.9±4.8 vs 25.3±2.5, p<0.001) and SDMT scores (20.5±11.8 vs 39.7±11.3, p<0.001) than those with NCI.
Items which had a discrimination index of ≥1.4 were selected for the short-MoCA. As seen in table 1, a total of 10 items were selected, which added up to a total score of 12. These comprised 3 orientation items (country, year, place; 3 points), 2 visual/executive function items (trail making and cube copy; 2 points), 2 abstraction items (train-bicycle, watch-ruler; 2 points), 1 attention item (serial subtraction; 3 points), 1 language item (verbal fluency; 1 point) and 1 naming item (elephant; 1 point). The short-MoCA was then combined with the SDMT, forming the VasCog Screen.
Item selection for short-MoCA
The optimal cut-off score for the short-MoCA was found to be <9, with a similar classification accuracy to the standard MoCA (table 2). Patients who failed either the short-MoCA or the SDMT were screened as positive for CI. Discriminant indices of the MMSE, MoCA, short-MoCA, SDMT and VasCog Screen (short-MoCA with SDMT) are displayed in table 2. Using the diagnosis classification of the formal neuropsychological battery as a benchmark, the VasCog Screen was found to be superior to the MMSE (AUC: 0.82 vs 0.74, p<0.001), MoCA (AUC: 0.82 vs 0.76, p=0.02) and short-MoCA (AUC: 0.82 vs 0.76, p<0.001) in detecting CI.
Discriminant indices of the MMSE, MoCA, short-MoCA, SDMT and VasCog Screen
Our version of the short-MoCA was compared with two existing versions, Bocti et al’s short-MoCA27 and the NINDS-CSN short-MoCA (table 3).25 Bocti et al’s version comprised five subtests: five-word recall (5 points), verbal fluency (1 point), trail making (1 point), abstraction (2 points) and cube copy (1 point), equating to a total score of 10 points. At the optimal cut-off score of <7, AUC was reported to be 0.87, sensitivity 91% and specificity 83%. However, when Bocti et al’s short-MoCA was examined in the current study, the AUC dropped to 0.71 and sensitivity to 58%, while specificity remained constant at 84%.
Discriminant indices of Bocti et al’s short-MoCA, NINDS-CSN short-MoCA and this study’s short-MoCA, and when combined with SDMT
The NINDS-CSN version of the short-MoCA was proposed by experts in the NINDS-CSN committee,25 comprising orientation (6 points), five-word recall (5 points) and verbal fluency (1 point), equating to a total score of 12. However, validation of the proposed NINDS-CSN short-MoCA was not done; it was Bocti et al who provided one of the first empirical validations for this version.27 At an optimal cut-off score of <9, reported AUC was 0.82, sensitivity 87% and specificity 74%. The current study also sought to validate the NINDS-CSN short-MoCA and found that AUC dropped to 0.67, sensitivity to 48%, while specificity was good at 87%.
The VasCog Screen was found to be superior to Bocti et al’s short-MoCA (AUC: 0.82 vs 0.71, p<0.001) and NINDS-CSN short-MoCA (AUC: 0.82 vs 0.67, p<0.001) in detecting CI.
We also combined the SDMT with Bocti et al and the NINDS-CSN short-MoCA to examine whether the addition of a processing speed measure would consistently increase diagnostic accuracy. As seen in table 3, the addition of the SDMT consistently increased the AUC, sensitivity and classification accuracy, while maintaining relatively good specificity. Statistically significant differences in the discriminatory performance between the VasCog Screen and the two versions of the short-MoCA became non-significant when the SDMT was added.
Discussion
This study aimed to develop a screening test, the VasCog Screen, to improve diagnostic accuracy and sensitivity in screening patients who had a stroke or with HF. The establishment of the VasCog Screen involved the development of a short-MoCA (Singapore), which was subsequently combined with a processing speed test, the SDMT. Patients who failed either the short-MoCA or the SDMT were screened as positive for CI.
The short-MoCA established in this study had reasonable sensitivity and good specificity. The short-MoCA was compared with Bocti et al’s short-MoCA and the NINDS-CSN short-MoCA. When Bocti et al’s short-MoCA was examined in the current study, the AUC and sensitivity dropped, while specificity remained constant. This difference is likely due to discriminant indices of Bocti et al’s short-MoCA having been determined by the MoCA, and not an independent diagnostic neuropsychological battery, a limitation noted in their study. The reported accuracy of the discriminant indices of their short-MoCA might thus have been somewhat compromised. The current study similarly found that AUC and sensitivity of the NINDS-CSN short-MoCA dropped, while specificity remained constant. As aforementioned, the difference in findings is likely due to the lack of an independent diagnostic neuropsychological battery being used. The current study sought to address this limitation by including a formal neuropsychological battery.
Patients with HF and/or who had a stroke tend to exhibit frontosubcortical deficits, which present as impairments in processing speed and executive functioning. This can be explained by their impact to subcortical white matter in the brain, contributing to similar clinical cognitive expressions. The inclusion of a processing speed test was decided on with the knowledge that processing speed is a domain commonly impaired in patients with HF and who had a stroke, but is however not typically included in common cognitive screening measures to date. As expected, the inclusion of a measure of processing speed resulted in increased sensitivity, specificity and diagnostic accuracy.
A formal neuropsychological battery was used as a benchmark to determine diagnostic accuracy, and the MoCA, MMSE and VasCog Screen were compared against it. The VasCog Screen exhibited good sensitivity and specificity for MCI, as well as a classification accuracy superior to the widely used MoCA and MMSE. The VasCog Screen also outperformed the other short-MoCA variants in detecting CI.
At present, there are other existing cognitive screening tools that are suitable for use with individuals with verbal, visual or physical limitations. The MoCA was adapted for individuals with hearing (MoCA-H) or visual impairments (MoCA-V); however, these versions have currently only been validated for use in the dementia population.37 The Oxford Cognitive Screen (OCS) was specifically developed for the stroke population, which is more inclusive towards individuals with aphasia and neglect.38 However, the OCS possesses certain limitations compared with the VasCog Screen, including lengthy administration time and lack of processing speed measure. Another aphasia-friendly measure is the Cognitive Assessment Scale for Stroke Patients, which similarly does not include processing speed measurement.39 Importantly, the VasCog Screen addresses the gap in the current literature on cognitive screening by incorporating a processing speed measure while paring down the most sensitive items of the MoCA, improving the detection of CI in vascular diseases.
Limitations
First, this study excluded patients with significant aphasia or dysarthria. However, this may result in the sample not being fully representative of stroke survivors as these are common stroke-related deficits. The study also excluded patients with major and active psychiatric illnesses and pre-existing dementia as these conditions will cause added cognitive dysfunction not resulting from the stroke. In practice, the VasCog Screen will have to be adapted when used with such patients, for instance, excluding certain items depending on the limitations of each patient. In the future, it is also recommended to develop alternate versions of the VasCog Screen, which will be inclusive towards individuals with verbal, visual or physical limitations.
Second, the VasCog Screen was not administered as an independent test, hence the total administration time has not yet been determined, although it is estimated to be faster than current screeners such as the MMSE and MoCA. Further studies need to be conducted to establish the official administration time.
Third, it would have been preferred to divide the sample into a train and test sample given the large sample and to run the GRM on the train to see how well it fits on the test sample. However, our study comprised two datasets (n=327 and n=100, respectively), hence it would be limiting to split the datasets further. Future studies are needed to further validate the VasCog Screen.
Conclusion
To conclude, this study aimed to develop the VasCog Screen, a screening test sensitive to certain characteristic cognitive deficits in patients with HF and who had a stroke. The development of the VasCog Screen was guided by a consideration of the diverse linguistic and cultural needs of the Singapore population. Comprising half the original MoCA items, it is considered to be easily administered in the community and primary care setting.
It was found to have good sensitivity, specificity and classification accuracy, outperforming the MoCA, MMSE and other versions of the short-MoCA. It is suitable for routine cognitive screening in busy cardiovascular or stroke clinics, enabling the early detection of CI to facilitate appropriate interventions. Future research can explore improving the inclusivity of the VasCog Screen and further validating its use.
Data availability statement
No data are available.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by the local ethics committee and conducted in conformity with the Declaration of Helsinki. Stroke and heart failure samples were derived from two separate studies which have been completed and published. The ethics application number may not be able to be traced. Participants gave informed consent to participate in the study before taking part.
Acknowledgments
The authors acknowledge the collaborative support of the COAST study team and ATTRaCT study team.
References
Footnotes
Contributors The study was conceptualised and designed by YD, who has secured the funding and collected data and also is responsible for the overall content as the guarantor. NYCC and YD drafted the manuscript. MYLT contributed to the editing and revision of the manuscript. JX conducted the statistical analysis and verified the results. All authors reviewed and approved the submitted version of the manuscript.
Funding YD is a recipient of the Singapore National Medical Research Council (NMRC) Transition Award (TA) (NMRC/TA/0060/2017).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.