Background Case-finding is a recommended approach for dementia early detection in the community.
Aims To investigate the discriminant validity and cost-effectiveness of a stepwise dementia case-finding approach in a Singaporean older adult community.
Methods The two-phase study was conducted in the community from 2009 to 2015 in Singapore. A total of 3780 participants (age ≥60 years) completed phase I (a brief cognitive screening); 918 completed phase II and were included in the final analysis. In phase I, all participants were administered the Abbreviated Mental Test (AMT) and the Progressive Forgetfulness Question (PFQ). Those who screened positive on either test were invited to phase II, whereby the Mini-Mental State Examination (MMSE), Montreal Cognitive Assessment (MoCA) and a formal neuropsychological battery were administered, followed by the research diagnosis of no cognitive impairment, cognitive impairment no dementia (CIND)-Mild (≤2 impaired cognitive domains), CIND-Moderate (>2 impaired domains) or dementia. Receiver operating characteristic curve analyses were conducted for the different cognitive instruments. All discriminant indices were calculated, including sensitivity, specificity, positive and negative predictive values (NPV) and accuracy. Cost-effectiveness analysis was conducted by estimating the amount of screening time needed and the number of older adults requiring re-evaluation in two case-finding scenarios, ie, with or without preselection by the PFQ.
Results The stepwise case-finding approach (preselection by the PFQ, then MMSE or MoCA or AMT) showed an excellent NPV (>99%) and accuracy (>86%) for excluding dementia-free cases. Without preselection by the PFQ, screening time for the three cognitive tools were 317.5, 317.5 and 254 hours, with 159, 302 and 175 screen-positive older adults involved in further evaluation. By adopting the stepwise case-finding approach, total screening time were 156.5, 156.5 and 126.2 hours, which decreased by 50.7%, 50.7% and 50.3% as compared with those without preselection. Furthermore, after preselection, only 98, 167 and 145 screen-positive older adults required further evaluation, corresponding to a reduction of 38.4%, 44.7% and 17.1% in the numbers compared with those without preselection.
Conclusions A stepwise approach for dementia case-finding should be implemented in the community to minimise the time and resources needed for large-scale early detection of dementia.
- Discriminant Analysis
- Community Health Services
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
WHAT IS ALREADY KNOWN ON THIS TOPIC
The practicality of present tools for measuring cognitive impairment and dementia is limited for community screening due to lengthy administration time and requiring professional expertise.
WHAT THIS STUDY ADDS
By using the Progressive Forgetfulness Question as a preselection assessment in conjunction with other cognitive tools for dementia detection, the stepwise case-finding approach not only showed better discriminant validity between participants with and without dementia but also demonstrated a substantial reduction in time cost and the number of older adults requiring further evaluation for the case-finding approach.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
The stepwise case-finding approach offers a more effective and cost-efficient strategy to accurately rule out most individuals at low risk of developing dementia in community-dwelling older adults.
Nearly 50 million people worldwide live with dementia, and this is expected to triple by 2050. It is estimated that the global annual expenditure on dementia prevention and treatment is around US$1 trillion.1 Without question, dementia seriously challenges healthcare systems worldwide, especially in low-income and middle-income countries.
Case-finding is a recommended approach in the early detection of dementia. It identifies individuals at higher risk, thereby reducing the overall pool size needing more detailed assessment and improving detection accuracy.2 Thus, this approach is especially suitable for community settings with high volumes of participants but scarce screening resources. In clinical settings, many cognitive assessment tools, such as the Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA), are widely used because of their high accuracy.3 However, It’s challenging to apply them in community settings because of lengthy evaluation times and high labour costs. To address these concerns, briefer assessments have been introduced that can be performed by the participants or their caregivers.
Subjective cognitive decline (SCD) is a crucial predictor for neurocognitive disorders, including dementia, and refers to self-reported persistent cognitive decline in the absence of objective cognitive impairment.4 Previous studies have confirmed that SCD can identify individuals at high risk of cognitive impairment.5 6 The use of a single question to assess SCD has been described in some studies.7 The Dementia Commissioning for Quality and Innovation document published by the UK Government Department of Health in 2012 recommended that people aged 75 years and above be screened for dementia by asking them a single question, ‘Have you become more forgetful in the past 12 months to the extent that it has significantly affected your life?’ If older adults verbally answered yes, a dementia risk assessment was initiated. By asking a single question, the gap was narrowed between observed and expected numbers of dementia diagnoses.8 The Progressive Forgetfulness Question (PFQ) is a useful preliminary assessment of SCD via a simple question that asks older adults or their caregivers about progressively worsening forgetfulness. It has been used for community-based dementia screening due to its high feasibility.9 The SPEED (The Stroke, Parkinson’s disease, Epilepsy, and Dementia in Singapore) study, conducted between 2001 and 2003, using the PFQ and Abbreviated Mental Test (AMT) to screen for dementia in a Chinese community-dwelling population over the age of 50 years, reported that the PFQ was simple but effective in screening for dementia in primary care settings, ruling out individuals at lower risk of dementia.9 10 Because of their high specificity in excluding healthy controls from large populations, single-question cognitive screening tools have been beneficial in ruling out low-risk older adults rather than identifying those with potentially high risk.7 As most older adults in the community are cognitively and functionally intact, conducting large-scale dementia screening can needlessly consume much time and resources. However, using a brief tool as a first step in community screening would be a cost-efficient strategy to rule out dementia among low-risk individuals accurately.11 Nonetheless, the efficacy and cost–benefit of such a stepwise screening approach for practical implementation in a large community of older adults remains to be determined.
Thus, the present study aimed to (1) examine whether the stepwise use of the PFQ as a preselection assessment, followed by the employment of other cognitive tools, can improve dementia discriminant utility; (2) evaluate whether the overall screening time and the number of older adults requiring further assessment can be reduced when the stepwise screening approach is adopted.
We hypothesised that applying objective cognitive tests to people who screened positive for the PFQ could quickly exclude older adults at lower risk of cognitive impairment and dementia. Second, the overall screening time and the number of older adults requiring further assessment would be reduced when the stepwise case-finding approach is adopted.
The Singapore Epidemiology of Eye Diseases (SEED) Study used an age-stratified random sampling strategy to select residents between the ages of 40 and 80+ years from 15 residential districts in the southwestern part of Singapore, an area fairly representative of the country’s population in age, housing and socioeconomic status. The cohort profile of this study has been published previously.12 The SEED Study comprised the Singapore Chinese Eye Study from 2009 to 2011, the Singapore Malays Eye Study from 2010 to 2013, and the Singapore Indian Eye Study from 2013 to 2015. Details of the above studies have been described previously.13–15 Among all Singaporean adult participants, a convenient sample of senior residents aged 60 years and older were included in the present Epidemiology of Dementia in Singapore (EDIS) Study. The study excluded older adults who met any of the following criteria: (1) suffering from a malignant disease, such as cancer, tumour, etc; (2) diagnosed with major depressive disorder or other psychiatric illnesses; (3) with severe visual, hearing or communication impairments. All eligible older adults and their caregivers were sent an invitation via telephone, email and/or home visit to go to the Singapore Eye Research Institute for the assessment. A person was termed ‘uncontactable’ after six unsuccessful telephone calls and/or home visits.
Phase I: epidemiological survey and cognitive screening
The EDIS Study was a two-phase study as the SPEED Study.10 In phase I, a questionnaire on demographic information and relevant risk factors, along with a cognitive screening test comprised of the AMT16 and the PFQ, were administered by trained investigators. Previously validated in Singapore, AMT’s adjusted optimal cut-off for participants with 0–6 years of education was 6/7 (sensitivity 89.6% and specificity 92.6%), and for those with more education was 8/9 (sensitivity 82.1% and specificity 92.9%).17 The PFQ refers to a single question of subjective cognitive complaints that asks the primary caregiver who had at least 10 hours of interaction with the participant weekly about the older adult’s experience of progressive forgetfulness18; an affirmative response was considered positive. Older adults who screened positive for either of the above two tests were invited to enter phase II of the study, at which participants underwent extensive clinical and neuropsychological evaluations, as described in more details previously.15
Phase II: cognitive assessments and dementia diagnosis
The MoCA, MMSE and a formal neuropsychological battery were performed in phase II. All the above tests have been locally validated for Singaporean older adults.19 20 The formal neuropsychological battery was administered by trained research psychologists. The domains of this battery included the following15:
Executive function: the Frontal Assessment Battery and Maze Task.
Attention: Digit Span, Visual Memory Span and Auditory Detection.
Language: the Boston Naming Test and Verbal Fluency.
Visuomotor speed: the Symbol Digit Modality Test and Digit Cancellation.
Visuoconstruction: the Weschler Memory Scale-Revised Visual Reproduction Copy task, Clock Drawing and the Weschler Adult Intelligence Scale-Revised subtest of Block Design.
Verbal memory: Word List Recall and Story Recall.
Visual memory: Picture Recall and the Weschler Memory Scale-Revised Visual Reproduction.
Cognitive impairment and dementia were diagnosed by consensus at formal research team meetings, using the results from the above-listed tests. Individual test scores below the education-adjusted cut-offs of 1.5 SDs were categorised as test failures.15 Impairment in a cognitive domain was defined as failure in at least half of the tests in that domain. Cognitive impairment with no dementia (CIND) was classified into CIND-Mild (when ≤2 cognitive domains were impaired) and CIND-Moderate (when >2 domains were impaired). Dementia was diagnosed according to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition criteria.21
One-way analysis of variance and Χ2 tests were used to compare differences in sample characteristics. Receiver operating characteristic curve analyses were conducted to establish the area under the curve (AUC) for different cognitive instruments (MoCA, MMSE, AMT) between PFQ=yes and PFQ=no groups. The discriminant indices, including sensitivity, specificity, positive and negative predictive values (NPV), and overall accuracy, were calculated using the optimal cut-off points for each screening tool.22 Meanwhile, the verification bias of the PFQ for diagnostic groups was adjusted using the Bayesian correction method for differential verification.23
We investigated the discriminant indices of the PFQ as a stepwise method, followed by other commonly used cognitive instruments, including the AMT, MoCA and MMSE, for dementia detection. Meanwhile, we further examined whether the AUCs of MoCA, MMSE and AMT for detecting dementia differed between the two PFQ groups (PFQ=yes and PFQ=no).
In addition, we calculated the overall screening time and the number of older adults requiring further comprehensive evaluation for different screening approaches in the following two scenarios. In the first scenario, only the MoCA, MMSE or AMT was used to identify dementia high-risk individuals. In the second scenario, those who screened positive on the PFQ were subsequently administered the MoCA, MMSE or AMT to identify participants who were at high risk of developing dementia. The administration time of the MoCA, MMSE and AMT was 10 min, 10 min and 8 min, respectively.17 24 25 We assumed the time required for the PFQ was 10 s.
Number with dementia: true positives need to be identified.
PR%: the prevalence of dementia in this study.
Timea: the time cost of step 1.
Timeb: the time cost of step 2.
All analyses were done on IBM SPSS V.26.0 (IBM Corp Released 2019. IBM SPSS Statistics for Windows), and a p value <0.05 was considered statistically significant.
Figure 1 shows the study recruitment flow chart. A total of 3800 community residents (age ≥60 years) from the SEED Study were eligible for inclusion, 20 of whom refused to participate. Finally, a total of 3780 older adults were included in the EDIS Study and completed the brief cognitive screening of phase I; among these, 918 completed phase II (887 PFQ=yes, 31 PFQ=no). The sample characteristics of phase II older adults selected by the PFQ are shown in table 1. In addition, we compared the characteristics of the two screened-positive groups that entered phase II versus those who did not enter phase II (see online supplemental table 1). Older adults who screened positive and entered phase II were younger (mean age 70.3 vs 72.1 years), less likely to be female (52.5% vs 59.0%) and less educated (60.5% vs 71.6%) than those who screened positive but did not enter phase II.
Discriminant indices of the stepwise case-finding approach for detecting dementia
Among the 918 older adults in phase II, 45 (4.9%) were finally diagnosed with dementia. We explored the discriminant indices of the PFQ with other cognitive tools for detecting dementia in the PFQ=yes and PFQ=no groups. Results showed good overall accuracy for the MMSE (93.3%), MoCA (86.2%) and AMT (88.4%), respectively; all three tools achieved an optimal NPV exceeding 99% in those older adults with a positive PFQ (table 2 and online supplemental table 2). Also, we further compared the AUCs of the above-mentioned cognitive tools for dementia detection between the PFQ=yes and PFQ=no groups (figure 2). In the PFQ=yes group, MoCA, MMSE and AMT had AUCs of 0.95 (95% CI: 0.62 to 0.98), 0.94 (95% CI: 0.90 to 0.98) and 0.93 (95% CI:0.87 to 0.97), respectively, for discriminating between participants with and without dementia. However, lower AUCs were found in the PFQ=no group as compared with the PFQ=yes group on the MoCA (0.87, 95% CI: 0.71 to 0.99), MMSE (0.83, 95% CI: 0.62 to 0.99) and AMT (0.78, 95% CI: 0.46 to 0.98). A more detailed table is presented in online supplemental table 2.
Cost-effective analysis of the stepwise dementia case-finding approach
We assumed two case-finding scenarios in the cost-effectiveness analysis for the case-finding approach, ie, without and with preselection by the PFQ. According to the present study, the prevalence of dementia was 4.9% (45 of 918) in this study sample.
The adjusted sensitivity and specificity with PFQ for dementia detection using the Bayesian correction method were 48.2% and 52.4%, respectively. It is estimated that a total of 1905 (refer to the Methods section for calculation formula) older adults would have been recruited for the PFQ preselected test to identify 45 individuals with dementia. In the first scenario (MMSE or MoCA or AMT), the 1905 individuals would have had to perform one of the following three tests to identify the 45 true positives: the MoCA (cut-off: 13/14, sensitivity: 91.1%, specificity: 85.8%), MMSE (cut-off: 17/18, sensitivity: 80.0%, specificity: 93.7%) or AMT (cut-off: 6/7, sensitivity: 75.6%, specificity: 92.8%). Without the preselection of the PFQ, a total of 159, 302 and 175 older adults would have failed the MMSE, MoCA or AMT, and hence would have been required to undergo further assessment for confirmation of diagnosis. The screening time for the three tools would have been 317.5 hours, 317.5 hours and 254 hours, respectively (figure 3A).
In the second scenario (preselection by the PFQ, then MMSE or MoCA or AMT), screening was done to the same 1905 individuals by the PFQ first, among whom 45 true positives and 862 false positives would have failed in the PFQ, leading to a total of 907 older adults entering the next step to perform MMSE, MoCA or AMT. Subsequently, the sensitivity and specificity of the MoCA in the PFQ=yes group were 92.7% and 85.9%, respectively. Thus, 167 older adults (45 true positives, 122 false positives) would have entered the final diagnostic evaluation. In this case, the total screening time for this stepwise approach with the PFQ (1905×10 s) followed by the MoCA (907×10 min) would be 156.5 hours. Following the same calculation logic, preselecting by the PFQ followed by the MMSE or AMT would have required 98 and 145 older adults for further testing, and the overall screening time would have been 156.5 hours and 126.2 hours, respectively (figure 3B).
Significantly, in the stepwise approach, the first step of using the PFQ and then adding the MoCA, MMSE or AMT resulted in a 50.7%, 50.7% and 50.3% reduction in the total screening time cost, respectively. The number of people requiring further evaluation would have decreased by 135 (44.7%), 61 (38.4%) and 30 (17.1%), respectively.
In summary, we provide a stepwise dementia case-finding strategies for community-dwelling older adults: apply a single-question assessment—the PFQ—as the first step to stratify case-finding of dementia in a large population and then perform other objective cognitive tests. Only individuals who test positive in both steps could be included in a final comprehensive cognitive assessment (online supplemental figure 1).
In this study, we found that the stepwise combination of using the PFQ with other cognitive tools can remarkably rule out older adults in the community who are at a lower risk of dementia. Hence, implementing dementia screening using this stepwise case-finding approach in large population settings can effectively exclude individuals at low risk of dementia and reduce the time and resources required for further assessment.
Previous studies have shown that single-question SCD assessment tools were not suitable for use alone in community settings to identify individuals with early dementia, including the 10th item on the Geriatric Depression Scale and the 8th item on the Ascertain Dementia 8, as they yielded limited discriminant validity for dementia detection.26 Evidence has shown that participants with cognitive decline might not be able to describe their own mental status accurately.27 Thus, one-question SCD tools might lead to numerous false results, illustrating that using the PFQ alone could not reliably detect cognitive impairment or dementia in community settings. Nevertheless, the forgetfulness emphasised by the PFQ is progressive, which is one of the key components in assessing memory loss in the Clinical Dementia Rating (CDR) instrument, especially in distinguishing normal cognition (CDR=0) from mild cognitive impairment (CDR=0.5). Thus, although the discriminant validity of the PFQ is limited when used alone, using it as a first step in a large-scale dementia screening setting can help identify older adults with possible advancing cognitive impairment so that they can receive an additional assessment as quickly as possible. Meanwhile, adding such single-question assessments before other objective cognitive tests can ease participants’ nervousness and foster good relationships between the investigator and the participants.11
Noteworthy, we found that using the PFQ stepwise with other cognitive tools (MoCA, MMSE or AMT) offers a promising approach for dementia case-finding, especially for ruling out those at low risk for dementia. Our results are consistent with the previous SPEED study, which demonstrated that without the PFQ, there were epidemiological reasons not to proceed with further AMT administration.9 10 Furthermore, we extended the findings of a previous study to the MoCA and MMSE. Among older adults who reported PFQ=yes in our study, the NPV of all the tools slightly improved and achieved excellent AUCs (MoCA=0.95, MMSE=0.94 and AMT=0.93) for detecting dementia. The overall accuracy also yielded favourable results (MoCA: 86.2%, MMSE: 93.3% and AMT: 88.4%). The optimal cut-offs of the MoCA and the MMSE in the present study were lower than in general population studies,28 29 mainly because the sample we included for the final analysis was a population with possibly high cognitive risk who performed poorly at phase I. However, these cut-off values were broadly consistent with previous studies in the Asian population at high risk of dementia.19 30 31 Inevitably, in large-scale dementia screening, older adults with cognitive impairment may report negative PFQ scores, leading to false negative results. However, in the present study, most older individuals who reported ‘no’ on the PFQ were dementia-free, with only 9.7% (3 in 31) of false negatives.
We have also shown that the stepwise case-finding approach can be time cost-effective when implementing early dementia screening within communities. Comprehensive neuropsychological cognitive testing, which is extensive and domain specific with a lengthy administration time of approximately 1 hour, requires a specialist to administer and cannot readily be given to every older community adult for formal cognitive impairment diagnosis.18 A stepwise case-finding approach can significantly reduce the number of participants requiring further assessment for diagnosis confirmation and the associated time costs. We calculated the overall screening time and the number of individuals requiring further evaluation by assuming two scenarios. The total screening time for the preselection by the PFQ and then performing the MMSE or MoCA or AMT would have been 156.5 hours, 156.5 hours and 126.2 hours, respectively, and would have required 98, 167 and 145 individuals for further testing. Without the preselection by the PFQ, the screening time for the three cognitive tools would have been 317.5 hours, 317.5 hours and 254 hours, respectively, with a total number of 159, 302 and 175 individuals who would have been entered for further evaluation. Thus, our study showed the overall screening time would have been decreased by 50.7%, 50.7% and 50.3%, respectively, when preselecting the positive PFQ participants to undergo the MMSE, MoCA or AMT. The number of individuals requiring further evaluation would have decreased by 61, 135 and 30 individuals, demonstrating the effectiveness of the stepwise case-finding approach in the community to minimise human resources and time costs. Therefore, considering the accuracy and time cost-savings, we recommend using a stepwise method that first asks the PFQ and then conducts other cognitive tests to rule out as many individuals at low risk of dementia as possible in the community setting.
We acknowledge several limitations of our study. First, as this study was conducted in a community setting in Singapore, the sample is specific, and the external validity of the stepwise case-finding approach in other settings or populations remains to be confirmed. Second, the 43% dropout rate in phase II may have impacted our results; participants who continued to phase II were younger, more educated and differed significantly in gender, ethnicity and hypertension history from those who refused further participation. Third, the optimal cut-off values for the MoCA and MMSE used in this study were lower than those for the general population, mainly because the vast majority of older adults included in our analysis were those with positive initial screening results, which may affect the extrapolation of the study results. Fourth, this study has the inherent property of verification bias, a measurement bias often associated with screening tests, which can conceal the diagnostic ability of the designated screening tool.23 Though we invited participants who screened negative in phase I to continue to phase II, only a few participants consented to further evaluation. Thus, the diagnostic performance of the stepwise case-finding approach may be biased by such attrition, the sensitivity and NPV may be overestimated, and the specificity may be underestimated, even after correction for verification bias. Moreover, it should be noted that the PFQ=no group in the current study may not be representative of the general Singapore population due to its small sample size. In addition, a previous Singapore-based study9 reported a much lower dementia rate of 1.6% (1 out of 61) in the PFQ=no group. As these epidemiological studies were conducted at different periods of time, with differential sample demographics, further comparisons should be made with caution. Future studies could consider including more participants who screened negative on the PFQ for a gold-standard evaluation to validate further the effectiveness of excluding low-risk populations in community settings. In addition, further studies could verify the feasibility and discriminant utility of other brief screening tools equivalent to the PFQ with the same stepwise strategies.
This study demonstrated a stepwise case-finding approach for dementia detection in large-scale screening. Using a single-question assessment, such as the PFQ, the first step excludes individuals at low risk of dementia and identifies those who may potentially be at high risk; the second step then involves performing another objective cognitive test on the high-risk individuals while minimising time costs.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
Patient consent for publication
This study involves human participants. The SEED Study was approved by the National Healthcare Group Domain Specific Review Board and was conducted in accordance with the Declaration of Helsinki (approval numbers: R1107/9/2014 and R498/47/2006). Written informed consent was obtained from all participants or their legally acceptable representatives by their preferred language.
Ting Pang is a PhD candidate in the School of Public Health at Zhejiang University in China. She obtained a bachelor’s degree in Management from the Guangxi University of Chinese Medicine and a master’s degree in Medicine from Guangxi Medical University in China. Her main research interests include screening for cognitive impairment/dementia in older adults and related implementation strategies.
Binte Xia is currently studying for a Medical Bachelor in Public Health at Zhejiang University in China. His main research interests include screening, intervention and caregiving for older adults with dementia and community mental health.
TP and BX are joint first authors.
TP and BX contributed equally.
Correction notice This article has been corrected since it was first published. Author name Wong Tien Yin is changed to Tien Yin Wong.
Contributors XX was responsible for the manuscript and controlled the decision to publish. XX designed the study, developed the protocol and obtained the ethics of this study. TP, BX and XZ performed data analysis and wrote the manuscript. YZ, CNK, SH, CC, NV, C-YC, CY and XX revised the manuscript. All authors read and approved the final manuscript.
Funding This study was funded by National Natural Science Foundation of China(72274170), Interdisciplinary Research Project of the Zhejiang University (519600*17222022201), National Medical Research Council (R-184-006-184-511) and Dean’s Fund Research of the Zhejiang University (188021-171257702/004/010).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.