A brief case-finding questionnaire for common mental disorders: the CMDQ

Abstract

Objectives. The aim of the study was to validate a new case-finding instrument for common mental disorders (CMDQ).

Methods. A cross-sectional, stratified, two-phase study was carried out in 28 general practices in Aarhus County, Denmark. 1785 consecutive patients, 18–65 years old, consulting 38 GPs with a new health problem participated. Patients were screened before consultation using a one-page screening questionnaire including subscales for somatisation (SCL-SOM and Whiteley-7), anxiety (SCL-ANX4), depression (SCL-DEP6) and alcohol abuse (CAGE). A stratified subsample of 701 patients was interviewed using the Schedules for Clinical Assessment in Neuropsychiatry (SCAN) interview. We tested the external validity of the scales using the SCAN interview as gold standard. All data were analysed using appropriate weighted procedures to control for the two-phase sampling design and non-response bias.

Results. Estimates of sensitivity and specificity for relevant ICD-10 diagnoses at theoretical optimal cut-off points on subscales: Depressive disorder: 78/86 (SCL-DEP6); Alcohol abuse or dependence: 78/97 (CAGE); Severe anxiety disorder: 77/85 (SCL-ANX4); Somatisation disorder: 83/56 (SCL-SOM); and 75/52 (Whiteley-7); any mental disorder: 72/72 (SCL-8). At the theoretical optimal cut-off points the CMDQ demonstrated higher diagnostic accuracy than GPs on any diagnosis evaluated.

Conclusion. The study results suggest that the CMDQ has excellent external validity for use as a diagnostic aid in primary care settings.

Christensen KS, Fink P, Toft T, Frostholm L, Ørnbøl E and Olesen F. A brief case-finding questionnaire for common mental disorders: the CMDQ. Family Practice 2005; 22: 448–457.

Introduction

Somatoform disorders, anxiety disorders, depressions and alcohol abuse disorders are common, costly, frequently unrecognised and not optimally treated in general practice.^1–3 In their capacity as family physicians and gatekeepers to the secondary health care system GPs play a central role in the recognition and treatment of mental disorders.

Patients often present with a mixture of physical and psychological complaints that are difficult to categorise.⁴ However, once a formal diagnosis has been made, appropriate treatment becomes more likely.⁵ Attempts to improve GPs' recognition of mental disorders may therefore be essential to promote mental health interventions and to prevent the somatic fixation⁶ of mental health problems.

Previous efforts to improve GPs' recognition of mental disorders have mainly included screening measures of general distress⁷ and screens for single specific mental disorders.⁸ However, general distress is not readily translated into specific, treatable disorders, and available screens for a single mental disorder may ignore the patients suffering from another or more than one disorder.⁹ The feasibility and validity of composite instruments that screen for a wider range of mental disorders in everyday practice are sparsely investigated.^9–12 The aim of this study was to introduce and validate a new screening instrument for common mental disorders in general practice. The instrument is called the CMDQ (Common Mental Disorder Questionnaire). The CMDQ is a brief and easy-to-use screening instrument designed for case-finding in the busy general practice setting.

Methods

The screening instrument

The CMDQ has three components: a one-page questionnaire, a handy scoring template, and a one-page guide for registration of the sum scores (Figs 1–3). The questionnaire consists of 36 questions covering symptoms and signs commonly associated with somatoform disorders, anxiety, depression and alcohol abuse, and a final question about the patient's overall health. Each question is rated on a five-point Likert scale of distress ranging from ‘not at all’ to ‘extremely’ (0 to 4), except for alcohol-questions that are rated as simple no/yes (0/1) answers. The time period covered by the CMDQ is the past 4 weeks for Likert scales, and the last year for alcohol-questions. Answers on single items are easily dichotomised and added to a sum score with corresponding positive predictive values (PPVs) on separate subscales. The completion of the CMDQ by the patient takes 2–5 minutes, the rating by the GP less than one minute.

The CMDQ is constructed by combining six different subscales: from the SCL-90-R, the symptom checklist (SCL-SOM)^13,14 the SCL-8^15,16 for emotional psychiatric disorders and two closely related subscales, the SCL-ANX4 for anxiety (unpublished data) and the SCL-DEP6 for depression (unpublished data). Illness worry and conviction is assessed using the Whiteley-7 scale.¹⁷ The SCL-8, SCL-ANX4, SCL-DEP6, and Whiteley-7 are all developed by reducing larger scales using modern item response theory.^18,19 Alcohol abuse and dependence is assessed using a well-known screening instrument, CAGE.^20,21

Study population

Aarhus County has a population of about 600 000 people living in rural and urban areas. The county is served by 431 GPs working in 271 practices. All the GPs were invited to participate in a randomised controlled trial on recognition and treatment of functional illness in primary care (the FIP study). Thirty-eight GPs (8.8%) working in 28 practices accepted. Included in the study were consecutive patients aged 18–65 years presenting with a new health problem during a 3-week period (study period 3 March to 1 May 2000). Excluded were patients of non-Scandinavian descent, patients unable to speak or read Danish, and patients who were too ill or demented to read and fill in questionnaires. Only patients enrolled in the National Health Care Programme, which covers 98% of the Danish population, were included. Among 2424 patients assessed for eligibility, 227 met the exclusion criteria, 274 patients declined, and 138 could not participate for other reasons (time pressure in clinic, patients not bringing their reading glasses, etc.). Declining patients had a mean age of 42.2 years compared with 38.8 years among included patients (P < 0.001, t-test), whereas we found no statistically significant gender differences. No significant differences were found in age or gender between included patients and patients not participating for other reasons. Each patient was included only once; thus 1785 patients (81.2%) joined the study (Fig. 4). All the patients included completed the CMDQ in the waiting room before the consultation.

Selection of patients for diagnostic psychiatric interview

After the consultation, a stratified second phase sample including every ninth eligible patient and all patients with high SQ scores was selected for a semi-structured standardised psychiatric interview, the Schedules for Clinical Assessment in Neuropsychiatry (SCAN).²² Each item was dichotomised between ‘a little’ and ‘moderately’ and added to a sum score for each subscale. High-scorers were defined by dichotomised sum scores on SCL-SOM > 3, Whiteley-7 > 1, SCL-8 > 1 or CAGE > 1. Among the 894 patients selected for the SCAN interview, 701 (78.4%) accepted. Patients with low scores, younger patients and men refused the interview more often than other groups, but the differences were small. Inter-rater agreement on psychiatric diagnosis was high (κ = 0.86) (for details, see Toft et al.²³).

The psychiatric research interview

ICD-10 diagnoses generated from the SCAN interview were used as gold standard. Six psychiatrically trained physicians, certified at the Copenhagen WHO-SCAN Training Centre, conducted the interviews. The interviews were conducted as soon as possible after the index contact, either in the clinic, in the research department or in the patient's home. Interviewers were blinded to screening results (for details see Toft et al.²³).

Statistics and data analysis

Unanswered questions on the SQ were automatically set to ‘not at all’ (i.e. zero). The relative missing response rates and 95% confidence intervals (CIs) were calculated by comparing missing response rates for patients having a SCAN-diagnosed mental disorder to missing response rates for patients not having a mental health diagnosis (according to SCAN or GP). Estimation of sensitivity (SE), specificity (SP), positive and negative predictive values (PPV and NPV) and their 95% CIs, was obtained by weighted logistic regression analysis with the observed sampling fractions of second-phase patients as sample-weights.²⁴ Performance characteristics for the Likert-scales (SCL-scales and Whiteley-7) were assessed after dichotomisation of each item response between 0 and 1, and 1 and 2. Further analyses were made using continuous (non-dichotomised) scores. We conducted receiver operating curve (ROC) analyses to completely determine the diagnostic performance of the scales across the range of cut-off points and to allow evaluation of the diagnostic performance of the different scales by examining their areas under the curve (AUCs).²⁵ 95% CI for AUCs were calculated using a jackknife estimated standard error.²⁶ The theoretical optimal cut-off was assessed estimating the maximum area under the QROC-curve.²⁷ Diagnostic properties of subscales were compared to GPs unaided diagnostic performance as previously assessed.²⁸ Data were processed by means of STATA 8.0 and SPSS 10.0 for Windows.

Results

Missing responses on subscales

80.8% of the 1785 questionnaires were fully completed. 9.6% of questionnaires had 1 missing, 2.4% had 2 missing and 7.2% had 3 or more missing responses. Patients with mental disorders had more missing responses on corresponding subscales than patients without any mental disorder: The relative missing response-rates were: 1.8 (95% CI 1.4–2.4) for SCL-SOM; 2.8 (95% CI 1.9–4.1) for Whiteley-7; 2.0 (95% CI 1.1–3.5) for SCL-8; 1.3 (95% CI 0.5–3.9) for SCL-ANX4; 3.8 (95% CI 2.0–7.7) for SCL-DEP6 (especially question no. 29, bothered by thoughts of ending your life); and 6.4 (95% CI 2.4–17.9) for CAGE (especially questions no. 34, been annoyed by people criticising your drinking; and 35, felt bad or guilty about your drinking).

Outcomes and estimation

The diagnostic properties of each subscale, using 0/1 dichotomisation of item responses, are presented in Tables 1 and 2. We used AUC for evaluating the overall psychometric performance of each subscale across the whole range of scores. An AUC = 1 matches the optimal test, whereas AUC = 0.5 equals the random test result. Psychometric characteristics were excellent for specific ICD-10 disorders on the following subscales. AUC for CAGE: 0.94 (95% CI 0.90–0.98); AUC for SCL-DEP6: 0.88 (95% CI 0.84–0.91); and AUC for SCL-ANX4: 0.87 (95% CI 0.82–0.92). The SCL-8 performed well with an AUC reaching 0.78 (95% CI 0.71–0.85) for any emotional psychiatric disorder. For SCL-SOM and Whiteley-7 performances were modest to good with AUCs ranging from 0.67 (95% CI 0.59–0.74) to 0.78 (95% CI 0.71–0.85) with highest values for Somatisation disorder.

For SCL-SOM and Whiteley-7 the optimal theoretical cut-offs were 4/2 for males, and 5/3 for females. Corresponding cut-offs were 4/5 for patients aged ≤40 and 2/4 for patients aged >40. Overall Whiteley-7 performed best among patients ≤40 years.

The theoretical optimal cut-offs for SCL-ANX4, SCL-8, SCL-DEP6 and CAGE were unaffected by gender and age. SCL-8 and SCL-DEP6 demonstrated best performances among men ≤40 years.

Performance characteristics on all continuous SCL-subscales were slightly better than on scales dichotomised between 0 and 1, whereas Whiteley-7 was unaffected by 0/1-dichotomisation of items.

Further details on performance characteristics, including 95% CIs on SE, SP, PPV and NPV, can be obtained from the electronic version of this article or by application to the corresponding author.

In Table 3 we compared the diagnostic performances of the CMDQ with the GPs' unaided assessments. At the theoretical optimal cut-offs the CMDQ demonstrated higher diagnostic accuracy than the GPs on any diagnosis evaluated.

Discussion

Despite the brevity of the screening questionnaire, results from this study suggest that the CMDQ has excellent external validity for use in primary care settings. An additional novel finding is that missing responses on separate items need careful evaluation. Especially for SCL-DEP6, CAGE, Whiteley-7 and SCL-SOM one or more missing item-responses may indicate a markedly increased risk of having the disorder inquired about. Furthermore, our findings suggest that using the CMDQ may improve GPs' diagnostic accuracy.

We used a state-of-the-art independent standardised psychiatric research interview²⁹ performed by trained psychiatric physicians as external criteria or gold standard. We included all types of psychiatric diagnoses in the analysis, and relatively few patients refused to be interviewed. The study is free from any obvious selection bias due to sociodemographic factors as primary health care in Denmark is free of charge for patients. We made a conservative evaluation of psychometric performances as missing answers to items were scored zero.

The Whiteley-7 and SCL-8 subscales are both validated for use in general practice.^17,30 Forthcoming results on the closely related anxiety (SCL-ANX4) and depression (SCL-DEP6) subscales will demonstrate their high internal validity (personal communication). The SCL-SOM serves as a simple symptom checklist and has therefore not been formally internally validated.¹⁴ Previous studies suggest the CAGE to be a valid a screening tool for alcohol abuse and dependence in primary care.²⁰

All measurements should demonstrate external (criterion) and internal (construct) validity, reliability, repeatability, sensitivity and responsiveness to be clinically useful.¹⁹ Few rating scales for mental disorders, if any, have documented all these characteristics. When it comes to making psychiatric diagnoses in general practice, one has to accept that diagnoses are largely based on GPs' clinical experience. Psychiatric rating scales should therefore demonstrate high internal validity/homogeneity (i.e. each item in a scale belongs to the same dimension and can be added to a sum score) to convince GPs about their clinical significance. Furthermore, scales should be brief and easy to interpret to be useful in general practice.^11,31 Results from a concomitant qualitative study (in Danish) actually indicated that the CMDQ may be useful in general practice due to its ease of use, usefulness in the reframing of physical symptoms, diagnostic accuracy and documentation of patients' mental health status.³²

A limitation of the study is that the CMDQ has not been tested in other cultures. So far, only the SCL-8 has demonstrated modest cultural transferability.¹⁶ Our finding that psychometric characteristics were improved for SCL-SOM and Whiteley-7 by lowering the point of dichotomisation might show some cultural variation. However, a work by Kessler et al. demonstrated that normalising attributions are predominant in general practice and a major cause for non-recognition of anxiety and depression.³³ The tendency to minimize symptoms and worries might apply to somatisation as well. Another limitation to our findings is that subscales await internal validation in the current study. Furthermore, we have not tested the ability of the instrument to measure changes over time and its sensitivity in monitoring intervention effect (i.e. responsiveness).

To our knowledge this is the first brief and easy-to-use screening instrument enumerating PPVs on a range of common mental disorders that has been validated for use in general practice. The first version of the instrument²⁸ used a higher dichotomisation of item responses. The scale-performances were based on studies³⁰ which used a 4-point scale. In our study we chose the original SCL-90-R 5-point Likert scale¹³ and used a dichotomisation of item-responses that, in fact, weakened the performances of the rating scales. By lowering the point of dichotomisation our results became fully comparable to previous findings.^17,30

No studies have shown whether the use of standardised research interviews actually predicts clinical outcome.³⁴ In other words: what is the optimal diagnostic accuracy of mental disorders in primary care? Until such studies have been conducted, the CMDQ may, however, serve as an effective adjunct to the GPs' unstructured diagnostic interviews.

For research and routine screening, cut-off scores may be selected to optimise the sensitivity or specificity of the subscales. With constant sensitivity and specificity, the PPV of a test is higher (and the NPV lower) in samples with high versus low prevalence of disease. For clinical use, one may benefit from using a selective case-finding approach artificially increasing disease prevalence by using a few opening key questions.³⁵ Used as a diagnostic aid, we believe that screening results are most effectively presented to the physician as continuous measures of psychopathology and not simply as dichotomised positive or negative reports.³¹

Generally, there is limited evidence to support mental health screening in primary care.^10,11 The draft guidelines on depression from the UK's National Institute for Clinical Excellence (NICE) recommend screening among high risk groups only. The US Preventive Services Task Force³⁶ stated in 2002 that there is sufficient evidence to encourage primary care clinicians to screen their adult patients for depression, as long as they have systems in place to assure accurate diagnosis, effective treatment, and follow-up. It is our hope, that the CMDQ can improve the effectiveness of the diagnostic evaluation by guiding clinicians towards symptom areas requiring particular assessment. We suggest that the instrument should be used and validated as a case-finding instrument in general practice. A forthcoming paper will document the monitoring capacity (sensitivity to change) of the instrument. Future studies should assess the impact of using the CMDQ on patient health outcomes. Computer-based versions of the instrument are about to be developed. The CMDQ is freely available at http://www.auh.dk/cl_psych/uk/research.htm.

Declaration

Funding: the Interdisciplinary Research Programme of the Danish National Research Council: [Sundhedsfremme og forebyggelsesforskning] (grant number 9801278) and the Regional Health Insurance, Aarhus County (project number 0871) funded this study.

Ethical approval: the Scientific Ethics Committee in the County of Aarhus approved this study.

Conflicts of interest: none.

We thank the GPs, the secretaries and the patients who participated in this study.

References

Author notes

aThe Research Unit for Functional Disorders, Aarhus University Hospital and bThe Research Unit for General Practice, University of Aarhus, Denmark