An item response theory evaluation of three depression assessment instruments in a clinical sample

Abstract

Background: This study investigates whether an analysis, based on Item Response Theory (IRT), can be used for initial evaluations of depression assessment instruments in a limited patient sample from an affective disorder outpatient clinic, with the aim to finding major advantages and deficiencies of the instruments.

Methods: Three depression assessment instruments, the depression module from the Patient Health Questionnaire (PHQ9), the depression subscale of Affective Self Rating Scale (AS-18-D) and the Montgomery-Åsberg Depression Rating Scale (MADRS) were evaluated in a sample of 61 patients with affective disorder diagnoses, mainly bipolar disorder. A '3- step IRT strategy' was used.

Results: In a first step, the Mokken non-parametric analysis showed that PHQ9 and AS-18-D had strong overall scalabilities of 0.510 [C.I. 0.42, 0.61] and 0,513 [C.I. 0.41, 0.63] respectively, while MADRS had a weak scalability of 0.339 [C.I. 0.25, 0.43]. In a second step, a Rasch model analysis indicated large differences concerning the item discriminating capacity and was therefore considered not suitable for the data. In third step, applying a more flexible two parameter model, all three instruments showed large differences in item information and items had a low capacity to reliably measure respondents at low levels of depression severity.

Conclusions: We conclude that a stepwise IRT-approach, as performed in this study, is a suitable tool for studying assessment instruments at early stages of development. Such an analysis can give useful information, even in small samples, in order to construct more precise measurements or to evaluate existing assessment instruments. The study suggests that the PHQ9 and AS-18-D can be useful for measurement of depression severity in an outpatient clinic for affective disorder, while the MADRS shows weak measurement properties for this type of patients.

Figures

Figure 1

An insufficient coverage of the items compared to the severity of depression in a large part of the sample.

Figure 2

An insufficient coverage of the items compared to the severity of depression in a large part of the sample.

Figure 3

An insufficient coverage of the items compared to the severity of depression in a large part of the sample.

Figure 4

The areas under the curves in Figure 4-6 represent the approximate amount of information that can be picked up by the individual items.

Figure 5

The areas under the curves in Figure 4-6 represent the approximate amount of information that can be picked up by the individual items.

Figure 6

The areas under the curves in Figure 4-6 represent the approximate amount of information that can be picked up by the individual items.

Figure 7

The relationship between the item locations in the three instruments.