Predicting Dropout from Inpatient Substance Use Disorder Treatment: A Prospective Validation Study of the OQ-Analyst

Abstract

Background and aims: There is an urgent need for tools allowing therapists to identify patients at risk of dropout. The OQ-Analyst, an increasingly popular computer-based system, is used to track patient progress and predict dropout. However, we have been unable to find empirical documentation regarding the ability of OQ-Analyst to predict dropout. The aim of the present study was to perform the first direct test of the ability of the OQ-Analyst to predict dropout.

Design: Patients were consecutively enlisted in a naturalistic, prospective, longitudinal clinical trial. As interventions based on feedback from the OQ-Analyst could alter the outcome and potentially render the prediction wrong, feedback was withheld from patients and therapists.

Setting: The study was carried out during 2011-2013 in an inpatient substance use disorder clinic in Oslo, Norway.

Participants: Patients aged 18 to 28 years who met criteria for a principal diagnosis of mental or behavioural disorder due to psychoactive substance use (ICD 10; F10.2-F19.2).

Measurements: Red signal (predictions of high risk) from the Norwegian version of the OQ-Analyst were compared with dropouts identified using patient medical records as the standard for predictive accuracy.

Findings: A total of 40 patients completed 647 OQ assessments resulting in 46 red signals. There were 27 observed dropouts, only one of which followed after a red signal. Patients indicated by the OQ-Analyst as being at high risk of dropping out were no more likely to do so than those indicated as being at low risk. Random intercept logistic regression predicting dropout from a red signal was statistically nonsignificant. Bayes factor supports no association.

Conclusions: The study does not support the predictive ability of the OQ-Analyst for the present patient population. In the absence of empirical evidence of predictive ability, it may be better not to assume such ability.

Keywords: OQ-45; OQ-Analyst; attrition; dropout; feedback technology; negative treatment outcome; prediction; progress monitoring; substance use disorder treatment.

Conflict of interest statement

Declaration of conflicting interest:The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Figures

Figure 1.

Simulation-based power estimates (y-axis) from runs with varying numbers of participants (delineated by colour), varying assumed sensitivity of ‘red signal’ to subsequent dropout (x-axis). Lines represent generalised linear models with a logit link, predicting observed powers for each number of simulated respondents by varying sensitivity.

Figure 2.

Flow chart compliant with STARD showing patient recruitment, OQ-Analyst predictions and observations extracted from medical journals.

Figure 3.

Length of stay (x-axis) for included patients, sorted by total length of stay. Red x denotes dropout, blue triangle completed treatment. Red reflects red signal, grey otherwise. No line indicates that patient was not admitted, for example, due to dropout or out on leave. Treatment cessation with no marker due to any other reason, such as the patient being moved to another ward.

Figure 4.

Simulation-based Bayes factor estimates based on assumed prior relative risk of dropout following ‘red signal’ and the observed patient data. The line is a fitted exponential curve on risk. BF01 falls below with assumed prior relative risks exceeding approximately 1.9.