Set-valued dynamic treatment regimes for competing outcomes

Abstract

Dynamic treatment regimes (DTRs) operationalize the clinical decision process as a sequence of functions, one for each clinical decision, where each function maps up-to-date patient information to a single recommended treatment. Current methods for estimating optimal DTRs, for example Q-learning, require the specification of a single outcome by which the "goodness" of competing dynamic treatment regimes is measured. However, this is an over-simplification of the goal of clinical decision making, which aims to balance several potentially competing outcomes, for example, symptom relief and side-effect burden. When there are competing outcomes and patients do not know or cannot communicate their preferences, formation of a single composite outcome that correctly balances the competing outcomes is not possible. This problem also occurs when patient preferences evolve over time. We propose a method for constructing DTRs that accommodates competing outcomes by recommending sets of treatments at each decision point. Formally, we construct a sequence of set-valued functions that take as input up-to-date patient information and give as output a recommended subset of the possible treatments. For a given patient history, the recommended set of treatments contains all treatments that produce non-inferior outcome vectors. Constructing these set-valued functions requires solving a non-trivial enumeration problem. We offer an exact enumeration algorithm by recasting the problem as a linear mixed integer program. The proposed methods are illustrated using data from the CATIE schizophrenia study.

Trial registration: ClinicalTrials.gov NCT00014001.

Keywords: Competing outcomes; Composite outcomes; Dynamic treatment regimes; Personalized medicine; Preference elicitation.

© 2014, The International Biometric Society.

Figures

Figure 1

Diagram showing how the output of π2ΔIdeal(h2) depends on ΔY and ΔZ, and on the location of the point (r2Y(h2), r2Z(h2)).

Figure 2

Left: Diagram showing how the output of π̂1Δ(h1) depends on ΔP (clinically significant difference in PANSS) and ΔB (clinically significant difference in BMI), and on the joint treatment effect, at Phase 1. The cloud of points shows the possible joint treatment effects that can be realized by a single patient with history (panss = −25.5, bmi = −15.6) if the patient follows some feasible decision rule at Phase 2. That is, each point is associated with a different choice of Phase 2 decision rule. Note that for some future decision rules, the point lies in the {−1, 1} region, and for others it lies in the {−1} region; taking the union we have π̂1Δ(h1) = {−1, 1} for this patient. Center: Diagram showing how the output of π̂2Δ(h2) depends on ΔP and ΔB, and on the location of the point (r̂2P(h2), r̂2B(h2)) for all patients in the Phase 2 Tolerability group. Each plotted point shows the estimated joint treatment effect for a different patient in the dataset. Since Phase 2 is the last phase, there are no future decision rules to consider and each history is associated with a unique joint treatment effect. Right: Analogous plot for the Phase 2 Efficacy group.

Figure 3

Q̂1B(h1, a1, τ2) against Q̂1P(h1, a1, τ2) across τ2 ∈ F̃ (π̂2Δ) for a single patient with history (panss = −25.5, bmi = −15.6) in the CATIE data. Note, that while there are 61,659 polices in F̃(π̂2Δ) many of these yield similar predicted values for Q̂1B(h1, a1, τ2) and Q̂1P(h1, a1, τ2); we have plotted a random subset to make individual points more clearly visible. This display suggests that a patient presenting with H1 = h1, choosing perphenazine (PERP) is associated with better expected outcomes on BMI but worse on PANSS under feasible second stage rules.