Bounding the per-protocol effect in randomized trials: an application to colorectal cancer screening

Abstract

Background: The per-protocol effect is the effect that would have been observed in a randomized trial had everybody followed the protocol. Though obtaining a valid point estimate for the per-protocol effect requires assumptions that are unverifiable and often implausible, lower and upper bounds for the per-protocol effect may be estimated under more plausible assumptions. Strategies for obtaining bounds, known as "partial identification" methods, are especially promising in randomized trials.

Results: We estimated bounds for the per-protocol effect of colorectal cancer screening in the Norwegian Colorectal Cancer Prevention trial, a randomized trial of one-time sigmoidoscopy screening in 98,792 men and women aged 50-64 years. The screening was not available to the control arm, while approximately two thirds of individuals in the treatment arm attended the screening. Study outcomes included colorectal cancer incidence and mortality over 10 years of follow-up. Without any assumptions, the data alone provide little information about the size of the effect. Under the assumption that randomization had no effect on the outcome except through screening, a point estimate for the risk under no screening and bounds for the risk under screening are achievable. Thus, the 10-year risk difference for colorectal cancer was estimated to be at least -0.6 % but less than 37.0 %. Bounds for the risk difference for colorectal cancer mortality (-0.2 to 37.4 %) and all-cause mortality (-5.1 to 32.6 %) had similar widths. These bounds appear helpful in quantifying the maximum possible effectiveness, but cannot rule out harm. By making further assumptions about the effect in the subpopulation who would not attend screening regardless of their randomization arm, narrower bounds can be achieved.

Conclusions: Bounding the per-protocol effect under several sets of assumptions illuminates our reliance on unverifiable assumptions, highlights the range of effect sizes we are most confident in, and can sometimes demonstrate whether to expect certain subpopulations to receive more benefit or harm than others.

Trial registration: Clinicaltrials.gov identifier NCT00119912 (registered 6 July 2005).

Figures

Fig. 1

Bounds for the per-protocol 10-year risk difference when restricting the maximum value of the risk under screening in the “never-takers”, aged 55–64 years. Gray area in nested plots indicates the area of detail presented in the outer plots

Fig. 2

Age-standardized bounds (for ages 50–64) for the per-protocol 10-year risk difference under various sets of assumptions. Sets of assumptions include: a No assumptions. b The instrumental conditions (relevance, exclusion restriction, and exchangeability). c The instrumental conditions plus an assumed maximum risk under screening in the “never-takers” of 2 %, 1 %, and 40 % for the CRC incidence, CRC mortality, and all-cause mortality, respectively. d The instrumental conditions plus an assumed maximum risk under screening in the “never-takers” of 1.5 %, 0.75 %, and 30 % for the CRC incidence, CRC mortality, and all-cause mortality, respectively. e The instrumental conditions plus an assumed maximum risk under screening in the “never-takers” of 1 %, 0.5 %, and 20 % for the CRC incidence, CRC mortality, and all-cause mortality, respectively. f The instrumental conditions plus additive effect homogeneity. g The instrumental conditions plus multiplicative effect homogeneity. The dotted line indicates the intention-to-treat effect

Fig. 3

Age-standardized bounds (for ages 50–64) for the per-protocol 10-year risk ratio under various sets of assumptions. Sets of assumptions include: a No assumptions. b The instrumental conditions (relevance, exclusion restriction, and exchangeability). c The instrumental conditions plus an assumed maximum risk under screening in the “never-takers” of 2 %, 1 %, and 40 % for the CRC incidence, CRC mortality, and all-cause mortality, respectively. d The instrumental conditions plus an assumed maximum risk under screening in the “never-takers” of 1.5 %, 0.75 %, and 30 % for the CRC incidence, CRC mortality, and all-cause mortality, respectively. e The instrumental conditions plus an assumed maximum risk under screening in the “never-takers” of 1 %, 0.5 %, and 20 % for the CRC incidence, CRC mortality, and all-cause mortality, respectively. f The instrumental conditions plus additive effect homogeneity. g The instrumental conditions plus multiplicative effect homogeneity. The dotted line indicates the intention-to-treat effect