A tutorial on sensitivity analyses in clinical trials: the what, why, when and how

Lehana Thabane, Lawrence Mbuagbaw, Shiyuan Zhang, Zainab Samaan, Maura Marcucci, Chenglin Ye, Marroon Thabane, Lora Giangregorio, Brittany Dennis, Daisy Kosa, Victoria Borg Debono, Rejane Dillenburg, Vincent Fruci, Monica Bawor, Juneyoung Lee, George Wells, Charles H Goldsmith, Lehana Thabane, Lawrence Mbuagbaw, Shiyuan Zhang, Zainab Samaan, Maura Marcucci, Chenglin Ye, Marroon Thabane, Lora Giangregorio, Brittany Dennis, Daisy Kosa, Victoria Borg Debono, Rejane Dillenburg, Vincent Fruci, Monica Bawor, Juneyoung Lee, George Wells, Charles H Goldsmith

Abstract

Background: Sensitivity analyses play a crucial role in assessing the robustness of the findings or conclusions based on primary analyses of data in clinical trials. They are a critical way to assess the impact, effect or influence of key assumptions or variations--such as different methods of analysis, definitions of outcomes, protocol deviations, missing data, and outliers--on the overall conclusions of a study.The current paper is the second in a series of tutorial-type manuscripts intended to discuss and clarify aspects related to key methodological issues in the design and analysis of clinical trials.

Discussion: In this paper we will provide a detailed exploration of the key aspects of sensitivity analyses including: 1) what sensitivity analyses are, why they are needed, and how often they are used in practice; 2) the different types of sensitivity analyses that one can do, with examples from the literature; 3) some frequently asked questions about sensitivity analyses; and 4) some suggestions on how to report the results of sensitivity analyses in clinical trials.

Summary: When reporting on a clinical trial, we recommend including planned or posthoc sensitivity analyses, the corresponding rationale and results along with the discussion of the consequences of these analyses on the overall findings of the study.

References

    1. Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, Robson R, Thabane M, Giangregorio L, Goldsmith CH. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol. 2010;10:1. doi: 10.1186/1471-2288-10-1.
    1. Schneeweiss S. Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol Drug Saf. 2006;15(5):291–303. doi: 10.1002/pds.1200.
    1. Viel JF, Pobel D, Carre A. Incidence of leukaemia in young people around the La Hague nuclear waste reprocessing plant: a sensitivity analysis. Stat Med. 1995;14(21–22):2459–2472.
    1. Goldsmith CH, Gafni A, Drummond MF, Torrance GW, Stoddart GL. Sensitivity Analysis and Experimental Design: The Case of Economic Evaluation of Health Care Programmes. Proceedings of the Third Canadian Conference on Health Economics 1986. Winnipeg MB: The University of Manitoba Press; 1987.
    1. Saltelli A, Tarantola S, Campolongo F, Ratto M. Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models. New York, NY: Willey; 2004.
    1. Saltelli A, Ratto M, Andres T, Campolongo F, Cariboni J, Gatelli D, Saisana M, Tarantola S. Global Sensitivity Analysis: The Primer. New York, NY: Wiley-Interscience; 2008.
    1. Hunink MGM, Glasziou PP, Siegel JE, Weeks JC, Pliskin JS, Elstein AS, Weinstein MC. Decision Making in Health and Medicine: Integrating Evidence and Values. Cambridge: Cambridge University Press; 2001.
    1. USFDA. International Conference on Harmonisation; Guidance on Statistical Principles for Clinical Trials. Guideline E9. Statistical principles for clinical trials. Federal Register, 16 September 1998, Vol. 63, No. 179, p. 49583 . [ ]
    1. NICE. Guide to the methods of technology appraisal . [ ]
    1. Ma J, Thabane L, Kaczorowski J, Chambers L, Dolovich L, Karwalajtys T, Levitt C. Comparison of Bayesian and classical methods in the analysis of cluster randomized controlled trials with a binary outcome: the Community Hypertension Assessment Trial (CHAT) BMC Med Res Methodol. 2009;9:37. doi: 10.1186/1471-2288-9-37.
    1. Peters TJ, Richards SH, Bankhead CR, Ades AE, Sterne JA. Comparison of methods for analysing cluster randomized trials: an example involving a factorial design. Int J Epidemiol. 2003;32(5):840–846. doi: 10.1093/ije/dyg228.
    1. Chu R, Thabane L, Ma J, Holbrook A, Pullenayegum E, Devereaux PJ. Comparing methods to estimate treatment effects on a continuous outcome in multicentre randomized controlled trials: a simulation study. BMC Med Res Methodol. 2011;11:21. doi: 10.1186/1471-2288-11-21.
    1. Kleinbaum DG, Klein M. Survival Analysis – A-Self Learning Text. 3. Springer; 2012.
    1. Barnett V, Lewis T. Outliers in Statistical Data. 3. John Wiley & Sons; 1994.
    1. Grubbs FE. Procedures for detecting outlying observations in samples. Technometrics. 1969;11:1–21. doi: 10.1080/00401706.1969.10490657.
    1. Thabane L, Akhtar-Danesh N. Guidelines for reporting descriptive statistics in health research. Nurse Res. 2008;15(2):72–81.
    1. Williams NH, Edwards RT, Linck P, Muntz R, Hibbs R, Wilkinson C, Russell I, Russell D, Hounsome B. Cost-utility analysis of osteopathy in primary care: results from a pragmatic randomized controlled trial. Fam Pract. 2004;21(6):643–650. doi: 10.1093/fampra/cmh612.
    1. Zetta S, Smith K, Jones M, Allcoat P, Sullivan F. Evaluating the Angina Plan in Patients Admitted to Hospital with Angina: A Randomized Controlled Trial. Cardiovascular Therapeutics. 2011;29(2):112–124. doi: 10.1111/j.1755-5922.2009.00109.x.
    1. Morden JP, Lambert PC, Latimer N, Abrams KR, Wailoo AJ. Assessing methods for dealing with treatment switching in randomised controlled trials: a simulation study. BMC Med Res Methodol. 2011;11:4. doi: 10.1186/1471-2288-11-4.
    1. White IR, Walker S, Babiker AG, Darbyshire JH. Impact of treatment changes on the interpretation of the Concorde trial. AIDS. 1997;11(8):999–1006. doi: 10.1097/00002030-199708000-00008.
    1. Borrelli B. The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent. 2011;71(Suppl 1):S52–S63.
    1. Lawton J, Jenkins N, Darbyshire JL, Holman RR, Farmer AJ, Hallowell N. Challenges of maintaining research protocol fidelity in a clinical care setting: a qualitative study of the experiences and views of patients and staff participating in a randomized controlled trial. Trials. 2011;12:108. doi: 10.1186/1745-6215-12-108.
    1. Ye C, Giangregorio L, Holbrook A, Pullenayegum E, Goldsmith CH, Thabane L. Data withdrawal in randomized controlled trials: Defining the problem and proposing solutions: a commentary. Contemp Clin Trials. 2011;32(3):318–322. doi: 10.1016/j.cct.2011.01.016.
    1. Horwitz RI, Horwitz SM. Adherence to treatment and health outcomes. Arch Intern Med. 1993;153(16):1863–1868. doi: 10.1001/archinte.1993.00410160017001.
    1. Peduzzi P, Wittes J, Detre K, Holford T. Analysis as-randomized and the problem of non-adherence: an example from the Veterans Affairs Randomized Trial of Coronary Artery Bypass Surgery. Stat Med. 1993;12(13):1185–1195. doi: 10.1002/sim.4780121302.
    1. Montori VM, Guyatt GH. Intention-to-treat principle. CMAJ. 2001;165(10):1339–1341.
    1. Gibaldi M, Sullivan S. Intention-to-treat analysis in randomized trials: who gets counted? J Clin Pharmacol. 1997;37(8):667–672. doi: 10.1002/j.1552-4604.1997.tb04353.x.
    1. Porta M. A dictionary of epidemiology. 5. Oxford: Oxford University Press, Inc; 2008.
    1. Everitt B. Medical statistics from A to Z. 2. Cambridge: Cambridge University Press; 2006.
    1. Sainani KL. Making sense of intention-to-treat. PM R. 2010;2(3):209–213. doi: 10.1016/j.pmrj.2010.01.004.
    1. Bendtsen P, McCambridge J, Bendtsen M, Karlsson N, Nilsen P. Effectiveness of a proactive mail-based alcohol internet intervention for university students: dismantling the assessment and feedback components in a randomized controlled trial. J Med Internet Res. 2012;14(5):e142. doi: 10.2196/jmir.2062.
    1. Brox JI, Nygaard OP, Holm I, Keller A, Ingebrigtsen T, Reikeras O. Four-year follow-up of surgical versus non-surgical therapy for chronic low back pain. Ann Rheum Dis. 2010;69(9):1643–1648. doi: 10.1136/ard.2009.108902.
    1. McKnight PE, McKnight KM, Sidani S, Figueredo AJ. Missing Data: A Gentle Introduction. New York, NY: Guilford; 2007.
    1. Graham JW. Missing data analysis: making it work in the real world. Annu Rev Psychol. 2009;60:549–576. doi: 10.1146/annurev.psych.58.110405.085530.
    1. Little RJ, D'Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT, Frangakis C, Hogan JW, Molenberghs G, Murphy SA. et al.The Prevention and Treatment of Missing Data in Clinical Trials. New England Journal of Medicine. 2012;367(14):1355–1360. doi: 10.1056/NEJMsr1203730.
    1. Little RJA, Rubin DB. Statistical Analysis with Missing Data. 2. New York NY: Wiley; 2002.
    1. Rubin DB. Multiple Imputation for Nonresponse in Surveys. John Wiley & Sons, Inc: New York NY; 1987.
    1. Schafer JL. Analysis of Incomplete Multivariate Data. New York: Chapman and Hall; 1997.
    1. Son H, Friedmann E, Thomas SA. Application of pattern mixture models to address missing data in longitudinal data analysis using SPSS. Nursing research. 2012;61(3):195–203. doi: 10.1097/NNR.0b013e3182541d8c.
    1. Peters SA, Bots ML, den Ruijter HM, Palmer MK, Grobbee DE, Crouse JR 3rd, O'Leary DH, Evans GW, Raichlen JS, Moons KG. et al.Multiple imputation of missing repeated outcome measurements did not add to linear mixed-effects models. J Clin Epidemiol. 2012;65(6):686–695. doi: 10.1016/j.jclinepi.2011.11.012.
    1. Zhang H, Paik MC. Handling missing responses in generalized linear mixed model without specifying missing mechanism. J Biopharm Stat. 2009;19(6):1001–1017. doi: 10.1080/10543400903242761.
    1. Chen HY, Gao S. Estimation of average treatment effect with incompletely observed longitudinal data: application to a smoking cessation study. Statistics in medicine. 2009;28(19):2451–2472. doi: 10.1002/sim.3617.
    1. Ma J, Akhtar-Danesh N, Dolovich L, Thabane L. Imputation strategies for missing binary outcomes in cluster randomized trials. BMC Med Res Methodol. 2011;11:18. doi: 10.1186/1471-2288-11-18.
    1. Kingsley GH, Kowalczyk A, Taylor H, Ibrahim F, Packham JC, McHugh NJ, Mulherin DM, Kitas GD, Chakravarty K, Tom BD. et al.A randomized placebo-controlled trial of methotrexate in psoriatic arthritis. Rheumatology (Oxford) 2012;51(8):1368–1377. doi: 10.1093/rheumatology/kes001.
    1. de Pauw BE, Sable CA, Walsh TJ, Lupinacci RJ, Bourque MR, Wise BA, Nguyen BY, DiNubile MJ, Teppler H. Impact of alternate definitions of fever resolution on the composite endpoint in clinical trials of empirical antifungal therapy for neutropenic patients with persistent fever: analysis of results from the Caspofungin Empirical Therapy Study. Transpl Infect Dis. 2006;8(1):31–37. doi: 10.1111/j.1399-3062.2006.00127.x.
    1. A randomized, double-blind, futility clinical trial of creatine and minocycline in early Parkinson disease. Neurology. 2006;66(5)):664–671.
    1. Song P-K. Correlated Data Analysis: Modeling, Analytics and Applications. New York, NY: Springer Verlag; 2007.
    1. Pintilie M. Competing Risks: A Practical Perspective. New York, NY: John Wiley; 2006.
    1. Tai BC, Grundy R, Machin D. On the importance of accounting for competing risks in pediatric brain cancer: II. Regression modeling and sample size. Int J Radiat Oncol Biol Phys. 2011;79(4):1139–1146. doi: 10.1016/j.ijrobp.2009.12.024.
    1. Holbrook JT, Wise RA, Gold BD, Blake K, Brown ED, Castro M, Dozor AJ, Lima JJ, Mastronarde JG, Sockrider MM. et al.Lansoprazole for children with poorly controlled asthma: a randomized controlled trial. JAMA. 2012;307(4):373–381.
    1. Holbrook A, Thabane L, Keshavjee K, Dolovich L, Bernstein B, Chan D, Troyan S, Foster G, Gerstein H. Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial. CMAJ: Canadian Medical Association journal = journal de l’Association medicale canadienne. 2009;181(1–2):37–44.
    1. Hilbe JM. Negative Binomial Regression. 2. Cambridge: Cambridge University Press; 2011.
    1. Forsblom C, Harjutsalo V, Thorn LM, Waden J, Tolonen N, Saraheimo M, Gordin D, Moran JL, Thomas MC, Groop PH. Competing-risk analysis of ESRD and death among patients with type 1 diabetes and macroalbuminuria. J Am Soc Nephrol. 2011;22(3):537–544. doi: 10.1681/ASN.2010020194.
    1. Grams ME, Coresh J, Segev DL, Kucirka LM, Tighiouart H, Sarnak MJ. Vascular disease, ESRD, and death: interpreting competing risk analyses. Clin J Am Soc Nephrol. 2012;7(10):1606–1614. doi: 10.2215/CJN.03460412.
    1. Lim HJ, Zhang X, Dyck R, Osgood N. Methods of competing risks analysis of end-stage renal disease and mortality among people with diabetes. BMC Med Res Methodol. 2010;10:97. doi: 10.1186/1471-2288-10-97.
    1. Chu R, Walter SD, Guyatt G, Devereaux PJ, Walsh M, Thorlund K, Thabane L. Assessment and implication of prognostic imbalance in randomized controlled trials with a binary outcome–a simulation study. PLoS One. 2012;7(5):e36677. doi: 10.1371/journal.pone.0036677.
    1. Bowen A, Hesketh A, Patchick E, Young A, Davies L, Vail A, Long AF, Watkins C, Wilkinson M, Pearl G. et al.Effectiveness of enhanced communication therapy in the first four months after stroke for aphasia and dysarthria: a randomised controlled trial. BMJ. 2012;345:e4407. doi: 10.1136/bmj.e4407.
    1. Spiegelhalter DJ, Best NG, Lunn D, Thomas A. Bayesian Analysis using BUGS: A Practical Introduction. New York, NY: Chapman and Hall; 2009.
    1. Byers AL, Allore H, Gill TM, Peduzzi PN. Application of negative binomial modeling for discrete outcomes: a case study in aging research. J Clin Epidemiol. 2003;56(6):559–564. doi: 10.1016/S0895-4356(03)00028-3.
    1. Yusuf S, Wittes J, Probstfield J, Tyroler HA. Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. JAMA: the journal of the American Medical Association. 1991;266(1):93–98. doi: 10.1001/jama.1991.03470010097038.
    1. Altman DG. Better reporting of randomised controlled trials: the CONSORT statement. BMJ. 1996;313(7057):570–571. doi: 10.1136/bmj.313.7057.570.
    1. Mauskopf JA, Sullivan SD, Annemans L, Caro J, Mullins CD, Nuijten M, Orlewska E, Watkins J, Trueman P. Principles of good practice for budget impact analysis: report of the ISPOR Task Force on good research practices–budget impact analysis. Value Health. 2007;10(5):336–347. doi: 10.1111/j.1524-4733.2007.00187.x.

Source: PubMed

Sponzoři a spolupracovníci

Zdravotní podmínky

Drogové intervence

3
Předplatit