Practical issues encountered while determining Minimal Clinically Important Difference in Patient-Reported Outcomes

Pascal Woaye-Hune, Jean-Benoit Hardouin, Paul-Antoine Lehur, Guillaume Meurette, Antoine Vanier, Pascal Woaye-Hune, Jean-Benoit Hardouin, Paul-Antoine Lehur, Guillaume Meurette, Antoine Vanier

Abstract

Background: Using a real dataset, we highlighted several major methodological issues raised by the estimation of the Minimal Clinically Important Difference (MCID) of a Patient-Reported Outcomes instrument. We especially considered the management of missing data and the use of more than two times of measurement. While inappropriate missing data management and inappropriate use of multiple time points can lead to loss of precision and/or bias in MCID estimation, these issues are almost never dealt with and require cautious considerations in the context of MCID estimation.

Methods: We used the LIGALONGO study (French Randomized Controlled Trial). We estimated MCID on the SF-36 General Health score by comparing many methods (distribution or anchor-based). Different techniques for imputation of missing data were performed (simple and multiple imputations). We also consider all measurement occasions by longitudinal modeling, and the dependence of the score difference on baseline.

Results: Three hundred ninety-three patients were studied. With distribution-based methods, a great variability in MCID was observed (from 3 to 26 points for improvement). Only 0.2 SD and 1/3 SD distribution methods gave MCID values consistent with anchor-based methods (from 4 to 7 points for improvement). The choice of missing data imputation technique clearly had an impact on MCID estimates. Simple imputation by mean score seemed to lead to out-of-range estimate, but as missing not at random mechanism can be hypothesized, even multiple imputations techniques can have led to an slight underestimation of MCID. Using 3 measurement occasions for improvement led to an increase in precision but lowered estimates.

Conclusion: This practical example illustrates the substantial impact of some methodological issues that are usually never dealt with for MCID estimation. Simulation studies are needed to investigate those issues.

Trial registration: NCT01240772 (ClinicalTrials.gov) registered on November 15, 2010.

Keywords: Longitudinal modeling; Methodology; Minimal clinically important difference; Minimal important difference; Missing data; Patient-reported outcomes.

Conflict of interest statement

The authors declare that they have no competing interests.

Figures

Fig. 1
Fig. 1
Distributions of the patients with no change, little improvement and little degradation between first (t1) and second measurement (t2): intersection points are here considered as a possible estimate for MCID
Fig. 2
Fig. 2
Illustration of the imputation methods used. MD Missing data. X completed item; MICE Multivariate Imputation with Chained Equations. 1PMS Personal Mean Score : each item is imputed 2Missing data in scores were imputed by the mean of observed scores ; missing data in anchor were imputed using a random sample weighted with observed probabilities of answers at corresponding anchor. 3Missing scores were imputed using personal mean matching, missing anchors were imputed using a polytomous regression, both using clinical and demographic variables. 4Missing scores were imputed using personal mean matching, missing anchors were imputed using a polytomous regression, both using clinical and demographic variables, and all scores from other dimensions
Fig. 3
Fig. 3
General characteristics of the LIGALONGO illustration study sample
Fig. 4
Fig. 4
Variations of MCID values for improved patients (Missing data imputed by simple MICE). Note: Red are anchor methods. Blue are distribution-based methods. Missing scores were imputed using personal mean matching, anchor was imputed using a polytomous regression, both using a demographic nariables, and General Health scores. SDc Standard Deviation of the change in scores. SDb Standard of teh baseline score. ROC01 Closet-point to rhe left of the reciever Operating curve diagram. Intesect Intersection point between the distributions of the change om scores between unchanged and improved patient. SEM Standard Error of Measurement. MDC Minimal Detectable Change

References

    1. McLeod LD, Coon CD, Martin SA, Fehnel SE, Hays RD. Interpreting patient-reported outcome results: US FDA guidance and emerging methods. Expert Rev Pharmacoecon Outcomes Res. 2011;11:163–169.
    1. Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10:407–415.
    1. Ferreira ML, Herbert RD, Ferreira PH, Latimer J, Ostelo RW, Nascimento DP, et al. A critical review of methods used to determine the smallest worthwhile effect of interventions for low back pain. J Clin Epidemiol. 2012;65:253–261.
    1. Copay AG, Subach BR, Glassman SD, Polly DW, Schuler TC. Understanding the minimum clinically important difference: a review of concepts and methods. Spine J Off J North Am Spine Soc. 2007;7:541–546.
    1. US Food and Drug Administration . Guidance for industry. Patient-reported outcome measures: use in medical product development to support labeling claims. 2009.
    1. Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol. 2008;61:102–109.
    1. Terwee CB, Roorda LD, Dekker J, Bierma-Zeinstra SM, Peat G, Jordan KP, et al. Mind the MIC: large variation among populations and methods. J Clin Epidemiol. 2010;63:524–534.
    1. Vanier A, Woaye-Hune P, Toscano A, Sébille V, Hardouin J-B. What are all the proposed methods to estimate the minimal clinically important difference of a patient-reported outcome measure? A systematic review. Qual Life Res. 2017;26(S1):20.
    1. Stucki G, Daltroy L, Katz JN, Johannesson M, Liang MH. Interpretation of change scores in ordinal clinical scales and health status measures: the whole may not equal the sum of the parts. J Clin Epidemiol. 1996;49:711–717.
    1. Tubach F, Wells GA, Ravaud P, Dougados M. Minimal clinically important difference, low disease activity state, and patient acceptable symptom state: methodological issues. J Rheumatol. 2005;32:2025–2029.
    1. Lehur PA, Didnée AS, Faucheron J-L, Meurette G, Zerbib P, Siproudhis L, et al. Cost-effectiveness of new surgical treatments for hemorrhoidal disease: a multicentre randomized controlled trial comparing transanal Doppler-guided hemorrhoidal artery ligation with mucopexy and circular stapled hemorrhoidopexy. Ann Surg. 2016;264:710–716.
    1. Angst F, Aeschlimann A, Stucki G. Smallest detectable and minimal clinically important differences of rehabilitation intervention with their implications for required sample sizes using WOMAC and SF-36 quality of life measurement instruments in patients with osteoarthritis of the lower extremities. Arthritis Rheum. 2001;45:384–391.
    1. Angst F, Aeschlimann A, Angst J. The minimal clinically important difference raised the significance of outcome effects above the statistical level, with methodological implications for future studies. J Clin Epidemiol. 2017;82:128–136.
    1. Leidy NK, Wyrwich KW. Bridging the gap: using triangulation methodology to estimate minimal clinically important differences (MCIDs) COPD J Chronic Obstr Pulm Dis. 2005;2:157–165.
    1. Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Med Care. 1989;27:S178–S189.
    1. Cohen J. Statistical power analysis for the behavioral sciences. 2. New York: Psychology Press; 2009.
    1. Stratford PW, Binkley JM, Riddle DL. Health status measures: strategies and analytic methods for assessing change scores. Phys Ther. 1996;76:1109–1123.
    1. Liang MH, Fossel AH, Larson MG. Comparisons of five health status instruments for orthopedic evaluation. Med Care. 1990;28:632–642.
    1. Nunnally JC, Bernstein IH. Psychometric theory. 3. New York: McGraw-Hill; 1994.
    1. Wyrwich KW, Tierney WM, Wolinsky FD. Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life. J Clin Epidemiol. 1999;52:861–873.
    1. Wyrwich KW, Nienaber NA, Tierney WM, Wolinsky FD. Linking clinical relevance and statistical significance in evaluating intra-individual changes in health-related quality of life. Med Care. 1999;37:469–478.
    1. Beaton DE. Understanding the relevance of measured change through studies of responsiveness. Spine. 2000;25:3192–3199.
    1. de Vet HCW, Terwee CB. The minimal detectable change should not replace the minimal important difference. J Clin Epidemiol. 2010;63:804–805.
    1. Angst F, Benz T, Lehmann S, Aeschlimann A, Angst J. Multidimensional minimal clinically important differences in knee osteoarthritis after comprehensive rehabilitation: a prospective evaluation from the Bad Zurzach Osteoarthritis Study. RMD Open. 2018;4:e000685.
    1. Azimi P, Yazdanian T, Benzel EC. Determination of minimally clinically important differences for JOABPEQ measure after discectomy in patients with lumbar disc herniation. J Spine Surg Hong Kong. 2018;4:102–108.
    1. Hays RD, Woolley JM. The concept of clinically meaningful difference in health-related quality-of-life research. How meaningful is it? Pharmacoeconomics. 2000;18:419–423.
    1. Redelmeier DA, Lorig K. Assessing the clinical importance of symptomatic improvements. An illustration in rheumatology. Arch Intern Med. 1993;153:1337–1342.
    1. Redelmeier DA, Guyatt GH, Goldstein RS. Assessing the minimal important difference in symptoms: a comparison of two techniques. J Clin Epidemiol. 1996;49:1215–1219.
    1. Gerlinger C, Schumacher U, Faustmann T, Colligs A, Schmitz H, Seitz C. Defining a minimal clinically important difference for endometriosis-associated pelvic pain measured on a visual analog scale: analyses of two placebo-controlled, randomized trials. Health Qual Life Outcomes. 2010;8:138.
    1. Barrett B, Brown R, Mundt M. Comparison of anchor-based and distributional approaches in estimating important difference in common cold. Qual Life Res Int J Qual Life Asp Treat Care Rehabil. 2008;17:75–85.
    1. Little R, Rubin D. Statistical analysis with missing data. New-York: Wiley; 1987.
    1. Leplège A, Ecosse E, Pouchot J, Coste J, Perneger T. Le questionnaire MOS SF-36: manuel de l’utilisateur et guide d’interprétation des scores. Paris: Editions ESTEM; 2001.
    1. Faria R, Gomes M, Epstein D, White IR. A guide to handling missing data in cost-effectiveness analysis conducted within randomised controlled trials. Pharmacoeconomics. 2014;32:1157–1170.
    1. van Buuren S. Multiple imputation of discrete and continuous data by fully conditional specification. Stat Methods Med Res. 2007;16:219–242.
    1. Ware JE, Jr, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care. 1992;30:473–483.
    1. R Development Core Team . R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2016.
    1. van Buuren S, Groothuis-Oudshoorn K. mice: multivariate imputation by chained equations in R. J Stat Softw. 2011;45:1–67.
    1. StataCorp . Stata Statistical Software: Release 13. College Station: StataCorp LP; 2013.
    1. Hardouin J-B. SF36FR: Stata module to compute the scores of the French versions of the SF36 questionnaire. 2013.
    1. Azimi P, Mohammadi HR, Benzel EC, Shahzadi S, Azhari S, Montazeri A. Artificial neural networks in neurosurgery. J Neurol Neurosurg Psychiatry. 2015;86:251–256.
    1. Rouquette A, Blanchin M, Sébille V, Guillemin F, Côté SM, Falissard B, et al. The minimal clinically important difference determined using item response theory models: an attempt to solve the issue of the association with baseline score. J Clin Epidemiol. 2014;67:433–440.
    1. Schwartz CE, Ayandeh A, Motl RW. Investigating the minimal important difference in ambulation in multiple sclerosis: a disconnect between performance-based and patient-reported outcomes? J Neurol Sci. 2014;347:268–274.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe