Psychometric validation and interpretation of the Nocturia Impact Diary in a clinical trial setting

Stacie Hudgens, Amy Howerter, Ela Polek, Fredrik L Andersson, Stacie Hudgens, Amy Howerter, Ela Polek, Fredrik L Andersson

Abstract

Purpose: Psychometric evaluation of the Nocturia Impact (NI) Diary was conducted to support its use as a trial endpoint.

Methods: As part of a randomized, controlled Phase 2 clinical trial investigating a novel drug candidate for nocturnal polyuria, adult nocturia patients completed the NI Diary and a voiding diary for three nights preceding their clinic visit at Baseline and Weeks 1, 4, 8, and 12 (end of treatment). Exit interviews were conducted to obtain patient impressions of the NI Diary.

Results: A total of N = 302 participants were included. Confirmatory factor analysis (CFA) indicated that the 11-item measure is unidimensional with values of CFI, TLI, and RMSEA meeting relevant thresholds. Good internal consistency (Cronbach's α 0.941) and test-retest reliability (intra-class correlation coefficients 0.730-0.880). Convergent validity with two reference measures was demonstrated with strong correlations of 0.573-0.730 were shown. Significant differences (P = 0.0018, standardized effect size = 0.372) between groups defined by number of night-time voids supported known-groups validity. Exit interviews in 66 patients indicated all participants experienced improvement in at least 1 NI Diary item and that a 1-point improvement on the item response scale and 1-void reduction per night (associated with an average best cut point on ROC analysis of - 11.6) constituted meaningful improvement. Anchor and distribution-based analyses identified a meaningful change threshold of - 15 to - 18 points on the NI Diary.

Conclusion: The NI Diary is a reliable and valid patient-reported psychometric instrument which is fit-for-purpose to evaluate the impact of nocturia on patient quality of life in the clinical trial setting. Trial registration number and registration date NCT03201419; June 28, 2017.

Keywords: Anchor-based; Meaningful change threshold; NI Diary; Nocturia; Patient-reported outcomes; Quality of life.

Conflict of interest statement

Clinical Outcomes Solutions received funding from Ferring Pharmaceuticals A/S to conduct this study. No other conflicts to report.

© 2021. The Author(s).

Figures

Fig. 1
Fig. 1
Overview of Psychometric Analyses performed for the current study
Fig. 2
Fig. 2
Item discrimination curve for all five response options for Item 5 (irritable or moody). Abscissa represents the Mean NI Diary total score at Baseline and ordinate represents Cumulative Proportion of Subjects. The curves for different response options are well-separated with direct correspondence between higher NI Diary scores below which all subjects score and higher severity of response of the item. For instance, for those responding 0 (Not at all) to indicate how much the experience moodiness or irritability, 100% of patients had a Baseline NI Diary score NI nocturia impact
Fig. 3
Fig. 3
Confirmatory factor analysis: standardized factor loadings. CFA confirmatory factor analysis, NI nocturia impact

References

    1. International Continence Society. Retrieved October 6, 2020, from (last updated June 2018).
    1. Bosch JL, Weiss JP. The prevalence and causes of nocturia. Journal of Urology. 2010;184(2):440–446. doi: 10.1016/j.juro.2010.04.011.
    1. Oelke M, De Wachter S, Drake MJ, Giannantoni A, Kirby M, Orme S, Rees J, van Kerrebroeck P, Everaert K. A practical approach to the management of nocturia. International Journal of Clinical Practice. 2017;71(11):e13027. doi: 10.1111/ijcp.13027.
    1. Weiss JP. Nocturia: Focus on etiology and consequences. Reviews in Urology. 2012;14(3–4):48–55.
    1. Dmochowski R, Brucker BM, Cole E, Kawahara S, Pulicharam R, Burk C, Tung A, Hale D. Economic burden of illness in adult patients with nocturia. Journal of Managed Care and Specialty Pharmacy. 2019;25(5):593–604. doi: 10.18553/jmcp.2019.18067.
    1. Holm-Larsen T. The economic impact of nocturia. Neurourology and Urodynamics. 2014;33(Suppl 1):S10–14. doi: 10.1002/nau.22593.
    1. Kupelian V, Wei JT, O'Leary MP, Norgaard JP, Rosen RC, McKinlay JB. Nocturia and quality of life: Results from the Boston area community health survey. European Urology. 2012;61(1):78–84. doi: 10.1016/j.eururo.2011.05.065.
    1. Kim SO, Choi HS, Kim YJ, Kim HS, Hwang IS, Hwang EC, Oh KJ, Jung SI, Kang TW, Kwon D, Park K, Ryu SB. Impact of nocturia on health-related quality of life and medical outcomes study sleep score in men. International Neurourology Journal. 2011;15(2):82–86. doi: 10.5213/inj.2011.15.2.82.
    1. Tikkinen KA, Johnson TM, II, Tammela TL, Sintonen H, Haukka J, Huhtala H, Auvinen A. Nocturia frequency, bother, and quality of life: How often is too often? A population-based study in Finland. European Urology. 2010;57(3):488–496. doi: 10.1016/j.eururo.2009.03.080.
    1. Zeng VY, Milligan G, Piercy J, Anderson P, Andersson FL. Impact of nocturia on patients' health-related quality of life and healthcare resource utilisation compared with OAB and BPH: Results from an observational survey in European and American patients. International Journal of Clinical Practice. 2019 doi: 10.1111/ijcp.13408.
    1. Booth JM, Lawrence M, O'Neill K, McMillan L. Exploring older peoples' experiences of nocturia: A poorly recognised urinary condition that limits participation. Disability and Rehabilitation. 2010;32(9):765–774. doi: 10.3109/09638280903295425.
    1. Andersson F, Anderson P, Holm-Larsen T, Piercy J, Everaert K, Holbrook T. Assessing the impact of nocturia on health-related quality-of-life and utility: Results of an observational survey in adults. Journal of Medical Economics. 2016;19(12):1200–1206. doi: 10.1080/13696998.2016.1211136.
    1. Trigg A, Andersson FL, Aldhouse NVJ, Bliwise DL, Kitchen H. Patients' lived experiences of nocturia: A qualitative study of the evening, the night, and the next day. Patient. 2017;10(6):711–718.
    1. Weidlich D, Andersson FL, Oelke M, Drake MJ, Jonasson AF, Guest JF. Annual direct and indirect costs attributable to nocturia in Germany, Sweden, and the UK. European Journal of Health Economics. 2017;18(6):761–771. doi: 10.1007/s10198-016-0826-x.
    1. Hafner, M., Pollard, J., Troxel, W. M., Yerushalmi, E., Fays, C., Whitmore, M., & Van Stolk, C. (2019). How frequent night-time bathroom visits can negatively impact sleep, well-being and productivity: Examining the associations between nocturia, well-being and economic outcomes in a working-age population. RAND Corporation. Retrieved March 31, 2021, from
    1. Holm-Larsen T, Albei C, Andersson F, Norgaard J. My sleep pattern is a series of naps. European Urology Supplements. 2013;12:e405. doi: 10.1016/S1569-9056(13)60890-8.
    1. Kupelian V, Fitzgerald MP, Kaplan SA, Norgaard JP, Chiu GR, Rosen RC. Association of nocturia and mortality: Results from the Third National Health and Nutrition Examination Survey. Journal of Urology. 2011;185(2):571–577. doi: 10.1016/j.juro.2010.09.108.
    1. Pesonen JS, Vernooij RWM, Cartwright R, Aoki Y, Agarwal A, Mangera A, Markland AD, Tsui JF, Santti H, Griebling TL, Pryalukhin AE, Riikonen J, Tahtinen RM, Vaughan CP, Johnson TM, II, Heels-Ansdell D, Guyatt GH, Tikkinen KAO. The impact of nocturia on falls and fractures: A systematic review and meta-analysis. Journal of Urology. 2020;203(4):674–683. doi: 10.1097/JU.0000000000000459.
    1. Abraham L, Hareendran A, Mills IW, Martin ML, Abrams P, Drake MJ, MacDonagh RP, Noble JG. Development and validation of a quality-of-life measure for men with nocturia. Urology. 2004;63(3):481–486. doi: 10.1016/j.urology.2003.10.019.
    1. Mock LL, Parmelee PA, Kutner N, Scott J, Johnson TM., II Content validation of symptom-specific nocturia quality-of-life instrument developed in men: Issues expressed by women, as well as men. Urology. 2008;72(4):736–742. doi: 10.1016/j.urology.2008.04.014.
    1. Food and Drug Administration. (2009, December). Guidance for industry: Patient-reported outcome measures: Use in medical product development to support labeling claims.
    1. Holm-Larsen T, Andersson F, van der Meulen E, Yankov V, Rosen RC, Nørgaard JP. The Nocturia Impact Diary: A self-reported impact measure to complement the voiding diary. Value in Health. 2014;17(6):696–706. doi: 10.1016/j.jval.2014.06.007.
    1. DAWN Clinical Trial. Retrieved March 31, 2021, from
    1. Everitt BS. Multivariate analysis: The need for data, and other problems. British Journal of Psychiatry. 1975;126:237–240. doi: 10.1192/bjp.126.3.237.
    1. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
    1. Cohen J. A power primer. Psychological Bulletin. 1992;112(1):155–159. doi: 10.1037/0033-2909.112.1.155.
    1. Frost MH, Reeve BB, Liepa AM, Stauffer JW, Hays RD, Mayo FDAP-ROCMG, Mayo/FDA Patient-Reported Outcomes Consensus Meeting Group What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value in Health. 2007;10(Suppl 2):S94–S105. doi: 10.1111/j.1524-4733.2007.00272.x.
    1. Giraudeau B, Mary JY. Planning a reproducibility study: How many subjects and how many replicates per subject for an expected width of the 95 per cent confidence interval of the intraclass correlation coefficient. Statistics in Medicine. 2001;20(21):3205–3214. doi: 10.1002/sim.935.
    1. Shoukri MM, Asyali M, Donner A. Sample size requirements for the design of reliability study: Review and new results. Statistical Methods in Medical Research. 2004;13(4):251–271. doi: 10.1191/0962280204sm365ra.
    1. Viktrup L, Hayes RP, Wang P, Shen W. Construct validation of patient global impression of severity (PGI-S) and improvement (PGI-I) questionnaires in the treatment of men with lower urinary tract symptoms secondary to benign prostatic hyperplasia. BMC Urology. 2012;12:30. doi: 10.1186/1471-2490-12-30.
    1. Fayers P, Machin D. Quality of life: The assessment, analysis and interpretation of patient-reported outcomes. Wiley; 2013.
    1. Browne MW, Cudeck R. Alternative ways of assessing model fit. Sociological Methods and Research. 1992;21(2):230–258. doi: 10.1177/0049124192021002005.
    1. Peugh J, Feldon DF. “How Well Does Your Structural Equation Model Fit Your Data?”: Is Marcoulides and Yuan's equivalence test the answer? CBE Life Sciences Education. 2020;19(3):es5. doi: 10.1187/cbe.20-01-0016.
    1. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334. doi: 10.1007/BF02310555.
    1. Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin. 1979;86(2):420. doi: 10.1037/0033-2909.86.2.420.
    1. Litwin, M. (1995). How to measure survey reliability and validity (7 ed.). Sage Publications.
    1. Vaz S, Falkmer T, Passmore AE, Parsons R, Andreou P. The case for using the repeatability coefficient when calculating test–retest reliability. PLoS ONE. 2013;8(9):e73990. doi: 10.1371/journal.pone.0073990.
    1. Bastien CH, Vallieres A, Morin CM. Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Medicine. 2001;2(4):297–307. doi: 10.1016/S1389-9457(00)00065-4.
    1. Hsu A, Nakagawa S, Walter LC, Van Den Eeden SK, Brown JS, Thom DH, Lee SJ, Huang AJ. The burden of nocturia among middle-aged and older women. Obstetrics and Gynecology. 2015;125(1):35–43. doi: 10.1097/AOG.0000000000000600.
    1. Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Medical Care. 1989;27(3 Suppl):S178–189. doi: 10.1097/00005650-198903001-00015.
    1. Food and Drug Administration. (2019). Discussion Document for Patient-Focused Drug Development Public Workshop on Guidance 4: Incorporating Clinical Outcome Assessments into Endpoints for Regulatory Decision-Making. Workshop date: December 6, 2019. Retrieved March 31, 2021, from .
    1. Coon CD, Cook KF. Moving from significance to real-world meaning: Methods for interpreting change in clinical outcome assessment scores. Quality of Life Research. 2018;27(1):33–40. doi: 10.1007/s11136-017-1616-3.
    1. Guyatt GH, Osoba D, Wu AW, Wyrwich KW, Norman GR, Clinical Significance Consensus Meeting Group Methods to explain the clinical significance of health status measures. Mayo Clinic Proceedings. 2002;77(4):371–383. doi: 10.4065/77.4.371.
    1. Revicki DA, Erickson PA, Sloan JA, Dueck A, Guess H, Santanello NC, Mayo/FDA Patient Reported Outcomes Consensus Meeting Group Interpreting and reporting results based on patient-reported outcomes. Value in Health. 2007;10(Suppl 2):S116–124. doi: 10.1111/j.1524-4733.2007.00274.x.
    1. Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. Journal of Clinical Epidemiology. 2008;61(2):102–109. doi: 10.1016/j.jclinepi.2007.03.012.
    1. Hedges L, Olkin I. Statistical methods for meta-analysis. Academic; 1985.
    1. Cohen J. Statistical power analysis for the social sciences. Routledge; 1988.
    1. Nunnally, J., & Bernstein, I. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
    1. Cole DA, Ciesla JA, Steiger JH. The insidious effects of failing to include design-driven correlated residuals in latent-variable covariance structure analysis. Psychological Methods. 2007;12(4):381–398. doi: 10.1037/1082-989X.12.4.381.
    1. Gerbing DW, Anderson JC. On the meaning of within-factor correlated measurement errors. Journal of Consumer Research. 1984;11(1):572–580. doi: 10.1086/208993.
    1. Whittakr T. Using the modification index and standardized expected parameter change for model modification. Journal of Experimental Education. 2012;80(1):26–44. doi: 10.1080/00220973.2010.531299.
    1. MacCallum RC, Roznowski M, Necowitz LB. Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin. 1992;111(3):490–504. doi: 10.1037/0033-2909.111.3.490.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe