Development of a theory-informed questionnaire to assess the acceptability of healthcare interventions

Mandeep Sekhon, Martin Cartwright, Jill J Francis, Mandeep Sekhon, Martin Cartwright, Jill J Francis

Abstract

Background: The theoretical framework of acceptability (TFA) was developed in response to recommendations that acceptability should be assessed in the design, evaluation and implementation phases of healthcare interventions. The TFA consists of seven component constructs (affective attitude, burden, ethicality, intervention coherence, opportunity costs, perceived effectiveness, and self-efficacy) that can help to identify characteristics of interventions that may be improved. The aim of this study was to develop a generic TFA questionnaire that can be adapted to assess acceptability of any healthcare intervention.

Methods: Two intervention-specific acceptability questionnaires based on the TFA were developed using a 5-step pre-validation method for developing patient-reported outcome instruments: 1) item generation; 2) item de-duplication; 3) item reduction and creation; 4) assessment of discriminant content validity against a pre-specified framework (TFA); 5) feedback from key stakeholders. Next, a generic TFA-based questionnaire was developed and applied to assess prospective and retrospective acceptability of the COVID-19 vaccine. A think-aloud method was employed with two samples: 10 participants who self-reported intention to have the COVID-19 vaccine, and 10 participants who self-reported receiving a first dose of the vaccine.

Results: 1) The item pool contained 138 items, identified from primary papers included in an overview of reviews. 2) There were no duplicate items. 3) 107 items were discarded; 35 new items were created to maximise coverage of the seven TFA constructs. 4) 33 items met criteria for discriminant content validity and were reduced to two intervention-specific acceptability questionnaires, each with eight items. 5) Feedback from key stakeholders resulted in refinement of item wording, which was then adapted to develop a generic TFA-based questionnaire. For prospective and retrospective versions of the questionnaire, no participants identified problems with understanding and answering items reflecting four TFA constructs: affective attitude, burden, perceived effectiveness, opportunity costs. Some participants encountered problems with items reflecting three constructs: ethicality, intervention coherence, self-efficacy.

Conclusions: A generic questionnaire for assessing intervention acceptability from the perspectives of intervention recipients was developed using methods for creating participant-reported outcome measures, informed by theory, previous research, and stakeholder input. The questionnaire provides researchers with an adaptable tool to measure acceptability across a range of healthcare interventions.

Keywords: Acceptability; Healthcare intervention; Pre-validation methods; Questionnaire development; Theoretical framework.

Conflict of interest statement

We declare that the authors have no competing interests as defined by BMC, or other interests that might be perceived to influence the results and/or discussion reported in this paper.

© 2022. The Author(s).

Figures

Fig. 1
Fig. 1
Adapted 5 step PRO methodology Flowchart applied to test content validity of the theoretical framework and to develop the patient and HCP acceptability questionnaires

References

    1. Haynes B. Can it work? Does it work? Is it worth it?: the testing of healthcare interventions is evolving. BMJ. Br Med J. 1999;319:652. doi: 10.1136/bmj.319.7211.652.
    1. Say RE, Thomson R. The importance of patient preferences in treatment decisions—challenges for doctors. Bmj. 2003;327:542–545. doi: 10.1136/bmj.327.7414.542.
    1. Sidani SaB CJ. Testing the acceptability and feasibility of interventions. Design, evaluation, and translation of nursing interventions. West Sussex: Wiley; 2011.
    1. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Bmj. 2008;337:a1655. doi: 10.1136/bmj.a1655.
    1. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. Bmj. 2015;350:h1258. doi: 10.1136/bmj.h1258.
    1. Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, Boyd KA, Craig N, French DP, McIntosh E, Petticrew M. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. Bmj. 2021;30:374.
    1. Sekhon M, Cartwright M, Francis JJ. Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC Health Serv Res. 2017;17:88. doi: 10.1186/s12913-017-2031-8.
    1. Arrowsmith ME, Aicken C, Saxena S, Majeed A. Strategies for improving the acceptability and acceptance of the copper intrauterine device. Cochrane Database Syst Rev. 2012;3.
    1. Berlim MT, McGirr A, Van den Eynde F, Fleck MP, Giacobbe P. Effectiveness and acceptability of deep brain stimulation (DBS) of the subgenual cingulate cortex for treatment-resistant depression: a systematic review and exploratory meta-analysis. J Affect Disord. 2014;159:31–38. doi: 10.1016/j.jad.2014.02.016.
    1. Cipriani A, Furukawa TA, Salanti G, Geddes JR, Higgins JP, Churchill R, et al. Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta analysis. Lancet. 2009;373:746–758. doi: 10.1016/S0140-6736(09)60046-5.
    1. Gillies K, Bower P, Elliott J, MacLennan G, Newlands RS, Ogden M, et al. Systematic techniques to enhance rEtention in randomised controlled trials: the STEER study protocol. Trials. 2018;19:197. doi: 10.1186/s13063-018-2572-0.
    1. Hughes-Morley A, Young B, Hempel RJ, Russell IT, Waheed W, Bower P. What can we learn from trial decliners about improving recruitment? Qualitative study. Trials. 2016;17:494. doi: 10.1186/s13063-016-1626-4.
    1. Koller I, Levenson MR, Glück J. What do you think you are measuring? A mixed-methods procedure for assessing the content validity of test items and theory-based scaling. Front Psychol. 2017;8:126. doi: 10.3389/fpsyg.2017.00126.
    1. Sekhon M, Cartwright M, Francis JJ. Acceptability of health care interventions: A theoretical framework and proposed research agenda. Br J Health Psychol. 2018;23:519-31. 10.1111/bjhp.12295.
    1. Wiering B, de Boer D, Delnoij D. Patient involvement in the development of patient-reported outcome measures: a scoping review. Health Expect. 2017;20(1):11–23. doi: 10.1111/hex.12442.
    1. Black N. Patient reported outcome measures could help transform health care. BMJ. 2013;346:f167. doi: 10.1136/bmj.f167.
    1. Renko E, Knittle K, Palsola M, Lintunen T, Hankonen N. Acceptability, reach and implementation of a training to enhance teachers’ skills in physical activity promotion. BMC Public Health. 2020;20(1):1–3. doi: 10.1186/s12889-020-09653-x.
    1. Keyworth C, O'Connor R, Quinlivan L, Armitage CJ. Acceptability of a brief web-based theory-based intervention to prevent and reduce self-harm: mixed methods evaluation. J Med Internet Res. 2021;23(9):e28349. doi: 10.2196/28349.
    1. Koller I, Levenson MR, Glück J. What Do You Think You Are Measuring? A Mixed-Methods.
    1. Prior ME, Hamzah JC, Francis JJ, Ramsay CR, Castillo MM, Campbell SE, et al. Pre-validation methods for developing a patient reported outcome instrument. BMC Med Res Methodol. 2011;11:112. doi: 10.1186/1471-2288-11-112.
    1. Turner RR, Quittner AL, Parasuraman BM, Kallich JD, Cleeland CS, Group MFPROCM Patientreported outcomes: instrument development and selection issues. Value Health. 2007;10:S86. doi: 10.1111/j.1524-4733.2007.00271.x.
    1. Johnston M, Dixon D, Hart J, Glidewell L, Schröder C, Pollard B. Discriminant content validity: a quantitative methodology for assessing content of theory-based measures, with illustrative applications. Br J Health Psychol. 2014;19:240–257. doi: 10.1111/bjhp.12095.
    1. Gould NJ, Lorencatto F, Stanworth SJ, Michie S, Prior ME, Glidewell L, et al. Application of theory to enhance audit and feedback interventions to increase the uptake of evidence-based transfusion practice: an intervention development protocol. Implement Sci. 2014;9:92. doi: 10.1186/s13012-014-0092-1.
    1. Hartley S, Foy R, Walwyn RE, Cicero R, Farrin AJ, Francis JJ, et al. The evaluation of enhanced feedback interventions to reduce unnecessary blood transfusions (AFFINITIE): protocol for two linked cluster randomised factorial controlled trials. Implement Sci. 2017;12:84. doi: 10.1186/s13012-017-0614-8.
    1. Lorencatto F, Gould NJ, McIntyre SA, During C, Bird J, Walwyn R, et al. A multidimensional approach to assessing intervention fidelity in a process evaluation of audit and feedback interventions to reduce unnecessary blood transfusions: a study protocol. Implement Sci. 2016;11:163. doi: 10.1186/s13012-016-0528-x.
    1. Wickwar S, McBain H, Newman SP, Hirani SP, Hurt C, Dunlop N, et al. Effectiveness and cost effectiveness of a patient-initiated botulinum toxin treatment model for blepharospasm and hemifacial spasm compared to standard care: study protocol for a randomised controlled trial. Trials. 2016;17:129. doi: 10.1186/s13063-016-1263-y.
    1. Kulier R, Helmerhorst FM, Maitra N, Gülmezoglu AM. Effectiveness and acceptability of progestogens in combined oral contraceptives–a systematic review. Reproductive health. 2004;1:1.
    1. Caldeira D, Goncalves N, Ferreira JJ, Pinto FJ, Costa J. Tolerability and acceptability of non-vitamin K antagonist oral anticoagulants in atrial fibrillation: systematic review and meta-analysis. Am J Cardiovascular Drugs. 2015;15(4):259-65.
    1. Glenton C, Khanna R, Morgan C, Nilsen ES. The effects, safety and acceptability of compact, pre-filled, autodisable injection devices when delivered by lay health workers. Tropical Med Int Health. 2013;18:1002-16.
    1. Dixon D, Johnston M, McQueen M. The Disabilities of the Arm, Shoulder and Hand Questionnaire (DASH) can measure the impairment, activity limitations and participation restriction constructs from the International Classification of Functioning, Disability and Health (ICF). BMC musculoskeletal disorders. 2008;9:114.
    1. Lynn MR. Determination and quantification of content validity. Nursing research. 1986.
    1. Waltz CF, Bausell RB. Nursing research: Design, statistics, and computer analysis. FA Davis company; 1981.
    1. French DP, Cooke R, Mclean N, Williams M, Sutton S. What do people think about when they answer Theory of Planned Behaviour questionnaires? A think aloud'study. Journal of health psychology. 2007;12(4):672-87.
    1. Green C, Gilhooly K. Protocol analysis: Practical implementation. Handbook of qualitative research methods for psychology and the social sciences. 1996:55-74.
    1. Cohen J. Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychological bulletin. 1968;70(4):213.
    1. Stemler SE. A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Practical Assessment, Research, and Evaluation. 2004;9(1):4.
    1. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PloS one. 2010;5:e13196.
    1. Blenkinsopp A, Hassey A. Effectiveness and acceptability of community pharmacy-based interventions in type 2 diabetes: a critical review of intervention design, pharmacist and patient perspectives. International Journal of Pharmacy Practice. 2005;13:231–40. doi: 10.1211/ijpp.13.4.0001.
    1. Kedge EM. A systematic review to investigate the effectiveness and acceptability of interventions for moist desquamation in radiotherapy patients. Radiography. 2009;15:247–57. doi: 10.1016/j.radi.2008.08.002.
    1. Halpern V, Lopez LM, Grimes DA, Stockton LL, Gallo MF. Strategies to improve adherence and acceptability of hormonal methods of contraception. Cochrane database of systematic reviews. 2013(10).
    1. Kaltenthaler E, Sutcliffe P, Parry G, Beverley C, Rees A, Ferriter M. The acceptability to patients of computerized cognitive behaviour therapy for depression: a systematic review. Psychological Med. 2008;38:1521-30.
    1. Koesters M, Guaiana G, Cipriani A, Becker T, Barbui C. Agomelatine efficacy and acceptability revisited: systematic review and meta-analysis of published and unpublished randomised trials. The British Journal of Psychiatry. 2013;203:179-87.
    1. Muftin Z, Thompson AR. A systematic review of self-help for disfigurement: Effectiveness, usability, and acceptability. Body Image. 2013;10:442-50.
    1. Robinson L, Hutchings D, Dickinson H, Corner L, Beyer F, Finch T, et al. Effectiveness and acceptability of nonpharmacological interventions to reduce wandering in dementia: a systematic review. International Journal of Geriatric Psychiatry: A journal of the psychiatry of late life and allied sciences. 2007;22:9-22.
    1. Newman PA, Logie C. HIV vaccine acceptability: a systematic review and meta-analysis. Aids. 2010;24:1749–56. doi: 10.1097/QAD.0b013e32833adbe8.
    1. Diepeveen S, Ling T, Suhrcke M, Roland M, Marteau TM. Public acceptability of government intervention to change healthrelated behaviours: a systematic review and narrative synthesis. BMC Public Health. 2013;13:756.

Source: PubMed

Sponsorzy i współpracownicy

Warunki medyczne

Interwencje lekowe

3
Subskrybuj