Use of Human-Centered Design to Improve Implementation of Evidence-Based Psychotherapies in Low-Resource Communities: Protocol for Studies Applying a Framework to Assess Usability

Aaron R Lyon, Sean A Munson, Brenna N Renn, David C Atkins, Michael D Pullmann, Emily Friedman, Patricia A Areán, Aaron R Lyon, Sean A Munson, Brenna N Renn, David C Atkins, Michael D Pullmann, Emily Friedman, Patricia A Areán

Abstract

Background: This paper presents the protocol for the National Institute of Mental Health (NIMH)-funded University of Washington's ALACRITY (Advanced Laboratories for Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness) Center (UWAC), which uses human-centered design (HCD) methods to improve the implementation of evidence-based psychosocial interventions (EBPIs). We propose that usability-the degree to which interventions and implementation strategies can be used with ease, efficiency, effectiveness, and satisfaction-is a fundamental, yet poorly understood determinant of implementation.

Objective: We present a novel Discover, Design/Build, and Test (DDBT) framework to study usability as an implementation determinant. DDBT will be applied across Center projects to develop scalable and efficient implementation strategies (eg, training tools), modify existing EBPIs to enhance usability, and create usable and nonburdensome decision support tools for quality delivery of EBPIs.

Methods: Stakeholder participants will be implementation practitioners/intermediaries, mental health clinicians, and patients with mental illness in nonspecialty mental health settings in underresourced communities. Three preplanned projects and 12 pilot studies will employ the DDBT model to (1) identify usability challenges in implementing EBPIs in underresourced settings; (2) iteratively design solutions to overcome these challenges; and (3) compare the solution to the original version of the EPBI or implementation strategy on usability, quality of care, and patient-reported outcomes. The final products from the center will be a streamlined modification and redesign model that will improve the usability of EBPIs and implementation strategies (eg, tools to support EBPI education and decision making); a matrix of modification targets (ie, usability issues) that are both common and unique to EBPIs, strategies, settings, and patient populations; and a compilation of redesign strategies and the relative effectiveness of the redesigned solution compared to the original EBPI or strategy.

Results: The UWAC received institutional review board approval for the three separate studies in March 2018 and was funded in May 2018.

Conclusions: The outcomes from this center will inform the implementation of EBPIs by identifying cross-cutting features of EBPIs and implementation strategies that influence the use and acceptability of these interventions, actively involving stakeholder clinicians and implementation practitioners in the design of the EBPI modification or implementation strategy solution and identifying the impact of HCD-informed modifications and solutions on intervention effectiveness and quality.

Trial registration: ClinicalTrials.gov NCT03515226 (https://ichgcp.net/clinical-trials-registry/NCT03515226), NCT03514394 (https://ichgcp.net/clinical-trials-registry/NCT03514394), and NCT03516513 (https://ichgcp.net/clinical-trials-registry/NCT03516513).

International registered report identifier (irrid): DERR1-10.2196/14990.

Keywords: evidence-based psychosocial interventions; human-centered design; implementation science.

Conflict of interest statement

Conflicts of Interest: DCA is a cofounder with equity stake in a technology company, Lyssn.io, focused on tools to support training, supervision, and quality assurance of psychotherapy and counseling.

©Aaron R Nikole Lyon, Sean A Munson, Brenna N Renn, David C Atkins, Michael D Pullmann, Emily Friedman, Patricia A Areán. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 09.10.2019.

Figures

Figure 1
Figure 1
Discover, design/build, and test (DDBT) framework. EBPI: evidence-based psychosocial interventions.

References

    1. Raue PJ, Weinberger MI, Sirey JA, Meyers BS, Bruce ML. Preferences for depression treatment among elderly home health care patients. Psychiatr Serv. 2011 May;62(5):532–7. doi: 10.1176/ps.62.5.pss6205_0532.
    1. Arean PA, Raue PJ, Sirey JA, Snowden M. Implementing evidence-based psychotherapies in settings serving older adults: challenges and solutions. Psychiatr Serv. 2012 Jun;63(6):605–7. doi: 10.1176/appi.ps.201100078.
    1. Quiñones Ana R, Thielke SM, Beaver KA, Trivedi RB, Williams EC, Fan VS. Racial and ethnic differences in receipt of antidepressants and psychotherapy by veterans with chronic depression. Psychiatr Serv. 2014 Feb 01;65(2):193–200. doi: 10.1176/appi.ps.201300057.
    1. Fernandez Y Garcia Erik, Franks P, Jerant A, Bell RA, Kravitz RL. Depression treatment preferences of Hispanic individuals: exploring the influence of ethnicity, language, and explanatory models. J Am Board Fam Med. 2011 Jan 05;24(1):39–50. doi: 10.3122/jabfm.2011.01.100118.
    1. McHugh RK, Whitton SW, Peckham AD, Welge JA, Otto MW. Patient preference for psychological vs pharmacologic treatment of psychiatric disorders. J Clin Psychiatry. 2013 Jun 15;74(06):595–602. doi: 10.4088/jcp.12r07757.
    1. Houle J, Villaggi B, Beaulieu M, Lespérance François, Rondeau G, Lambert J. Treatment preferences in patients with first episode depression. J Affect Disord. 2013 May;147(1-3):94–100. doi: 10.1016/j.jad.2012.10.016.
    1. Dwight Johnson M, Apesoa-Varano C, Hay J, Unutzer J, Hinton L. Depression treatment preferences of older white and Mexican origin men. Gen Hosp Psychiatry. 2013 Jan;35(1):59–65. doi: 10.1016/j.genhosppsych.2012.08.003.
    1. Linden M, Golier J. Challenges in the implementation of manualized psychotherapy in combat-related PTSD. CNS Spectr. 2008 Oct 07;13(10):872–80. doi: 10.1017/s1092852900016977.
    1. Chambers DA. Adm Policy Ment Health. USA: Springer; 2008. Mar 27, Advancing the science of implementation: a workshop summary; pp. 3–10.
    1. Goldman HH, Azrin ST. Public policy and evidence-based practice. Psychiatric Clinics of North America. 2003 Dec;26(4):899–917. doi: 10.1016/s0193-953x(03)00068-6.
    1. Institute of Medicine . Psychosocial Interventions For Mental And Substance Use Disorders: A Framework For Establishing Evidence-based Standards. Washington, DC: National Academies Press; 2015.
    1. Olfson M, Kroenke K, Wang S, Blanco C. Trends in office-based mental health care provided by psychiatrists and primary care physicians. J Clin Psychiatry. 2014 Mar 15;75(03):247–253. doi: 10.4088/jcp.13m08834.
    1. Farmer EMZ, Burns BJ, Phillips SD, Angold A, Costello EJ. Pathways into and through mental health services for children and adolescents. Psychiatr Serv. 2003 Jan;54(1):60–6. doi: 10.1176/appi.ps.54.1.60.
    1. Merikangas KR, He J, Burstein M, Swendsen J, Avenevoli S, Case B, Georgiades K, Heaton L, Swanson S, Olfson M. Service utilization for lifetime mental disorders in U.S. adolescents: results of the National Comorbidity Survey-Adolescent Supplement (NCS-A) J Am Acad Child Adolesc Psychiatry. 2011 Jan;50(1):32–45. doi: 10.1016/j.jaac.2010.10.006.
    1. Lyon AR, Ludwig K, Romano E, Koltracht J, Vander Stoep A, McCauley E. Using modular psychotherapy in school mental health: provider perspectives on intervention-setting fit. J Clin Child Adolesc Psychol. 2014 Oct 17;43(6):890–901. doi: 10.1080/15374416.2013.843460.
    1. Stewart RE, Chambless DL, Stirman SW. Decision making and the use of evidence-based practice: Is the three-legged stool balanced? Practice Innovations. 2018 Mar;3(1):56–67. doi: 10.1037/pri0000063.
    1. Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, Baker R, Eccles MP. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013 Mar 23;8(1):35. doi: 10.1186/1748-5908-8-35.
    1. Lewis CC, Klasnja P, Powell BJ, Lyon AR, Tuzzio L, Jones S, Walsh-Bailey C, Weiner B. From classification to causality: Advancing understanding of mechanisms of change in implementation science. Front Public Health. 2018 May 7;6:136. doi: 10.3389/fpubh.2018.00136. doi: 10.3389/fpubh.2018.00136.
    1. Cook CR, Lyon AR, Locke J, Waltz T, Powell BJ. Adapting a compilation of implementation strategies to advance school-based implementation research and practice. Prev Sci. 2019 Aug 31;20(6):914–935. doi: 10.1007/s11121-019-01017-1.
    1. Leeman J, Birken SA, Powell BJ, Rohweder C, Shea CM. Beyond "implementation strategies": classifying the full range of strategies used in implementation science and practice. Implement Sci. 2017 Nov 03;12(1):125. doi: 10.1186/s13012-017-0657-x.
    1. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, Proctor EK, Kirchner JE. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015 Feb 12;10(1):21. doi: 10.1186/s13012-015-0209-1.
    1. Lyon AR, Koerner K. User-centered design for psychosocial intervention development and implementation. Clin Psychol Sci Pract. 2016 Jun 17;23(2):180–200. doi: 10.1111/cpsp.12154.
    1. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implementation Sci. 2013 Dec 1;8(1) doi: 10.1186/1748-5908-8-139.
    1. International Organization for Standardization ISO: International Organization for Standardization. [2019-09-10]. ISO 9241-420:2011 - Ergonomics of human-system interaction — Part 420: Selection of physical input devices .
    1. Lyon AR, Bruns EJ. User-centered redesign of evidence-based psychosocial interventions to enhance implementation—hospitable soil or better seeds? JAMA Psychiatry. 2019 Jan 01;76(1):3. doi: 10.1001/jamapsychiatry.2018.3060.
    1. Alexopoulos GS, Raue PJ, Kiosses DN, Seirup JK, Banerjee S, Arean PA. Comparing engage with PST in late-life major depression: a preliminary report. Am J Geriatr Psychiatry. 2015 May;23(5):506–13. doi: 10.1016/j.jagp.2014.06.008.
    1. Sburlati ES, Schniering CA, Lyneham HJ, Rapee RM. A model of therapist competencies for the empirically supported cognitive behavioral treatment of child and adolescent anxiety and depressive disorders. Clin Child Fam Psychol Rev. 2011 Mar 4;14(1):89–109. doi: 10.1007/s10567-011-0083-6.
    1. Gehart D. The core competencies and MFT education: practical aspects of transitioning to a learning-centered, outcome-based pedagogy. J Marital Fam Ther. 2011 Jul;37(3):344–54. doi: 10.1111/j.1752-0606.2010.00205.x.
    1. Crabb R, Areán Patricia A, Hegel M. Sustained adoption of an evidence-based treatment: a survey of clinicians certified in problem-solving therapy. Depress Res Treat. 2012;2012:986547. doi: 10.1155/2012/986547. doi: 10.1155/2012/986547.
    1. Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implementation Sci. 2015 Nov 4;10(1) doi: 10.1186/s13012-015-0342-x.
    1. Dopp A, Parisi K, Munson S, Lyon A. A glossary of user-centered design strategies for implementation experts. Transl Behav Med. 2018 Dec 07; doi: 10.1093/tbm/iby119.
    1. Lyon AR, Comtois KA, Kerns SEU, Landes SJ, Lewis CC. Closing the science-practice gap in implementation before it opens. In: Mildon R, Albers B, Shlonsky A, editors. Effective Implementation Science. New York: Springer International Publishing; 2020.
    1. Norman D, Draper S. User centered system design: new perspectives on human-computer interaction. Hillsdale, N.J. Hillsdale, NJ: L Erlbaum Associates; 1986.
    1. Lyon AR, Koerner K, Chung J. How implementable is that evidence-based practice? A methodology for assessing complex innovation usability. Proceedings from the 11th Annual Conference on the Science of Dissemination and Implementation. Washington, DC, USA. Implementation Science. 2019 Mar 25;14(1):878. doi: 10.1186/s13012-019-0878-2.
    1. Alexopoulos GS, Raue PJ, Gunning F, Kiosses DN, Kanellopoulos D, Pollari C, Banerjee S, Arean PA. "Engage" therapy: Behavioral activation and improvement of late-life major depression. Am J Geriatr Psychiatry. 2016 Apr;24(4):320–6. doi: 10.1016/j.jagp.2015.11.006.
    1. Lyon A, Coifman J, Cook H, Liu F, Ludwig K, Dorsey S, Koerner K, Munson S, McCauley E. The Cognitive Walk-Through for Implementation Strategies (CWIS): A pragmatic methodology for assessing strategy usability. 11th Annual Conference on the Science of Dissemination and Implementation; 2018; Washington, DC. 2018. p. A.
    1. Mohr DC, Lyon AR, Lattie EG, Reddy M, Schueller SM. Accelerating digital mental health research from early design and creation to successful implementation and sustainment. J Med Internet Res. 2017 May 10;19(5):e153. doi: 10.2196/jmir.7725.
    1. Proctor E, Hooley C, Morse A, McCrary S, Kim H, Kohl PL. Intermediary/purveyor organizations for evidence-based interventions in the US child mental health: characteristics and implementation strategies. Implementation Sci. 2019 Jan 14;14(1) doi: 10.1186/s13012-018-0845-3.
    1. ISO: International Standards Organization. 2019. [2019-09-19]. ISO 9241-210:2019 Ergonomics of human-system interaction — Part 210: Human-centred design for interactive systems .
    1. Chorpita Bruce F, Becker Kimberly D, Daleiden Eric L. Understanding the common elements of evidence-based practice: misconceptions and clinical examples. J Am Acad Child Adolesc Psychiatry. 2007 May;46(5):647–52. doi: 10.1097/chi.0b013e318033ff71.
    1. Lyon AR, Stanick C, Pullmann MD. Toward high‐fidelity treatment as usual: Evidence‐based intervention structures to improve usual care psychotherapy. Clin Psychol Sci Pract. 2018 Dec 12;25(4):e12265. doi: 10.1111/cpsp.12265.
    1. Meza RD, Jungbluth N, Sedlar G, Martin P, Berliner L, Wiltsey-Stirman S, Dorsey S. Clinician-reported modification to a CBT approach in children’s mental health. Journal of Emotional and Behavioral Disorders. 2019 Feb 13;:106342661982836. doi: 10.1177/1063426619828369.
    1. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011 Mar 19;38(2):65–76. doi: 10.1007/s10488-010-0319-7.
    1. Dwight-Johnson M, Aisenberg E, Golinelli D, Hong S, O'Brien M, Ludman E. Telephone-based cognitive-behavioral therapy for latino patients living in rural areas: A randomized pilot study. Psychiatric Services. 2011 Aug 01;62(8) doi: 10.1176/appi.ps.62.8.936.
    1. Nezu AM. Efficacy of a social problem-solving therapy approach for unipolar depression. Journal of Consulting and Clinical Psychology. 1986;54(2):196–202. doi: 10.1037//0022-006x.54.2.196.
    1. Resick PA, Schnicke MK. Cognitive processing therapy for sexual assault victims. Journal of Consulting and Clinical Psychology. 1992;60(5):748–756. doi: 10.1037//0022-006x.60.5.748.
    1. Holtzblatt KW, Wood S. Rapid Contextual Design: A How-to Guide to Key Techniques for User-Centered Design. San Francisco, CA: Morgan Kaufmann; 2004.
    1. Maguire M. Methods to support human-centred design. International Journal of Human-Computer Studies. 2001 Oct;55(4):587–634. doi: 10.1006/ijhc.2001.0503.
    1. Rubin J, Chisnell D. Handbook of usability testing: how to plan, design, and conduct effective tests. 2nd ed. Indianapolis, IN: Wiley; 2008.
    1. Albert W, Tullis T. Measuring The User Experience: Collecting, Analyzing, And Presenting Usability Metrics (interactive Technologies) Waltham, MA: Morgan Kaufmann; 2019.
    1. Beyer H, Holtzblatt K. Contextual Design: Defining Customer-centered Systems (interactive Technologies) Waltham, MA: Morgan Kaufmann; 1998.
    1. Gould JD, Lewis C. Designing for usability: key principles and what designers think. Commun ACM. 1985;28(3):300–311. doi: 10.1145/3166.3170.
    1. Cooper A, Reimann R, Cronin D. About Face 3: The Essentials Of Interaction Design. Indianapolis, IN: Wiley; 2019.
    1. Grudin J, Pruitt J. Personas, Participatory Design and Product Development: An Infrastructure for Engagement. Proceedings of Participation and Design Conference; 2002; Sweden. 2002.
    1. Kujala S, Mäntylä M. How effective are user studies? In: McDonald S, Waern Y, Cockton G, editors. People and computers XIV - usability or else! London: Springer-Verlag; 2000. pp. 61–71.
    1. Kujala S, Kauppinen M. Identifying and Selecting Users for User-Centered Design. Nordic Conference on Computer Human Interaction; 2004; Tampere, Finland. 2004.
    1. Franks RP, Bory CT. Who supports the successful implementation and sustainability of evidence-based practices? Defining and understanding the roles of intermediary and purveyor organizations. New Dir Child Adolesc Dev. 2015 Sep 16;2015(149):41–56. doi: 10.1002/cad.20112.
    1. Cooper A, Reimann R, Cronin D. About face 3: The essentials of interaction design. Indianapolis, IN: Wiley Pub; 2007.
    1. Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation research. Adm Policy Ment Health. 2010 Oct 22;38(1):44–53. doi: 10.1007/s10488-010-0314-z.
    1. Tornatzky LG, Klein KJ. Innovation characteristics and innovation adoption-implementation: A meta-analysis of findings. IEEE Trans Eng Manage. 1982 Feb;EM-29(1):28–45. doi: 10.1109/tem.1982.6447463.
    1. Wei J, Salvendy G. The cognitive task analysis methods for job and task design: review and reappraisal. Behaviour & Information Technology. 2004 Jul;23(4):273–299. doi: 10.1080/01449290410001673036.
    1. Bligård L, Osvalder A. Enhanced cognitive walkthrough: Development of the cognitive walkthrough method to better predict, identify, and present usability problems. Advances in Human-Computer Interaction. 2013;2013:1–17. doi: 10.1155/2013/931698.
    1. Boren T, Ramey J. Thinking aloud: reconciling theory and practice. IEEE Trans Profess Commun. 2000;43(3):261–278. doi: 10.1109/47.867942.
    1. Jääskeläinen R. Think aloud protocol. In: Gambier Y, Doorslaer L, editors. Handbook of Translation Studies: Volume 1. Amsterdam, Netherlands: John Benjamins Publishing Company; 2010. pp. 371–373.
    1. Wilson J, Rosenberg D. Rapid prototyping for user interface design. In: Helander M, editor. Handbook of Human-Computer Interaction. Amsterdam, Netherlands: North-Holland; 1988. pp. 859–875.
    1. Nielsen J, Landauer T. A mathematical model of the finding of usability problems. Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems; 1993; Amsterdam, Netherlands. 1993. pp. 206–213.
    1. Hwang W, Salvendy G. Number of people required for usability evaluation. Commun ACM. 2010 May 01;53(5):130. doi: 10.1145/1735223.1735255.
    1. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs. Medical Care. 2012;50(3):217–226. doi: 10.1097/mlr.0b013e3182408812.
    1. Craig SD, Graesser AC, Perez RS. Advances from the Office of Naval Research STEM Grand Challenge: expanding the boundaries of intelligent tutoring systems. IJ STEM Ed. 2018 Apr 12;5(1) doi: 10.1186/s40594-018-0111-x.
    1. Brooke J. SUS: A quick and dirty usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland AL, editors. Usability evaluation in industry. London: Taylor and Francis; 1996.
    1. Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, Boynton MH, Halko H. Psychometric assessment of three newly developed implementation outcome measures. Implementation Sci. 2017 Aug 29;12(1) doi: 10.1186/s13012-017-0635-3.
    1. Vallis TM, Shaw BF, Dobson KS. The Cognitive Therapy Scale: Psychometric properties. Journal of Consulting and Clinical Psychology. 1986;54(3):381–385. doi: 10.1037//0022-006x.54.3.381.
    1. Raghavan R. The role of economic evaluation in disseminationimplementation research. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health: Translating science to practice. New York: Oxford Scholarship; 2012. pp. 94–113.
    1. Sheehan DV, Harnett-Sheehan K, Raj BA. The measurement of disability. International Clinical Psychopharmacology. 1996;11(Supplement 3):89–95. doi: 10.1097/00004850-199606003-00015.
    1. Leon AC, Olfson M, Portera L, Farber L, Sheehan DV. Assessing psychiatric impairment in primary care with the sheehan disability scale. Int J Psychiatry Med. 1997 Jul;27(2):93–105. doi: 10.2190/t8em-c8yh-373n-1uwd.
    1. Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001 Sep;16(9):606–13. doi: 10.1046/j.1525-1497.2001.016009606.x.
    1. Aarons GA, Fettes DL, Hurlburt MS, Palinkas LA, Gunderson L, Willging CE, Chaffin MJ. Collaboration, negotiation, and coalescence for interagency-collaborative teams to scale-up evidence-based practice. J Clin Child Adolesc Psychol. 2014 Mar 10;43(6):915–28. doi: 10.1080/15374416.2013.876642.
    1. Hurlburt M, Aarons GA, Fettes D, Willging C, Gunderson L, Chaffin MJ. Interagency collaborative team model for capacity building to scale-up evidence-based practice. Child Youth Serv Rev. 2014 Apr;39:160–168. doi: 10.1016/j.childyouth.2013.10.005.
    1. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, Mandell DS. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2015 Aug 21;44(2):177–194. doi: 10.1007/s11414-015-9475-6.
    1. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Sci. 2009 Aug 07;4(1) doi: 10.1186/1748-5908-4-50.
    1. Lavery D, Cockton G, Atkinson MP. Comparison of evaluation methods using structured usability problem reports. Behaviour & Information Technology. 1997 Jan;16(4-5):246–266. doi: 10.1080/014492997119824.
    1. Khajouei R, Peute L, Hasman A, Jaspers M. Classification and prioritization of usability problems using an augmented classification scheme. J Biomed Inform. 2011 Dec;44(6):948–57. doi: 10.1016/j.jbi.2011.07.002.
    1. Andre TS, Rex Hartson H, Belz SM, Mccreary FA. The user action framework: a reliable foundation for usability engineering support tools. International Journal of Human-Computer Studies. 2001;54(1):107–136. doi: 10.1006/ijhc.2000.0441.
    1. Wiltsey Stirman S, Baumann AA, Miller CJ. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implementation Sci. 2019 Jun 6;14(1) doi: 10.1186/s13012-019-0898-y.
    1. Faulkner L. Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, & Computers. 2003 Aug;35(3):379–383. doi: 10.3758/bf03195514.
    1. Hsieh H, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2016 Jul;15(9):1277–1288. doi: 10.1177/1049732305276687.
    1. Hill CE, Thompson BJ, Williams EN. A guide to conducting consensual qualitative research. The counseling psychologist. 2016 Jun 30;25(4):517–572. doi: 10.1177/0011000097254001.
    1. Hill CE, Knox S, Thompson BJ, Williams EN, Hess SA, Ladany N. Consensual qualitative research: An update. Journal of Counseling Psychology. 2005 Apr;52(2):196–205. doi: 10.1037/0022-0167.52.2.196.
    1. Colusso L, Bennett CL, Hsieh G, Munson SA. Translational resources: Reducing the gap between academic research and HCI practice. Proceedings of the Conference on Designing Interactive Systems; 2017; Edinburgh, United Kingdom. 2017. pp. 957–968.
    1. Williams NJ. Multilevel mechanisms of implementation strategies in mental health: Integrating theory, research, and practice. Adm Policy Ment Health. 2015 Oct 16;43(5):783–798. doi: 10.1007/s10488-015-0693-2.
    1. Beidas R, Kendall P. Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clin Psychol (New York) 2010 Mar;17(1):1–30. doi: 10.1111/j.1468-2850.2009.01187.x.
    1. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2010 Dec 14;38(1):4–23. doi: 10.1007/s10488-010-0327-7.
    1. Dopp AR, Parisi KE, Munson SA, Lyon AR. Integrating implementation and user-centred design strategies to enhance the impact of health services: protocol from a concept mapping study. Health Res Policy Sys. 2019 Jan 8;17(1) doi: 10.1186/s12961-018-0403-0.
    1. Chambers DA, Norton WE. The adaptome: Advancing the science of intervention adaptation. Am J Prev Med. 2016 Oct;51(4 Suppl 2):S124–31. doi: 10.1016/j.amepre.2016.05.011.

Source: PubMed

Sponsors et collaborateurs

Les conditions médicales

Interventions en matière de drogue

3
S'abonner