Designing clinical practice feedback reports: three steps illustrated in Veterans Health Affairs long-term care facilities and programs

Zach Landis-Lewis, Jennifer Kononowech, Winifred J Scott, Robert V Hogikyan, Joan G Carpenter, V S Periyakoil, Susan C Miller, Cari Levy, Mary Ersek, Anne Sales, Zach Landis-Lewis, Jennifer Kononowech, Winifred J Scott, Robert V Hogikyan, Joan G Carpenter, V S Periyakoil, Susan C Miller, Cari Levy, Mary Ersek, Anne Sales

Abstract

Background: User-centered design (UCD) methods are well-established techniques for creating useful artifacts, but few studies illustrate their application to clinical feedback reports. When used as an implementation strategy, the content of feedback reports depends on a foundational audit process involving performance measures and data, but these important relationships have not been adequately described. Better guidance on UCD methods for designing feedback reports is needed. Our objective is to describe the feedback report design method for refining the content of prototype reports.

Methods: We propose a three-step feedback report design method (refinement of measures, data, and display). The three steps follow dependencies such that refinement of measures can require changes to data, which in turn may require changes to the display. We believe this method can be used effectively with a broad range of UCD techniques.

Results: We illustrate the three-step method as used in implementation of goals of care conversations in long-term care settings in the U.S. Veterans Health Administration. Using iterative usability testing, feedback report content evolved over cycles of the three steps. Following the steps in the proposed method through 12 iterations with 13 participants, we improved the usability of the feedback reports.

Conclusions: UCD methods can improve feedback report content through an iterative process. When designing feedback reports, refining measures, data, and display may enable report designers to improve the user centeredness of feedback reports.

Keywords: Audit and feedback; Clinical quality improvement; Goals of care; Long-term care; User-centered design.

Conflict of interest statement

The authors declare that they have no competing interests.

Figures

Fig. 1
Fig. 1
Three refinement steps in a user-centered design process [14] for feedback reports: refine measures, data, and display
Fig. 2
Fig. 2
Example of a bulletin board that is used to post feedback reports
Fig. 3
Fig. 3
Prototype displays, version 1, that summed only conversations documented within 7 days. CLC, community living center
Fig. 4
Fig. 4
Prototype displays, version 12, that summed conversations ever documented (top) and the timeliness of documentation (bottom). CLC, community living center; LST, life-sustaining treatment
Fig. 5
Fig. 5
Chart from feedback report in June 2019 with refined measures addressing admitting service (short stay vs. long-term care). CLC, community living center; LST, life-sustaining treatment

References

    1. Confidential Physician Feedback Reports: Designing for Optimal Impact on Performance [Internet]. 2016 [cited 2016 May 16]. Available from:
    1. Ivers NM, Sales A, Colquhoun H, Michie S, Foy R, Francis JJ, Grimshaw JM. No more “business as usual” with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci. 2014;9(1):14. doi: 10.1186/1748-5908-9-14.
    1. Brehaut JC, Colquhoun HL, Eva KW, Carroll K, Sales A, Michie S, Ivers N, Grimshaw JM. Practice Feedback Interventions: 15 Suggestions for Optimizing Effectiveness. Ann Intern Med. 2016;164(6):435–441. doi: 10.7326/M15-2248.
    1. Colquhoun H, Michie S, Sales A, Ivers N, Grimshaw JM, Carroll K, Chalifoux M, Eva K, Brehaut J. Reporting and design elements of audit and feedback interventions: a secondary review. BMJ Qual Saf. 2017;26(1):54–60. 10.1136/bmjqs-2015-005004.
    1. Kluger AN, DeNisi A. The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull. 1996;119(2):254–284. doi: 10.1037/0033-2909.119.2.254.
    1. Ilgen DR, Fisher CD, Taylor MS. Consequences of individual feedback on behavior in organizations. J Appl Psychol. 1979;64(4):349–371. doi: 10.1037/0021-9010.64.4.349.
    1. Shaller D, Kanouse D. Private Performance Feedback Reporting for Physicians [Internet]. Rockville, MD: Agency for Healthcare Research and Quality; 2012 Nov [cited 2018 Oct 4]. Available from:
    1. Brown B, Balatsoukas P, Williams R, Sperrin M, Buchan I. Multi-method laboratory user evaluation of an actionable clinical performance information system: implications for usability and patient safety. J Biomed Inform. 2018;77:62–80. doi: 10.1016/j.jbi.2017.11.008.
    1. Part Two: Design of Physician Feedback Reporting Systems | Agency for Healthcare Research & Quality [Internet]. Rockville, MD: Agency for Healthcare Research and Quality; 2016 [cited 2018 Oct 4]. Available from:
    1. Tip 7. Test the Report With Your Audience [Internet]. [cited 2018 Dec 13]. Available from:
    1. Testing Your Report By Getting Feedback From Users [Internet]. [cited 2018 Dec 19]. Available from:
    1. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, Proctor EK, Kirchner JE. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):21. doi: 10.1186/s13012-015-0209-1.
    1. Colquhoun HL, Sattler D, Chan C, Walji T, Palumbo R, Chalmers I, Jokhio I, Ivers NM. Applying user-centered design to develop an audit and feedback intervention for the home care sector. Home Health Care Manag Pract. 2017;29(3):148–160. doi: 10.1177/1084822317700883.
    1. Witteman HO, Dansokho SC, Colquhoun H, Coulter A, Dugas M, Fagerlin A, Giguere AM, Glouberman S, Haslett L, Hoffman A, Ivers N, Légaré F, Légaré J, Levin C, Lopez K, Montori VM, Provencher T, Renaud J-S, Sparling K, Stacey D, Vaisson G, Volk RJ, Witteman W. User-centered design and the development of patient decision aids: protocol for a systematic review. Syst Rev. 2015;4:11. doi: 10.1186/2046-4053-4-11.
    1. Brown B, Balatsoukas P, Williams R, Sperrin M, Buchan I. Interface design recommendations for computerised clinical audit and feedback: Hybrid usability evidence from a research-led system. Int J Med Inf. 2016;94:191–206. doi: 10.1016/j.ijmedinf.2016.07.010.
    1. Sales AE, Ersek M, Intrator OK, Levy C, Carpenter JG, Hogikyan R, Kales HC, Landis-Lewis Z, Olsan T, Miller SC, Montagnini M, Periyakoil VS, Reder S. Implementing goals of care conversations with veterans in VA long-term care setting: a mixed methods protocol. Implement Sci. 2016;11:132. doi: 10.1186/s13012-016-0497-0.
    1. Foglia MB, Lowery J, Sharpe VA, Tompkins P, Fox E. A comprehensive approach to eliciting, documenting, and honoring patient wishes for care near the end of life: the Veterans Health Administration’s Life-Sustaining Treatment Decisions Initiative. Jt Comm J Qual Patient Saf. 2019;45(1):47–56. doi: 10.1016/j.jcjq.2018.04.007.
    1. McGlynn EA, Asch SM. Developing a clinical performance measure. Am J Prev Med. 1998;14(3, Supplement 1):14–21. doi: 10.1016/S0749-3797(97)00032-9.
    1. Campbell SM, Braspenning J, Hutchinson A, Marshall MN. Research methods used in developing and applying quality indicators in primary care. BMJ. 2003;326(7393):816–819. doi: 10.1136/bmj.326.7393.816.
    1. Roberts JP, Fisher TR, Trowbridge MJ, Bent C. A design thinking framework for healthcare management and innovation. Healthcare. 2016;4(1):11–14. doi: 10.1016/j.hjdsi.2015.12.002.
    1. Cohn M. User Stories Applied: For Agile Software Development. 1st ed. Boston: Addison-Wesley Professional; 2004. p. 304.
    1. Panicker V, Lee D, Wetmore M, Rampton J, Smith R, Moniz M, Landis-Lewis Z. Designing tailored displays for clinical practice feedback: developing requirements with user stories. Stud Health Technol Inform. 2019;264:1308–12. 10.3233/SHTI190438.
    1. Few S. Information dashboard design: displaying data for at-a-glance monitoring. Second ed. Burlingame, CA: Analytics Press; 2013. 260 p.
    1. Hegarty M. Advances in Cognitive Science and Information Visualization. In: Score Reporting Research and Applications [Internet]. Routledge; 2018 [cited 2019 Apr 30]. Available from:
    1. Eden AR, Hansen E, Hagen MD, Peterson LE. Physician perceptions of performance feedback in a quality improvement activity. Am J Med Qual. 2018;33(3):283–290. doi: 10.1177/1062860617738327.
    1. Buxton B. Sketching User Experiences: Getting the Design Right and the Right Design. 1st ed. San Francisco, CA: Morgan Kaufmann; 2007. 448 p.
    1. Gothelf J, Seiden J, Lean UX. Designing Great Products with Agile Teams. 2nd ed. O’Reilly Media: Sebastopol, CA; 2016. p. 208.
    1. Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform. 2004;37(1):56–76. doi: 10.1016/j.jbi.2004.01.003.
    1. Munzner T. Visualization Analysis and Design: CRC Press; 2014. p. 422.
    1. Zhang J. A representational analysis of relational information displays. Int J Hum Comput Stud. 1996;45(1):59–74. doi: 10.1006/ijhc.1996.0042.
    1. Vessey I. Cognitive fit: a theory-based analysis of the graphs versus tables literature*. Decis Sci. 1991;22(2):219–40.
    1. Krug S. Rocket surgery made easy: the do-it-yourself guide to finding and fixing usability problems. 1st ed. New Riders: Berkeley, CA; 2009. p. 168.
    1. Fried TR, Tinetti ME, Iannone L, O’Leary JR, Towle V, Van Ness PH. Health outcome prioritization as a tool for decision making among older persons with multiple chronic conditions. Arch Intern Med. 2011;171(20):1854–1856. doi: 10.1001/archinternmed.2011.424.
    1. Grimshaw JM, Ivers N, Linklater S, Foy R, Francis JJ, Gude WT, Hysong SJ. Audit and Feedback MetaLab. Reinvigorating stagnant science: implementation laboratories and a meta-laboratory to efficiently advance the science of audit and feedback. BMJ Qual Saf. 2019;28(5):416–423. doi: 10.1136/bmjqs-2018-008355.
    1. HEDIS Measures and Technical Resources [Internet]. NCQA. [cited 2018 Dec 19]. Available from:
    1. Borycki E, Kushniruk A, Nohr C, Takeda H, Kuwata S, Carvalho C, Bainbridge M, Kannry J. Usability methods for ensuring health information technology safety: evidence-based approaches. Contribution of the IMIA Working Group Health Informatics for Patient Safety. Yearb Med Inform. 2013;8:20–27.

Source: PubMed

Sponsorit ja yhteistyökumppanit

Sairaudet

Huumeiden interventiot

3
Tilaa