A Scalable Service to Improve Health Care Quality Through Precision Audit and Feedback: Proposal for a Randomized Controlled Trial

Zach Landis-Lewis, Allen Flynn, Allison Janda, Nirav Shah, Zach Landis-Lewis, Allen Flynn, Allison Janda, Nirav Shah

Abstract

Background: Health care delivery organizations lack evidence-based strategies for using quality measurement data to improve performance. Audit and feedback (A&F), the delivery of clinical performance summaries to providers, demonstrates the potential for large effects on clinical practice but is currently implemented as a blunt one size fits most intervention. Each provider in a care setting typically receives a performance summary of identical metrics in a common format despite the growing recognition that precisionizing interventions hold significant promise in improving their impact. A precision approach to A&F prioritizes the display of information in a single metric that, for each recipient, carries the highest value for performance improvement, such as when the metric's level drops below a peer benchmark or minimum standard for the first time, thereby revealing an actionable performance gap. Furthermore, precision A&F uses an optimal message format (including framing and visual displays) based on what is known about the recipient and the intended gist meaning being communicated to improve message interpretation while reducing the cognitive processing burden. Well-established psychological principles, frameworks, and theories form a feedback intervention knowledge base to achieve precision A&F. From an informatics perspective, precision A&F requires a knowledge-based system that enables mass customization by representing knowledge configurable at the group and individual levels.

Objective: This study aims to implement and evaluate a demonstration system for precision A&F in anesthesia care and to assess the effect of precision feedback emails on care quality and outcomes in a national quality improvement consortium.

Methods: We propose to achieve our aims by conducting 3 studies: a requirements analysis and preferences elicitation study using human-centered design and conjoint analysis methods, a software service development and implementation study, and a cluster randomized controlled trial of a precision A&F service with a concurrent process evaluation. This study will be conducted with the Multicenter Perioperative Outcomes Group, a national anesthesia quality improvement consortium with >60 member hospitals in >20 US states. This study will extend the Multicenter Perioperative Outcomes Group quality improvement infrastructure by using existing data and performance measurement processes.

Results: The proposal was funded in September 2021 with a 4-year timeline. Data collection for Aim 1 began in March 2022. We plan for a 24-month trial timeline, with the intervention period of the trial beginning in March 2024.

Conclusions: The proposed aims will collectively demonstrate a precision feedback service developed using an open-source technical infrastructure for computable knowledge management. By implementing and evaluating a demonstration system for precision feedback, we create the potential to observe the conditions under which feedback interventions are effective.

International registered report identifier (irrid): PRR1-10.2196/34990.

Keywords: anesthesiology; audit and feedback; human-centered design; knowledge-based system; learning health system.

Conflict of interest statement

Conflicts of Interest: ZLL has received research support, paid to the University of Michigan and related to this work, from the National Library of Medicine (K01 LM012528). AJ has received research support, paid to the University of Michigan and unrelated to this work, from Becton, Dickinson and Company. NS has received research support, paid to University of Michigan and unrelated to this work, from Merck & Co. NS received support, paid to the University of Michigan, for his role as Program Director of Anesthesiology Performance Improvement and Reporting Exchange (ASPIRE) Collaborative Quality Initiative, and has received research support from Edwards Lifesciences, Apple Inc, and National Institute on Aging (R01 AG059607), paid to the University of Michigan and unrelated to this work.

©Zach Landis-Lewis, Allen Flynn, Allison Janda, Nirav Shah. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 10.05.2022.

Figures

Figure 1
Figure 1
A precision feedback service. A&F: audit and feedback.
Figure 2
Figure 2
An example provider feedback email from the Multicenter Perioperative Outcomes Group (MPOG) setting.
Figure 3
Figure 3
A one size fits n spectrum.
Figure 4
Figure 4
A causal pathway model for precision audit and feedback (A&F) interventions.
Figure 5
Figure 5
Prototype precision feedback email messages.
Figure 6
Figure 6
A process model for feedback intervention success.
Figure 7
Figure 7
An information value chain for feedback intervention success.

References

    1. McClure RC, Macumber CL, Skapik JL, Smith AM. Igniting harmonized digital clinical quality measurement through terminology, CQL, and FHIR. Appl Clin Inform. 2020 Jan;11(1):23–33. doi: 10.1055/s-0039-3402755.
    1. Dowding D, Randell R, Gardner P, Fitzpatrick G, Dykes P, Favela J, Hamer S, Whitewood-Moores Z, Hardiker N, Borycki E, Currie L. Dashboards for improving patient care: review of the literature. Int J Med Inform. 2015 Feb;84(2):87–100. doi: 10.1016/j.ijmedinf.2014.10.001.S1386-5056(14)00189-0
    1. Tuti T, Nzinga J, Njoroge M, Brown B, Peek N, English M, Paton C, van der Veer SN. A systematic review of electronic audit and feedback: intervention effectiveness and use of behaviour change theory. Implement Sci. 2017 May 12;12(1):61. doi: 10.1186/s13012-017-0590-z. 10.1186/s13012-017-0590-z
    1. Westert GP, Groenewoud S, Wennberg JE, Gerard C, DaSilva P, Atsma F, Goodman DC. Medical practice variation: public reporting a first necessary step to spark change. Int J Qual Health Care. 2018 Nov 01;30(9):731–5. doi: 10.1093/intqhc/mzy092. 4990397
    1. Ivers NM, Grimshaw JM, Jamtvedt G, Flottorp S, O'Brien MA, French SD, Young J, Odgaard-Jensen J. Growing literature, stagnant science? Systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care. J Gen Intern Med. 2014 Nov;29(11):1534–41. doi: 10.1007/s11606-014-2913-y.
    1. Brehaut JC, Colquhoun HL, Eva KW, Carroll K, Sales A, Michie S, Ivers N, Grimshaw JM. Practice feedback interventions: 15 suggestions for optimizing effectiveness. Ann Intern Med. 2016 Mar 15;164(6):435–41. doi: 10.7326/M15-2248.2494536
    1. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O'Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012 Jun 13;(6):CD000259. doi: 10.1002/14651858.CD000259.pub3.
    1. Foy R, Skrypak M, Alderson S, Ivers NM, McInerney B, Stoddart J, Ingham J, Keenan D. Revitalising audit and feedback to improve patient care. BMJ. 2020 Feb 27;368:m213. doi: 10.1136/bmj.m213.
    1. Ivers NM, Sales A, Colquhoun H, Michie S, Foy R, Francis JJ, Grimshaw JM. No more 'business as usual' with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci. 2014 Jan 17;9:14. doi: 10.1186/1748-5908-9-14. 1748-5908-9-14
    1. Beasley JW, Wetterneck TB, Temte J, Lapin JA, Smith P, Rivera-Rodriguez AJ, Karsh BT. Information chaos in primary care: implications for physician performance and patient safety. J Am Board Fam Med. 2011;24(6):745–51. doi: 10.3122/jabfm.2011.06.100255. 24/6/745
    1. Sittig DF, Wright A, Ash J, Singh H. New unintended adverse consequences of electronic health records. Yearb Med Inform. 2016 Nov 10;(1):7–12. doi: 10.15265/IY-2016-023. me2016-023
    1. Hekler EB, Michie S, Pavel M, Rivera DE, Collins LM, Jimison HB, Garnett C, Parral S, Spruijt-Metz D. Advancing models and theories for digital behavior change interventions. Am J Prev Med. 2016 Nov;51(5):825–32. doi: 10.1016/j.amepre.2016.06.013. S0749-3797(16)30241-0
    1. Chevance G, Perski O, Hekler EB. Innovative methods for observing and changing complex health behaviors: four propositions. Transl Behav Med. 2021 Mar 16;11(2):676–85. doi: 10.1093/tbm/ibaa026. 5838784
    1. Shaban-Nejad A, Michalowski M, Peek N, Brownstein JS, Buckeridge DL. Seven pillars of precision digital health and medicine. Artif Intell Med. 2020 Mar;103:101793. doi: 10.1016/j.artmed.2020.101793.S0933-3657(19)31231-X
    1. Fox CR, Doctor JN, Goldstein NJ, Meeker D, Persell SD, Linder JA. Details matter: predicting when nudging clinicians will succeed or fail. BMJ. 2020 Sep 15;370:m3256. doi: 10.1136/bmj.m3256.
    1. Landis-Lewis Z, Brehaut JC, Hochheiser H, Douglas GP, Jacobson RS. Computer-supported feedback message tailoring: theory-informed adaptation of clinical audit and feedback for learning and behavior change. Implement Sci. 2015 Jan 21;10:12. doi: 10.1186/s13012-014-0203-z. s13012-014-0203-z
    1. Kluger AN, DeNisi A. The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull. 1996 Mar;119(2):254–84. doi: 10.1037/0033-2909.119.2.254.
    1. Carver CS, Scheier MF. Control theory: a useful conceptual framework for personality-social, clinical, and health psychology. Psychol Bull. 1982 Jul;92(1):111–35.
    1. Locke EA, Latham GP. Building a practically useful theory of goal setting and task motivation. A 35-year odyssey. Am Psychol. 2002 Sep;57(9):705–17. doi: 10.1037//0003-066x.57.9.705.
    1. Zhang J, Norman DA. Representations in distributed cognitive tasks. Cogn Sci. 1994 Jan 1;18(1):87–122. doi: 10.1207/s15516709cog1801_3.
    1. Zhang J. A representational analysis of relational information displays. Int J Hum Comput Stud. 1996 Jul;45(1):59–74. doi: 10.1006/ijhc.1996.0042.
    1. Galesic M, Garcia-Retamero R. Graph literacy: a cross-cultural comparison. Med Decis Making. 2011 May;31(3):444–57. doi: 10.1177/0272989X10373805.0272989X10373805
    1. Dowding D, Merrill JA, Onorato N, Barrón Y, Rosati RJ, Russell D. The impact of home care nurses' numeracy and graph literacy on comprehension of visual display information: implications for dashboard design. J Am Med Inform Assoc. 2018 Feb 01;25(2):175–82. doi: 10.1093/jamia/ocx042. 3778216
    1. Petit-Monéger A, Saillour-Glénisson F, Nouette-Gaulain K, Jouhet V, Salmi LR. Comparing graphical formats for feedback of clinical practice data. A multicenter study among anesthesiologists in France. Methods Inf Med. 2017 Jan 09;56(1):28–36. doi: 10.3414/ME15-01-0163.15-01-0163
    1. Zikmund-Fisher BJ. Helping people know whether measurements have good or bad implications: increasing the evaluability of health and science data communications. Policy Insights Behav Brain Sci. 2019 Mar 08;6(1):29–37. doi: 10.1177/2372732218813377.
    1. Zikmund-Fisher BJ. The right tool is what they need, not what we have: a taxonomy of appropriate levels of precision in patient risk communication. Med Care Res Rev. 2013 Feb;70(1 Suppl):37S–49S. doi: 10.1177/1077558712458541.1077558712458541
    1. Brainerd CJ, Reyna VF. Gist is the grist: fuzzy-trace theory and the new intuitionism. Developmental Review. 1990 Mar;10(1):3–47. doi: 10.1016/0273-2297(90)90003-m.
    1. Reyna VF. A scientific theory of gist communication and misinformation resistance, with implications for health, education, and policy. Proc Natl Acad Sci U S A. 2021 Apr 13;118(15):e1912441117. doi: 10.1073/pnas.1912441117. 1912441117
    1. Brown B, Gude WT, Blakeman T, van der Veer SN, Ivers N, Francis JJ, Lorencatto F, Presseau J, Peek N, Daker-White G. Clinical performance feedback intervention theory (CP-FIT): a new theory for designing, implementing, and evaluating feedback in health care based on a systematic review and meta-synthesis of qualitative research. Implement Sci. 2019 Apr 26;14(1):40. doi: 10.1186/s13012-019-0883-5. 10.1186/s13012-019-0883-5
    1. Ilgen DR, Fisher CD, Taylor MS. Consequences of individual feedback on behavior in organizations. Journal of Applied Psychology. 1979;64(4):349–71. doi: 10.1037/0021-9010.64.4.349.
    1. Higgins ET. Beyond pleasure and pain. Am Psychol. 1997 Dec;52(12):1280–300. doi: 10.1037//0003-066x.52.12.1280.
    1. Brockner J, Higgins E. Regulatory focus theory: implications for the study of emotions at work. Organ Behav Hum Decis Process. 2001 Sep;86(1):35–66. doi: 10.1006/obhd.2001.2972.
    1. Higgins ET. Value From Regulatory Fit. Curr Dir Psychol Sci. 2016 Jun 24;14(4):209–13. doi: 10.1111/j.0963-7214.2005.00366.x.
    1. Flynn AJ, Bahulekar N, Boisvert P, Lagoze C, Meng G, Rampton J, Friedman CP. Architecture and initial development of a digital library platform for computable knowledge objects for health. Stud Health Technol Inform. 2017;235:496–500.
    1. Flynn AJ, Friedman CP, Boisvert P, Landis-Lewis Z, Lagoze C. The knowledge object reference ontology (KORO): a formalism to support management and sharing of computable biomedical knowledge for learning health systems. Learn Health Syst. 2018 Apr;2(2):e10054. doi: 10.1002/lrh2.10054. doi: 10.1002/lrh2.10054.LRH210054
    1. Flynn AJ, Milstein JA, Boisvert P, Gittlen N, Lagoze C, Meng G. ScriptNumerate: a data-to-advice pipeline using compound digital objects to increase the interoperability of computable biomedical knowledge. AMIA Annu Symp Proc. 2018;2018:440–9.
    1. Panicker V, Lee D, Wetmore M, Rampton J, Smith R, Moniz M, Landis-Lewis Z. Designing tailored displays for clinical practice feedback: developing requirements with user stories. Stud Health Technol Inform. 2019 Aug 21;264:1308–12. doi: 10.3233/SHTI190438. SHTI190438
    1. Colquhoun DA, Shanks AM, Kapeles SR, Shah N, Saager L, Vaughn MT, Buehler K, Burns ML, Tremper KK, Freundlich RE, Aziz M, Kheterpal S, Mathis MR. Considerations forintegration of perioperative electronic health records across institutions for research and quality improvement: the approach taken by the multicenter perioperative outcomes group. Anesth Analg. 2020 May;130(5):1133–46. doi: 10.1213/ANE.0000000000004489. 00000539-202005000-00007
    1. Hansen P, Ombler F. A new method for scoring additive multi-attribute value models using pairwise rankings of alternatives. Journal of Multi-Criteria Decision Analysis. 2008 May;15(3-4):87–107. doi: 10.1002/mcda.428.
    1. Manning CD, Schütze H, Raghavan P. Introduction to Information Retrieval. Cambridge, UK: Cambridge University Press; 2009. Hierarchical Clustering; pp. 346–68.
    1. Flynn AJ, Shi W, Fischer R, Friedman CP. Digital knowledge objects and digital knowledge object clusters: unit holdings in a learning health system knowledge repository. 49th Hawaii International Conference on System Sciences; HICSS; Jan 5-8, 2016; Koloa, HI, USA. 2016.
    1. Kushniruk AW, Borycki EM, Kuwata S, Kannry J. Emerging approaches to usability evaluation of health information systems: towards in-situ analysis of complex healthcare systems and environments. Stud Health Technol Inform. 2011;169:915–9.
    1. Chokshi SK, Mann DM. Innovating from within: a process model for user-centered digital development in academic medical centers. JMIR Hum Factors. 2018 Dec 19;5(4):e11048. doi: 10.2196/11048. v5i4e11048
    1. Gude WT, van der Veer SN, de Keizer NF, Coiera E, Peek N. Optimizing digital health informatics interventions through unobtrusive quantitative process evaluations. Stud Health Technol Inform. 2016;228:594–8.
    1. Coiera E. Assessing technology success and failure using information value chain theory. Stud Health Technol Inform. 2019 Jul 30;263:35–48. doi: 10.3233/SHTI190109.SHTI190109
    1. ASPIRE Measures. Multicenter Perioperative Outcomes Group. [2022-04-14]. .
    1. Lyon AR, Brewer SK, Areán PA. Leveraging human-centered design to implement modern psychological science: return on an early investment. Am Psychol. 2020 Nov;75(8):1067–79. doi: 10.1037/amp0000652. 2020-88219-004
    1. Landis-Lewis Z, Kononowech J, Scott WJ, Hogikyan RV, Carpenter JG, Periyakoil VS, Miller SC, Levy C, Ersek M, Sales A. Designing clinical practice feedback reports: three steps illustrated in Veterans Health Affairs long-term care facilities and programs. Implement Sci. 2020 Jan 21;15(1):7. doi: 10.1186/s13012-019-0950-y. 10.1186/s13012-019-0950-y
    1. Manning C, Raghavan P, Schütze H. Introduction to Information Retrieval. Cambridge, UK: Cambridge University Press; 2008. p. 482.
    1. Lee D, Panicker V, Gross C, Zhang J, Landis-Lewis Z. What was visualized? A method for describing content of performance summary displays in feedback interventions. BMC Med Res Methodol. 2020 Apr 23;20(1):90. doi: 10.1186/s12874-020-00951-x. 10.1186/s12874-020-00951-x
    1. Arp R, Smith B, Spear AD. Building Ontologies with Basic Formal Ontology. Cambridge, MA, USA: The MIT Press; 2015. Jul 31,
    1. Performance Summary Display Ontology. BioPortal, National Center for Biomedical Ontology. [2022-04-14]. .
    1. Lewis CC, Klasnja P, Powell BJ, Lyon AR, Tuzzio L, Jones S, Walsh-Bailey C, Weiner B. From classification to causality: advancing understanding of mechanisms of change in implementation science. Front Public Health. 2018 May 7;6:136. doi: 10.3389/fpubh.2018.00136. doi: 10.3389/fpubh.2018.00136.
    1. Michie S, West R, Finnerty AN, Norris E, Wright AJ, Marques MM, Johnston M, Kelly MP, Thomas J, Hastings J. Representation of behaviour change interventions and their evaluation: development of the upper level of the behaviour change intervention ontology. Wellcome Open Res. 2020 Jan 6;5:123. doi: 10.12688/wellcomeopenres.15902.2.
    1. Rao VR. Applied Conjoint Analysis. Switzerland: Springer, 2014 edition; 2014. Feb 20,
    1. Green PE, Rao VR. Conjoint measurement- for quantifying judgmental data. Journal of Marketing Research. 1971 Aug 1;8(3):355–63. doi: 10.1177/002224377100800312.
    1. Bridges JF, Hauber AB, Marshall D, Lloyd A, Prosser LA, Regier DA, Johnson FR, Mauskopf J. Conjoint analysis applications in health--a checklist: a report of the ISPOR good research practices for conjoint analysis task force. Value Health. 2011 Jun;14(4):403–13. doi: 10.1016/j.jval.2010.11.013. S1098-3015(10)00083-5
    1. Fraenkel L, Nowell WB, Michel G, Wiedmeyer C. Preference phenotypes to facilitate shared decision-making in rheumatoid arthritis. Ann Rheum Dis. 2018 May;77(5):678–83. doi: 10.1136/annrheumdis-2017-212407. annrheumdis-2017-212407
    1. Pinto D, Bockenholt U, Lee J, Chang RW, Sharma L, Finn DJ, Heinemann AW, Holl JL, Hansen P. Preferences for physical activity: a conjoint analysis involving people with chronic knee pain. Osteoarthritis Cartilage. 2019 Feb;27(2):240–7. doi: 10.1016/j.joca.2018.10.002. S1063-4584(18)31500-0
    1. Conjoint Analysis Software. 1000minds. [2022-04-14]. .
    1. NbClust: Determining the Best Number of Clusters in a Data Set. 2015. [2022-04-14]. .
    1. The R Project for Statistical Computing. [2022-04-14].
    1. Wilkinson MD, Dumontier M, Aalbersberg IJ, Appleton G, Axton M, Baak A, Blomberg N, Boiten JW, da Silva Santos LB, Bourne PE, Bouwman J, Brookes AJ, Clark T, Crosas M, Dillo I, Dumon O, Edmunds S, Evelo CT, Finkers R, Gonzalez-Beltran A, Gray AJ, Groth P, Goble C, Grethe JS, Heringa J, 't Hoen PA, Hooft R, Kuhn T, Kok R, Kok J, Lusher SJ, Martone ME, Mons A, Packer AL, Persson B, Rocca-Serra P, Roos M, van Schaik R, Sansone SA, Schultes E, Sengstag T, Slater T, Strawn G, Swertz MA, Thompson M, van der Lei J, van Mulligen E, Velterop J, Waagmeester A, Wittenburg P, Wolstencroft K, Zhao J, Mons B. The FAIR guiding principles for scientific data management and stewardship. Sci Data. 2016 Mar 15;3:160018. doi: 10.1038/sdata.2016.18. doi: 10.1038/sdata.2016.18.sdata201618
    1. Michie S, Johnston M. Theories and techniques of behaviour change: developing a cumulative science of behaviour change. Health Psychol Rev. 2012 Feb 15;6(1):1–6. doi: 10.1080/17437199.2012.654964.
    1. Brehaut JC, Eva KW. Building theories of knowledge translation interventions: use the entire menu of constructs. Implement Sci. 2012 Nov 22;7:114. doi: 10.1186/1748-5908-7-114. 1748-5908-7-114
    1. Cane J, O'Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012 Apr 24;7:37. doi: 10.1186/1748-5908-7-37. 1748-5908-7-37
    1. Michie S, West R, Campbell R, Brown J, Gainforth H. ABC of Behaviour Change Theories (ABC of Behavior Change) Sutton, UK: Silverback Publishing; 2014. May 31, An Essential Resource for Researchers, Policy Makers and Practitioners; p. A.
    1. West R, Godinho CA, Bohlen LC, Carey RN, Hastings J, Lefevre CE, Michie S. Development of a formal system for representing behaviour-change theories. Nat Hum Behav. 2019 May 8;3(5):526–36. doi: 10.1038/s41562-019-0561-2.10.1038/s41562-019-0561-2
    1. Kluger AN, Van Dijk D. Feedback, the various tasks of the doctor, and the feedforward alternative. Med Educ. 2010 Dec;44(12):1166–74. doi: 10.1111/j.1365-2923.2010.03849.x.
    1. Coiera E. Guide to Health Informatics. 3rd Edition. Boca Raton, FL, USA: CRC Press; 2015. Mar 12,
    1. Gude WT, Brown B, van der Veer SN, Colquhoun HL, Ivers NM, Brehaut JC, Landis-Lewis Z, Armitage CJ, de Keizer NF, Peek N. Clinical performance comparators in audit and feedback: a review of theory and evidence. Implement Sci. 2019 Apr 24;14(1):39. doi: 10.1186/s13012-019-0887-1. 10.1186/s13012-019-0887-1
    1. Ivers NM, Halperin IJ, Barnsley J, Grimshaw JM, Shah BR, Tu K, Upshur R, Zwarenstein M. Allocation techniques for balance at baseline in cluster randomized trials: a methodological review. Trials. 2012 Aug 01;13:120. doi: 10.1186/1745-6215-13-120. 1745-6215-13-120
    1. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O'Cathain A, Tinati T, Wight D, Baird J. Process evaluation of complex interventions: medical research council guidance. BMJ. 2015 Mar 19;350:h1258. doi: 10.1136/bmj.h1258.
    1. Grimshaw JM, Ivers N, Linklater S, Foy R, Francis JJ, Gude WT, Hysong SJ, Audit and Feedback MetaLab Reinvigorating stagnant science: implementation laboratories and a meta-laboratory to efficiently advance the science of audit and feedback. BMJ Qual Saf. 2019 May;28(5):416–23. doi: 10.1136/bmjqs-2018-008355. bmjqs-2018-008355
    1. Agency for Healthcare Research and Quality. Part Three: Three Strategies for Continuous Improvement of Physician Feedback Reporting Systems. 2016. Feb, [2022-04-14]. .
    1. Confidential Physician Feedback Reports: Designing for Optimal Impact on Performance. 2016. Feb, [2022-04-14]. .
    1. Hegarty M. Score Reporting Research and Applications, 1st Edition. London, UK: Routledge; 2018. Advances in Cognitive Science and Information Visualization.
    1. Munzner T. A nested model for visualization design and validation. IEEE Trans Vis Comput Graph. 2009 Nov;15(6):921–8. doi: 10.1109/tvcg.2009.111.
    1. Kluger AN, DeNisi A. Feedback interventions: toward the understanding of a double-edged sword. Curr Dir Psychol Sci. 1998 Jun;7(3):67–72. doi: 10.1111/1467-8721.ep10772989.
    1. Hattie J, Timperley H. The power of feedback. Review of Educational Research. 2007 Mar 1;77(1):81–112. doi: 10.3102/003465430298487.
    1. Van Dijk D, Kluger AN. Task type as a moderator of positive/negative feedback effects on motivation and performance: a regulatory focus perspective. J Organ Behav. 2010 Sep 07;32(8):1084–105. doi: 10.1002/job.725.
    1. Higgins ET, Shah J, Friedman R. Emotional responses to goal attainment: strength of regulatory focus as moderator. J Pers Soc Psychol. 1997 Mar;72(3):515–25. doi: 10.1037//0022-3514.72.3.515.
    1. Sirajuddin AM, Osheroff JA, Sittig DF, Chuo J, Velasco F, Collins DA. Implementation pearls from a new guidebook on improving medication use and outcomes with clinical decision support. Effective CDS is essential for addressing healthcare performance improvement imperatives. J Healthc Inf Manag. 2009;23(4):38–45.
    1. Brown B, Peek N, Buchan I. The case for conceptual and computable cross-fertilization between audit and feedback and clinical decision support. Stud Health Technol Inform. 2015;216:419–23.
    1. Colquhoun H, Michie S, Sales A, Ivers N, Grimshaw JM, Carroll K, Chalifoux M, Eva K, Brehaut J. Reporting and design elements of audit and feedback interventions: a secondary review. BMJ Qual Saf. 2017 Jan;26(1):54–60. doi: 10.1136/bmjqs-2015-005004.bmjqs-2015-005004
    1. Shortliffe EH, Sepúlveda MJ. Clinical decision support in the era of artificial intelligence. JAMA. 2018 Dec 04;320(21):2199–200. doi: 10.1001/jama.2018.17163.2713901
    1. Hysong SJ, Best RG, Pugh JA. Audit and feedback and clinical practice guideline adherence: making feedback actionable. Implement Sci. 2006 Apr 28;1:9. doi: 10.1186/1748-5908-1-9. 1748-5908-1-9
    1. Vessey I. Cognitive fit: a theory-based analysis of the graphs versus tables literature. Decision Sciences. 1991 Mar;22(2):219–40. doi: 10.1111/j.1540-5915.1991.tb00344.x.
    1. De Smedt K, Koureas D, Wittenburg P. FAIR digital objects for science: from data pieces to actionable knowledge units. Publications, Multidisciplinary Digital Publishing Institute. 2020 Apr 11;8(2):21. doi: 10.3390/publications8020021.
    1. Wittenburg P, Strawn G, Mons B, Boninho L, Schultes E. Digital objects as drivers towards convergence in data infrastructures. B2Share Eudat. 2019 Jan 4;10 doi: 10.23728/b2share.b605d85809ca45679b110719b6c6cb11.

Source: PubMed

Patrocinadores e Colaboradores

Condições médicas

Intervenções de drogas

3
Se inscrever