Increasing the quantity and quality of searching for current best evidence to answer clinical questions: protocol and intervention design of the MacPLUS FS Factorial Randomized Controlled Trials

Thomas Agoritsas, Emma Iserman, Nicholas Hobson, Natasha Cohen, Adam Cohen, Pavel S Roshanov, Miguel Perez, Chris Cotoi, Rick Parrish, Eleanor Pullenayegum, Nancy L Wilczynski, Alfonso Iorio, R Brian Haynes, Thomas Agoritsas, Emma Iserman, Nicholas Hobson, Natasha Cohen, Adam Cohen, Pavel S Roshanov, Miguel Perez, Chris Cotoi, Rick Parrish, Eleanor Pullenayegum, Nancy L Wilczynski, Alfonso Iorio, R Brian Haynes

Abstract

Background & aims: Finding current best evidence for clinical decisions remains challenging. With 3,000 new studies published every day, no single evidence-based resource provides all answers or is sufficiently updated. McMaster Premium LiteratUre Service--Federated Search (MacPLUS FS) addresses this issue by looking in multiple high quality resources simultaneously and displaying results in a one-page pyramid with the most clinically useful at the top. Yet, additional logistical and educational barriers need to be addressed to enhance point-of-care evidence retrieval. This trial seeks to test three innovative interventions, among clinicians registered to MacPLUS FS, to increase the quantity and quality of searching for current best evidence to answer clinical questions.

Methods & design: In a user-centered approach, we designed three interventions embedded in MacPLUS FS: (A) a web-based Clinical Question Recorder; (B) an Evidence Retrieval Coach composed of eight short educational videos; (C) an Audit, Feedback and Gamification approach to evidence retrieval, based on the allocation of 'badges' and 'reputation scores.' We will conduct a randomized factorial controlled trial among all the 904 eligible medical doctors currently registered to MacPLUS FS at the hospitals affiliated with McMaster University, Canada. Postgraduate trainees (n=429) and clinical faculty/staff (n=475) will be randomized to each of the three following interventions in a factorial design (AxBxC). Utilization will be continuously recorded through clinicians’ accounts that track logins and usage, down to the level of individual keystrokes. The primary outcome is the rate of searches per month per user during the six months of follow-up. Secondary outcomes, measured through the validated Impact Assessment Method questionnaire, include: utility of answers found (meeting clinicians’ information needs), use (application in practice), and perceived usefulness on patient outcomes.

Discussion: Built on effective models for the point-of-care teaching, these interventions approach evidence retrieval as a clinical skill. If effective, they may offer the opportunity to enhance it for a large audience, at low cost, providing better access to relevant evidence across many top EBM resources in parallel.

Trial registration: ClinicalTrials.Gov NCT02038439.

Figures

Figure 1
Figure 1
MacPLUS FS search output.
Figure 2
Figure 2
Illustration of the Clinical Question Recorder and Reminder. A,B,C,D: For a detailed description of each feature displayed, see the result section in the section "Intervention A - clinical question recorder".
Figure 3
Figure 3
Illustration of a video embedded in MacPLUS FS in the Evidence Retrieval Coach.
Figure 4
Figure 4
Illustration of the components of the Audit, Feedback & Gamification. A - E: For a detailed description of each feature displayed, see the result section in the section "Intervention C - audit, feedback and gamification.

References

    1. Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA. 1992;268:240–248. doi: 10.1001/jama.1992.03490020088036.
    1. Eymin G, Jaffer AK. Evidence behind quality of care measures for venous thromboembolism and atrial fibrillation. J Thromb Thrombolysis. 2013;37(2):87–96. doi: 10.1007/s11239-013-0874-3.
    1. Gialamas A, Yelland LN, Ryan P, Willson K, Laurence CO, Bubner TK, Tideman P, Beilby JJ. Does point-of-care testing lead to the same or better adherence to medication? A randomised controlled trial: the PoCT in General Practice Trial. Med J Aust. 2009;191:487–491.
    1. Green ML, Ciampi MA, Ellis PJ. Residents’ medical information needs in clinic: are they being met? Am J Med. 2000;109:218–223. doi: 10.1016/S0002-9343(00)00458-7.
    1. Gonzalez-Gonzalez AI, Dawes M, Sanchez-Mateos J, Riesgo-Fuertes R, Escortell-Mayor E, Sanz-Cuesta T, Hernandez-Fernandez T. Information needs and information-seeking behavior of primary care physicians. Ann Fam Med. 2007;5:345–352. doi: 10.1370/afm.681.
    1. Graber MA, Randles BD, Ely JW, Monnahan J. Answering clinical questions in the ED. Am J Emerg Med. 2008;26:144–147. doi: 10.1016/j.ajem.2007.03.031.
    1. Straus SE. Evidence-based medicine : how to practice and teach EBM. Edinburgh; New York: Elsevier/Churchill Livingstone; 2005.
    1. Glasziou P, Burls A, Gilbert R. Evidence based medicine and the medical curriculum. BMJ. 2008;337:a1253. doi: 10.1136/bmj.a1253.
    1. Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7:e1000326. doi: 10.1371/journal.pmed.1000326.
    1. Hoogendam A, Stalenhoef AF, Robbe PF, Overbeke AJ. Answers to questions posed during daily patient care are more likely to be answered by UpToDate than PubMed. J Med Internet Res. 2008;10:e29. doi: 10.2196/jmir.1012.
    1. Hoogendam A, Stalenhoef AF, Robbe PF, Overbeke AJ. Analysis of queries sent to PubMed at the point of care: observation of search behaviour in a medical teaching hospital. BMC Med Inform Decis Mak. 2008;8:42. doi: 10.1186/1472-6947-8-42.
    1. Thiele RH, Poiro NC, Scalzo DC, Nemergut EC. Speed, accuracy, and confidence in Google, Ovid, PubMed, and UpToDate: results of a randomised trial. Postgrad Med J. 2010;86:459–465. doi: 10.1136/pgmj.2010.098053.
    1. Banzi R, Liberati A, Moschetti I, Tagliabue L, Moja L. A review of online evidence-based practice point-of-care information summary providers. J Med Internet Res. 2010;12:e26. doi: 10.2196/jmir.1288.
    1. Haynes RB. ACP Journal Club: the best new evidence for patient care. ACP J Club. 2008;148:2.
    1. Prorok JC, Iserman EC, Wilczynski NL, Haynes RB. The quality, breadth, and timeliness of content updating vary substantially for 10 online medical texts: an analytic survey. J Clin Epidemiol. 2012;65:1289–1295. doi: 10.1016/j.jclinepi.2012.05.003.
    1. Banzi R, Cinquini M, Liberati A, Moschetti I, Pecoraro V, Tagliabue L, Moja L. Speed of updating online evidence based point of care summaries: prospective cohort analysis. BMJ. 2011;343:d5856. doi: 10.1136/bmj.d5856.
    1. Ely JW, Osheroff JA, Ebell MH, Bergus GR, Levy BT, Chambliss ML, Evans ER. Analysis of questions asked by family doctors regarding patient care. BMJ. 1999;319:358–361. doi: 10.1136/bmj.319.7206.358.
    1. Eden J, Wheatley B, McNeil B HS. Committee on Reviewing Evidence to Identify Highly Effective Clinical Services. Washington, D.C: Institute Of Medicine of The National Academies The National Academies Press; 2008.
    1. Ely JW, Osheroff JA, Ebell MH, Chambliss ML, Vinson DC, Stevermer JJ, Pifer EA. Obstacles to answering doctors’ questions about patient care with evidence: qualitative study. BMJ. 2002;324:710. doi: 10.1136/bmj.324.7339.710.
    1. DiCenso A, Bayley L, Haynes RB. ACP Journal Club. Editorial: Accessing preappraised evidence: fine-tuning the 5S model into a 6S model. Ann Intern Med. 2009;151:JC3-2–JC3-3. doi: 10.7326/0003-4819-151-6-200909150-02002.
    1. Haynes RB, Cotoi C, Holland J, Walters L, Wilczynski N, Jedraszewski D, McKinlay J, Parrish R, McKibbon KA. Second-order peer review of the medical literature for clinical practitioners. JAMA. 2006;295:1801–1808. doi: 10.1001/jama.295.15.1801.
    1. Wilczynski NL, McKibbon KA, Walter SD, Garg AX, Haynes RB. MEDLINE clinical queries are robust when searching in recent publishing years. J Am Med Inform Assoc. 2013;20(2):363–368. doi: 10.1136/amiajnl-2012-001075.
    1. Haynes RB, Holland J, Cotoi C, McKinlay RJ, Wilczynski NL, Walters LA, Jedras D, Parrish R, McKibbon KA, Garg A, Walter SD. McMaster PLUS: a cluster randomized clinical trial of an intervention to accelerate clinical use of evidence-based information from digital libraries. J Am Med Inform Assoc. 2006;13:593–600. doi: 10.1197/jamia.M2158.
    1. EvidenceUpdates from BMJ.., []
    1. Anderson HJ. Hitting a moving target. Choosing a “standard” for devices that clinicians use to access data proves challenging. Health Data Manag. 2009;17(32):34–36.
    1. Ebell MH. How to find answers to clinical questions. Am Fam Physician. 2009;79:293–296.
    1. McAlister FA, Graham I, Karr GW, Laupacis A. Evidence-based medicine and the practicing clinician. J Gen Intern Med. 1999;14:236–242. doi: 10.1046/j.1525-1497.1999.00323.x.
    1. McColl A, Smith H, White P, Field J. General practitioner’s perceptions of the route to evidence based medicine: a questionnaire survey. BMJ. 1998;316:361–365. doi: 10.1136/bmj.316.7128.361.
    1. Young JM, Ward JE. Evidence-based medicine in general practice: beliefs and barriers among Australian GPs. J Eval Clin Pract. 2001;7:201–210. doi: 10.1046/j.1365-2753.2001.00294.x.
    1. Neher JO, Gordon KC, Meyer B, Stevens N. A five-step “microskills” model of clinical teaching. J Am Board Fam Pract. 1992;5:419–424.
    1. Furney SL, Orsini AN, Orsetti KE, Stern DT, Gruppen LD, Irby DM. Teaching the one-minute preceptor. A randomized controlled trial. J Gen Intern Med. 2001;16:620–624. doi: 10.1046/j.1525-1497.2001.016009620.x.
    1. Neher JO, Stevens NG. The one-minute preceptor: shaping the teaching conversation. Fam Med. 2003;35:391–393.
    1. Morrison EH, Rucker L, Boker JR, Gabbert CC, Hubbell FA, Hitchcock MA, Prislin MD. The effect of a 13-hour curriculum to improve residents’ teaching skills: a randomized trial. Ann Intern Med. 2004;141:257–263. doi: 10.7326/0003-4819-141-4-200408170-00005.
    1. Parrot S, Dobbie A, Chumley H, Tysinger JW. Evidence-based office teaching-the five-step microskills model of clinical teaching. Fam Med. 2006;38:164–167.
    1. Coiera E. When conversation is better than computation. J Am Med Inform Assoc. 2000;7:277–286. doi: 10.1136/jamia.2000.0070277.
    1. Coughlan P, Suri JF, Canales K. Prototypes as (Design) Tools for Behavioral and Organizational Change: A Design-Based Approach to Help Organizations Change Work Behaviors. J Appl Behav Sci. 2007;43:122–134. doi: 10.1177/0021886306297722.
    1. Treweek S, Oxman AD, Alderson P, Bossuyt PM, Brandt L, Brozek J, Davoli M, Flottorp S, Harbour R, Hill S, Liberati A, Liira H, Schünemann HJ, Rosenbaum S, Thornton J, Vandvik PO, Alonso-Coello P. Developing and evaluating communication strategies to support informed decisions and practice based on evidence (DECIDE): protocol and preliminary results. Implement Sci. 2013;8:6. doi: 10.1186/1748-5908-8-6.
    1. Rosenbaum SE, Glenton C, Nylund HK, Oxman AD. User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews. J Clin Epidemiol. 2010;63:607–619. doi: 10.1016/j.jclinepi.2009.12.013.
    1. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O’Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6:CD000259.
    1. Kapp KM. The gamification of learning and instruction : game-based methods and strategies for training and education. San Francisco, CA: Pfeiffer; 2012.
    1. Grad R, Pluye P, Granikov V, Johnson-Lafleur J, Shulha M, Sridhar SB. Physicians’ assessment of the value of clinical information: Operationalization of a theoretical model. J Am Soc Inf Sci Technol. 2011;62:1884–1891. doi: 10.1002/asi.21590.
    1. Pluye P, Grad RM, Granikov V, Jagosh J, Leung K. Evaluation of email alerts in practice: part 1 - review of the literature on clinical emailing channels. J Eval Clin Pract. 2010;16:1227–1235. doi: 10.1111/j.1365-2753.2009.001301.x.
    1. Pluye P, Grad RM, Johnson-Lafleur J, Bambrick T, Burnand B, Mercer J, Marlow B, Campbell C. Evaluation of email alerts in practice: Part 2 - validation of the information assessment method. J Eval Clin Pract. 2010;16:1236–1243. doi: 10.1111/j.1365-2753.2009.01313.x.
    1. Pluye P, Grad RM, Shulha M, Granikov V, Leung K. Using electronic knowledge resources for person-centered medicine - I: An evaluation model. Int J Pers Cent Med. 2011;1:385–394.
    1. Pluye P, Grad RM, Mysore N, Shulha M, Jorhnson-Lafleur J. Using electronic knowledge resources for person-centered med- icine - II: The Number Needed to Benefit from Information (NNBI) Int J Pers Cent Med. 2011;1:395–404.
    1. McAlister FA, Straus SE, Sackett DL, Altman DG. Analysis and reporting of factorial trials: a systematic review. JAMA. 2003;289:2545–2553. doi: 10.1001/jama.289.19.2545.
    1. Mickan S, BOccThy, Tilson JK, Atherton H, Roberts NW, Heneghan C. Evidence of effectiveness of health care professionals using handheld computers: a scoping review of systematic reviews. J Med Internet Res. 2013;15(10):e212. doi: 10.2196/jmir.2530.
    1. Ozdalga E, Ozdalga A, Ahuja N. The smartphone in medicine: a review of current and potential use among physicians and students. J Med Internet Res. 2012;14:e128. doi: 10.2196/jmir.1994.
    1. Straus SE. Evidence-based medicine : how to practice and teach EBM. 4th Edition edn. Edinburgh. New York: Elsevier/Churchill Livingstone; 2011.

Source: PubMed

Спонсоры и соавторы

Заболевания

Медикаментозные вмешательства

Подписаться