Comparative Usability Analysis and Parental Preferences of Three Web-Based Knowledge Translation Tools: Multimethod Study

Harrison Anzinger, Sarah Alexandra Elliott, Lisa Hartling, Harrison Anzinger, Sarah Alexandra Elliott, Lisa Hartling

Abstract

Background: Connecting parents to research evidence is known to improve health decision making. However, guidance on how to develop effective knowledge translation (KT) tools that synthesize child-health evidence into a form understandable by parents is lacking.

Objective: The aim of this study was to conduct a comparative usability analysis of three Web-based KT tools to identify differences in tool effectiveness, identify which format parents prefer, and better understand what factors affect usability for parents.

Methods: We evaluated a Cochrane plain language summary (PLS), Blogshot, and a Wikipedia page on a specific child-health topic (acute otitis media). A mixed method approach was used involving a knowledge test, written usability questionnaire, and a semistructured interview. Differences in knowledge and usability questionnaire scores for each of the KT tools were analyzed using Kruskal-Wallis tests, considering a critical significance value of P=.05. Thematic analysis was used to synthesize and identify common parent preferences among the semistructured interviews. Key elements parents wanted in a KT tool were derived through author consensus using questionnaire data and parent interviews.

Results: In total, 16 parents (9 female) with a mean age of 39.6 (SD 11.9) years completed the study. Parents preferred the Blogshot over the PLS and Wikipedia page (P=.002) and found the Blogshot to be the most aesthetic (P=.001) and easiest to use (P=.001). Knowledge questions and usability survey data also indicated that the Blogshot was the most preferred and effective KT tool at relaying information about the topic. Four key themes were derived from thematic analysis, describing elements parents valued in KT tools. Parents wanted tools that were (1) simple, (2) quick to access and use, and (3) trustworthy, and which (4) informed how to manage the condition. Out of the three KT tools assessed, Blogshots were the most preferred tool by parents and encompassed these four key elements.

Conclusions: It is important that child health evidence be available in formats accessible and understandable by parents to improve decision making, use of health care resources, and health outcomes. Further usability testing of different KT tools should be conducted involving broader populations and other conditions (eg, acute vs chronic) to generate guidelines to improve KT tools for parents.

Keywords: child health; comparative study; health information,consumer; internet; knowledge translation; parents.

Conflict of interest statement

Conflicts of Interest: Two authors (SAE and LH) of this paper are members of the Cochrane Collaboration and Cochrane Child Health. Cochrane Child Health produced the Blogshot. Although Wikipedia is an independent organization, there is an agreement between Wikipedia and the Cochrane Collaboration to allow dissemination of Cochrane studies on Wikipedia articles.

©Harrison Anzinger, Sarah Alexandra Elliott, Lisa Hartling. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 13.03.2020.

Figures

Figure 1
Figure 1
Participant flow diagram. Participants (n=16) were randomized to 1 of 3 groups: Cochrane plain language summary (n=5), Cochrane Blogshot (n=5), or Wikipedia Page (n=6). Participants had unlimited time to read the knowledge translation (KT) tool. Participants were then given a knowledge test without being able to refer back to the KT tool. Participants then completed a written usability questionnaire and semistructured interview focusing on the KT tool randomly assigned. Finally, participants were asked to read the other two KT tools, and a second semistructured interview was completed focusing on comparing the three tools and broader participant preferences for KT tools. All participants completed the study. KT: knowledge translation; PLS: plain language summary.
Figure 2
Figure 2
The honeycomb model of user experience.
Figure 3
Figure 3
Plain language summary, Blogshot, and Wikipedia ranked in order of most (Rank 1) to least (Rank 3) in categories of aesthetics, ease of use, credibility, and general preference. Dot area indicates number of participants placing each tool at a given rank (N=16). Asterisk indicates statistically significant difference in rank order between groups determined via a Friedman test (P<.05). PLS: plain language summary.

References

    1. Kothari A, Wathen CN. Integrated knowledge translation: digging deeper, moving forward. J Epidemiol Community Health. 2017 Jun;71(6):619–23. doi: 10.1136/jech-2016-208490.jech-2016-208490
    1. Albrecht L, Scott SD, Hartling L. Knowledge translation tools for parents on child health topics: a scoping review. BMC Health Serv Res. 2017 Sep 29;17(1):686. doi: 10.1186/s12913-017-2632-2. 10.1186/s12913-017-2632-2
    1. International Organization for Standardization. 2018. [2019-05-01]. Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts .
    1. Buljan I, Malički M, Wager E, Puljak L, Hren D, Kellie F, West H, Alfirević Ž, Marušić A. No difference in knowledge obtained from infographic or plain language summary of a Cochrane systematic review: three randomized controlled trials. J Clin Epidemiol. 2018 May;97:86–94. doi: 10.1016/j.jclinepi.2017.12.003. S0895-4356(17)30490-0
    1. Kools M, Ruiter RA, van de Wiel MW, Kok G. Testing the usability of access structures in a health education brochure. Br J Health Psychol. 2007 Nov;12(Pt 4):525–41. doi: 10.1348/135910706X132930.
    1. Marquez C, Johnson AM, Jassemi S, Park J, Moore JE, Blaine C, Bourdon G, Chignell M, Ellen ME, Fortin J, Graham ID, Hayes A, Hamid J, Hemmelgarn B, Hillmer M, Holmes B, Holroyd-Leduc J, Hubert L, Hutton B, Kastner M, Lavis JN, Michell K, Moher D, Ouimet M, Perrier L, Proctor A, Noseworthy T, Schuckel V, Stayberg S, Tonelli M, Tricco AC, Straus SE. Enhancing the uptake of systematic reviews of effects: what is the best format for health care managers and policy-makers? A mixed-methods study. Implement Sci. 2018 Jun 22;13(1):84. doi: 10.1186/s13012-018-0779-9. 10.1186/s13012-018-0779-9
    1. Rosenbaum SE, Glenton C, Nylund HK, Oxman AD. User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews. J Clin Epidemiol. 2010 Jun;63(6):607–19. doi: 10.1016/j.jclinepi.2009.12.013.S0895-4356(10)00024-7
    1. Taddio A, Shah V, Leung E, Wang J, Parikh C, Smart S, Hetherington R, Ipp M, Riddell RP, Sgro M, Jovicic A, Franck L. Knowledge translation of the HELPinKIDS clinical practice guideline for managing childhood vaccination pain: usability and knowledge uptake of educational materials directed to new parents. BMC Pediatr. 2013 Feb 8;13:23. doi: 10.1186/1471-2431-13-23. 1471-2431-13-23
    1. Statistics Canada. 2012. [2019-05-01]. Internet Use by Internet Activity, Age Group, Sex, Level of Education and Household Income .
    1. Fox S, Rainie L, Horrigan J, Lenhart A, Spooner T, Burke M, Lewis O, Cornelia C. Office of The Assistant Secretary for Planning and Evaluation. Washington, DC: Pew Research Center; 2001. [2019-05-01]. The Online Health Care Revolution: How the Web Helps Americans Take Better Care of Themselves .
    1. Eysenbach G, Powell J, Kuss O, Sa E. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. J Am Med Assoc. 2002;287(20):2691–700. doi: 10.1001/jama.287.20.2691.jrv10005
    1. Parvizi M, Talai N, Parvizi Z. Quality of healthcare information on the internet: the case of Apicectomies. Oral Surg. 2017;10(4):e35–9. doi: 10.1111/ors.12263.
    1. Morville P. Semantic Studios. 2004. Jun 21, [2019-04-10]. User Experience Design
    1. Fearns N, Graham K, Johnston G, Service D. Improving the user experience of patient versions of clinical guidelines: user testing of a Scottish Intercollegiate Guideline Network (SIGN) patient version. BMC Health Serv Res. 2016 Feb 2;16:37. doi: 10.1186/s12913-016-1287-8. 10.1186/s12913-016-1287-8
    1. Rosenbaum SE, Glenton C, Cracknell J. User experiences of evidence-based online resources for health professionals: user testing of The Cochrane Library. BMC Med Inform Decis Mak. 2008 Jul 28;8:34. doi: 10.1186/1472-6947-8-34. 1472-6947-8-34
    1. Venekamp RP, Sanders SL, Glasziou PP, Del Mar CB, Rovers MM. Antibiotics for acute otitis media in children. Cochrane Database Syst Rev. 2015 Jun 23;(6):CD000219. doi: 10.1002/14651858.CD000219.pub4.
    1. Dubé E, de Wals P, Gilca V, Boulianne N, Ouakki M, Lavoie F, Bradet R. Burden of acute otitis media on Canadian families. Can Fam Physician. 2011 Jan;57(1):60–5. 57/1/60
    1. Dubé E, de Wals P, Ouakki M. Quality of life of children and their caregivers during an AOM episode: development and use of a telephone questionnaire. Health Qual Life Outcomes. 2010 Jul 26;8:75. doi: 10.1186/1477-7525-8-75. 1477-7525-8-75
    1. Wang EE, Einarson TR, Kellner JD, Conly JM. Antibiotic prescribing for Canadian preschool children: evidence of overprescribing for viral respiratory infections. Clin Infect Dis. 1999 Jul;29(1):155–60. doi: 10.1086/520145.
    1. McCaig LF, Hughes JM. Trends in antimicrobial drug prescribing among office-based physicians in the United States. J Am Med Assoc. 1995 Jan 18;273(3):214–9.
    1. Finkelstein JA, Stille CJ, Rifas-Shiman SL, Goldmann D. Watchful waiting for acute otitis media: are parents and physicians ready? Pediatrics. 2005 Jun;115(6):1466–73. doi: 10.1542/peds.2004-1473.115/6/1466
    1. Meherali S, Campbell A, Hartling L, Scott S. Understanding parents' experiences and information needs on pediatric acute otitis media: a qualitative study. J Patient Exp. 2019 Mar;6(1):53–61. doi: 10.1177/2374373518771362. 10.1177_2374373518771362
    1. Featherstone RM, Leggett C, Knisley L, Jabbour M, Klassen TP, Scott SD, van de Mosselaer G, Hartling L. Creation of an integrated knowledge translation process to improve pediatric emergency care in Canada. Health Commun. 2018 Aug;33(8):980–7. doi: 10.1080/10410236.2017.1323538.
    1. Cochrane Child Health. 2015. Apr 27, [2019-04-10]. Evidence-Based Treatments for Acute Otitis Media (Middle Ear Infection)
    1. Wikipedia. 2019. [2019-04-12]. Otitis Media .
    1. Schünemann HJ, Oxman AD, Higgins JP, Vist GE, Glasziou P, Guyatt GH. Presenting results and ‘summary of findings’ tables. In: Higgins J, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions. Hoboken, New Jersey, United States: Wiley; 2008. pp. 335–57.
    1. Cochrane Methods. London: Cochrane Collaboration; 2013. [2019-04-01]. Methodological Expectations of Cochrane Intervention Reviews (MECIR): Standards for the reporting of Plain Language Summaries in new Cochrane Intervention Reviews .
    1. Arienti C, Gimigliano F, Ryan-Vig S, Kiekens C, Negrini S. Cochrane Rehabilitation blogshots: a modern method to spread Cochrane Evidence. Eur J Phys Rehabil Med. 2018 Jun;54(3):466–8. doi: 10.23736/S1973-9087.18.05313-3. S1973-9087.18.05313-3
    1. Schmidt L, Friedel J, Adams CE. SEED: a tool for disseminating systematic review data into Wikipedia. Syst Rev. 2017 Oct 17;6(1):206. doi: 10.1186/s13643-017-0607-3. 10.1186/s13643-017-0607-3
    1. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. doi: 10.1191/1478088706qp063oa.
    1. MacDermid JC, Uddin Z. A knowledge translation perspective on the two quantitative sensory tests and their usability with clinicians. J Nov Physiother. 2015;5:257. doi: 10.4172/2165-7025.1000257.
    1. Hornbæk K. Current practice in measuring usability: challenges to usability studies and research. Int J Hum Comput Stud. 2006;64(2):79–102. doi: 10.1016/j.ijhcs.2005.06.002.
    1. Reid K, Hartling L, Ali S, Le A, Norris A, Scott SD. Development and usability evaluation of an art and narrative-based knowledge translation tool for parents with a child with pediatric chronic pain: multi-method study. J Med Internet Res. 2017 Dec 14;19(12):e412. doi: 10.2196/jmir.8877. v19i12e412
    1. Fogg BJ, Soohoo C, Danielson DR, Marable L, Stanford J, Tauber ER. How Do Users Evaluate the Credibility of Web Sites?: A Study With Over 2,500 Participants. Proceedings of the 2003 conference on Designing for user experiences; DUX'03; June 6 - 7, 2003; California, San Francisco. 2003. pp. 1–15.
    1. Robins D, Holmes J, Stansbury M. Consumer health information on the web: the relationship of visual design and perceptions of credibility. J Am Soc Inf Sci Technol. 2010;61(1):13–29. doi: 10.1002/asi.21224.

Source: PubMed

스폰서 및 공동 작업자

건강 상태

약물 개입

3
구독하다