Can consumers trust web-based information about celiac disease? Accuracy, comprehensiveness, transparency, and readability of information on the internet

Shawna L McNally, Michael C Donohue, Kimberly P Newton, Sandra P Ogletree, Kristen K Conner, Sarah E Ingegneri, Martin F Kagnoff, Shawna L McNally, Michael C Donohue, Kimberly P Newton, Sandra P Ogletree, Kristen K Conner, Sarah E Ingegneri, Martin F Kagnoff

Abstract

Background: Celiac disease is an autoimmune disease that affects approximately 1% of the US population. Disease is characterized by damage to the small intestinal lining and malabsorption of nutrients. Celiac disease is activated in genetically susceptible individuals by dietary exposure to gluten in wheat and gluten-like proteins in rye and barley. Symptoms are diverse and include gastrointestinal and extraintestinal manifestations. Treatment requires strict adherence to a gluten-free diet. The Internet is a major source of health information about celiac disease. Nonetheless, information about celiac disease that is available on various websites often is questioned by patients and other health care professionals regarding its reliability and content.

Objectives: To determine the accuracy, comprehensiveness, transparency, and readability of information on 100 of the most widely accessed websites that provide information on celiac disease.

Methods: Using the search term celiac disease, we analyzed 100 of the top English-language websites published by academic, commercial, nonprofit, and other professional (nonacademic) sources for accuracy, comprehensiveness, transparency, and reading grade level. Each site was assessed independently by 3 reviewers. Website accuracy and comprehensiveness were probed independently using a set of objective core information about celiac disease. We used 19 general criteria to assess website transparency. Website readability was determined by the Flesch-Kincaid reading grade level. Results for each parameter were analyzed independently. In addition, we weighted and combined parameters to generate an overall score, termed website quality.

Results: We included 98 websites in the final analysis. Of these, 47 (48%) provided specific information about celiac disease that was less than 95% accurate (ie, the predetermined cut-off considered a minimum acceptable level of accuracy). Independent of whether the information posted was accurate, 51 of 98 (52%) websites contained less than 50% of the core celiac disease information that was considered important for inclusion on websites that provide general information about celiac disease. Academic websites were significantly less transparent (P = .005) than commercial websites in attributing authorship, timeliness of information, sources of information, and other important disclosures. The type of website publisher did not predict website accuracy, comprehensiveness, or overall website quality. Only 4 of 98 (4%) websites achieved an overall quality score of 80 or above, which a priori was set as the minimum score for a website to be judged trustworthy and reliable.

Conclusions: The information on many websites addressing celiac disease was not sufficiently accurate, comprehensive, and transparent, or presented at an appropriate reading grade level, to be considered sufficiently trustworthy and reliable for patients, health care providers, celiac disease support groups, and the general public. This has the potential to adversely affect decision making about important aspects of celiac disease, including its appropriate and proper diagnosis, treatment, and management.

Keywords: Celiac disease; health information; website accuracy; website comprehensiveness; website quality; website transparency.

Conflict of interest statement

Conflicts of Interest: The Warren Center publishes a website that provides information about celiac disease. This website scored among the top 4 websites in terms of the measure of quality. The authors disclose that the individuals who developed the Wm. K Warren Medical Research Center website did not participate in the design of the evaluation tools, data collection, and data analysis of this study and had no contact with the website reviewers. One website reviewer (SPO) was a University of California, San Diego medical student. None of the website reviewers had a prior or subsequent association with the Wm. K. Warren Medical Research Center or prior familiarity with its website. MFK is a member of the medical advisory board of the Celiac Disease Foundation, whose website also ranked among the top 4 websites in terms of the measure of quality. MFK played no role in the design or contents of materials presented on the Celiac Disease Foundation website.

Figures

Figure 1
Figure 1
Distribution of accuracy (A), comprehensiveness (B), transparency (C), reading grade level (D), PageRank (E), and quality scores (F) for the studied websites. The bottom and top of the box-and-whisker plots represent the 25th and 75th percentiles (ie, lower and upper quartiles, respectively). Crossbar is the median (50th percentile). Ends of whiskers represent data within 1.5 times the interquartile range of the lower and upper quartiles. Data not included between whiskers are shown as outliers (small circle). Spacing between the parts of the box indicates degree of dispersion and skewness of the data. Panel A, F3,94 = 0.70; Panel B, F3,94 = 0.34; Panel C, F3,94 = 4.07; panel D, F3,90 = 1.00; panel E, F3,87 = 2.71; panel F, F3,94 = 0.91.
Figure 2
Figure 2
Matrix of scatter plots of accuracy, comprehensiveness, and transparency of the websites. The correlation coefficients (r) and significance values (P) are shown within the boxes. There was a significant positive correlation between website accuracy and comprehensiveness (r = .25, P = .01) (dot plot shown in left middle panel) and between website comprehensiveness and transparency (r = .26, P = .01) (dot plot shown in bottom middle panel). There was no significant correlation between accuracy and transparency (r = -.04) (dot plot shown in left bottom panel).
Figure 3
Figure 3
Scatter plots of test-retest scores (left panels), and line plots of interrater reliability (right panels). Each dot on the left set of panels shows the test score (horizontal axis) and retest score (vertical axis) for a single website. For the right set of panels, each website is represented by a single gray line. There is a statistically significant positive correlation between the test and retest scores for accuracy (r = .75, 95% confidence interval [CI] .52-.88), comprehensiveness (r = 0.94, 95% CI .87-.97), and transparency (r = .79, 95% CI .59-.90). Interrater reliability statistics show statistically significant intraclass correlation coefficients among the 3 reviewers for the scoring of accuracy (r = .68, 95% CI .6-.8), comprehensiveness (r = .85, 95% CI .8-.9), and transparency (r = .61, 95% CI .5-.7).

References

    1. Fox S. Online health search 2006. Washington, DC: Pew Internet & American Life Project; 2006. Oct 29, [2012-03-01]. .
    1. Morahan-Martin JM. How internet users find, evaluate, and use online health information: a cross-cultural review. Cyberpsychol Behav. 2004 Oct;7(5):497–510.
    1. Cima RR, Anderson KJ, Larson DW, Dozois EJ, Hassan I, Sandborn WJ, Loftus EV, Pemberton JH. Internet use by patients in an inflammatory bowel disease specialty clinic. Inflamm Bowel Dis. 2007 Oct;13(10):1266–70. doi: 10.1002/ibd.20198.
    1. Alarcón O, Baudet JS, Sánchez Del Río A, Dorta MC, De La Torre M, Socas MR, Blasco P. Internet use to obtain health information among patients attending a digestive diseases office. Gastroenterol Hepatol. 2006 May;29(5):286–90.
    1. Halpert A, Dalton CB, Palsson O, Morris C, Hu Y, Bangdiwala S, Hankins J, Norton N, Drossman DA. Irritable bowel syndrome patients' ideal expectations and recent experiences with healthcare providers: a national survey. Dig Dis Sci. 2010 Feb;55(2):375–83. doi: 10.1007/s10620-009-0855-8.
    1. Rostom A, Murray JA, Kagnoff MF. American Gastroenterological Association (AGA) Institute technical review on the diagnosis and management of celiac disease. Gastroenterology. 2006 Dec;131(6):1981–2002. doi: 10.1053/j.gastro.2006.10.004.
    1. Kagnoff MF. Celiac disease: pathogenesis of a model immunogenetic disease. J Clin Invest. 2007 Jan;117(1):41–9. doi: 10.1172/JCI30253.
    1. Fasano A, Berti I, Gerarduzzi T, Not T, Colletti RB, Drago S, Elitsur Y, Green PH, Guandalini S, Hill ID, Pietzak M, Ventura A, Thorpe M, Kryszak D, Fornaroli F, Wasserman SS, Murray JA, Horvath K. Prevalence of celiac disease in at-risk and not-at-risk groups in the United States: a large multicenter study. Arch Intern Med. 2003 Feb 10;163(3):286–92.
    1. Green PHR, Stavropoulos SN, Panagi SG, Goldstein SL, Mcmahon DJ, Absan H, Neugut AI. Characteristics of adult celiac disease in the USA: results of a national survey. Am J Gastroenterol. 2001 Jan;96(1):126–31. doi: 10.1111/j.1572-0241.2001.03462.x.
    1. Green PH. The many faces of celiac disease: clinical presentation of celiac disease in the adult population. Gastroenterology. 2005 Apr;128(4 Suppl 1):S74–8.
    1. Zipser RD, Farid M, Baisch D, Patel B, Patel D. Physician awareness of celiac disease: a need for further education. J Gen Intern Med. 2005 Jul;20(7):644–6. doi: 10.1111/j.1525-1497.2005.0107.x.
    1. Copelton DA, Valle G. "You don't need a prescription to go gluten-free": the scientific self-diagnosis of celiac disease. Soc Sci Med. 2009 Aug;69(4):623–31. doi: 10.1016/j.socscimed.2009.05.012.
    1. McClung HJ, Murray RD, Heitlinger LA. The Internet as a source for current patient information. Pediatrics. 1998 Jun;101(6):E2.
    1. Sutherland LA, Wildemuth B, Campbell MK, Haines PS. Unraveling the web: an evaluation of the content quality, usability, and readability of nutrition web sites. J Nutr Educ Behav. 2005;37(6):300–5.
    1. Berland GK, Elliott MN, Morales LS, Algazy JI, Kravitz RL, Broder MS, Kanouse DE, Muñoz JA, Puyol JA, Lara M, Watkins KE, Yang H, McGlynn EA. Health information on the Internet: accessibility, quality, and readability in English and Spanish. JAMA. 2001;285(20):2612–21.
    1. Kim J, Kim S. Physicians' perception of the effects of Internet health information on the doctor-patient relationship. Inform Health Soc Care. 2009 Sep;34(3):136–48. doi: 10.1080/17538150903102422.
    1. Crocco AG, Villasis-Keever M, Jadad AR. Analysis of cases of harm associated with use of health information on the internet. JAMA. 2002 Jun 5;287(21):2869–71.
    1. Crocco AG, Villasis-Keever M, Jadad AR. Two wrongs don't make a right: harm aggravated by inaccurate information on the Internet. Pediatrics. 2002 Mar;109(3):522–3.
    1. Murray E, Lo B, Pollack L, Donelan K, Catania J, Lee K, Zapert K, Turner R. The impact of health information on the Internet on health care and the physician-patient relationship: national U.S. survey among 1.050 U.S. physicians. J Med Internet Res. 2003;5(3):e17. doi: 10.2196/jmir.5.3.e17.
    1. Iverson SA, Howard KB, Penney BK. Impact of internet use on health-related behaviors and the patient-physician relationship: a survey-based study and review. J Am Osteopath Assoc. 2008 Dec;108(12):699–711.
    1. England CY, Nicholls AM. Advice available on the Internet for people with coeliac disease: an evaluation of the quality of websites. J Hum Nutr Diet. 2004 Dec;17(6):547–59. doi: 10.1111/j.1365-277X.2004.00561.x.
    1. nielsenwire. 2010. Sep 14, [2012-03-01]. Bing Overtakes Yahoo! as the #2 U.S. Search Engine .
    1. Goldman E. Search engine bias and the demise of search engine utopianism. Yale J Law Technol. 2005;8:188–200.
    1. Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor--Let the reader and viewer beware. JAMA. 1997 Apr 16;277(15):1244–5.
    1. Boyer C, Selby M, Scherrer JR, Appel RD. The Health On the Net Code of Conduct for medical and health Websites. Comput Biol Med. 1998 Sep;28(5):603–10.
    1. Oermann MH. Using health web sites for patient education. J Wound Ostomy Continence Nurs. 2003 Jul;30(4):217–23. doi: 10.1067/mjw.2003.136.
    1. Provost M, Koompalum D, Dong D, Martin BC. The initial development of the WebMedQual scale: domain assessment of the construct of quality of health web sites. Int J Med Inform. 2006 Jan;75(1):42–57. doi: 10.1016/j.ijmedinf.2005.07.034.
    1. Bernstam EV, Walji MF, Sagaram S, Sagaram D, Johnson CW, Meric-Bernstam F. Commonly cited website quality criteria are not effective at identifying inaccurate online information about breast cancer. Cancer. 2008 Mar 15;112(6):1206–13. doi: 10.1002/cncr.23308.
    1. Hanif F, Read JC, Goodacre JA, Chaudhry A, Gibbs P. The role of quality tools in assessing reliability of the internet for health information. Inform Health Soc Care. 2009 Dec;34(4):231–43. doi: 10.3109/17538150903359030.
    1. Cotugna N, Vickery CE, Carpenter-Haefele KM. Evaluation of literacy level of patient education pages in health-related journals. J Community Health. 2005 Jun;30(3):213–9.
    1. Bates BR, Romina SM, Ahmed R. The effect of improved readability scores on consumers' perceptions of the quality of health information on the internet. J Cancer Educ. 2007;22(1):15–20. doi: 10.1080/08858190701348067.
    1. Moult B, Franck LS, Brady H. Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information. Health Expect. 2004 Jun;7(2):165–75. doi: 10.1111/j.1369-7625.2004.00273.x.
    1. Wikipedia. 2011. Oct 23, [2011-11-15]. Flesch-Kincaid Readabiity Test .
    1. . [2011-11-16]. Tests Document Readability: Readability Calculator. _improve .
    1. Altman A, Tennenholtz. M Ranking systems: the PageRank axioms. Proceedings; 6th ACM conference on Electronic Commerce (EC-05); Jun 5-8, 2005; Vancouver, BC, Canada. 2005.
    1. Holm S. A simple sequentially rejective multiple test procedure. Scand J Stat. 1979;6:65–70.
    1. Wikipedia. 2011. Oct 27, [2011-11-16]. Box Plot .
    1. Frigge M, Hoaglin. DC. Iglewicz B. Some implementations of the boxplot. Am Stat. 1989;43(1):50–4.
    1. Cleveland WS. Robust locally weighted regression and smoothing scatterplots. J Am Stat Assoc. 1979;74:829–36.
    1. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979 Mar;86(2):420–8.
    1. Kirsch IS, Jungeblut A, Jenkins L, Kolstad A. Adult Literacy in America. Washington, DC: National Center for Education Statistics, US Department of Education; 1993.
    1. Charnock D, Shepperd S. Radcliffe online. [2011-11-16]. Discern Online
    1. Tangri V, Chande N. Quality of Internet-based information on gastrointestinal diseases. Can J Gastroenterol. 2011 Feb;25(2):93–6.
    1. Rees CE, Ford JE, Sheard CE. Evaluating the reliability of DISCERN: a tool for assessing the quality of written patient information on treatment choices. Patient Educ Couns. 2002 Jul;47(3):273–5.
    1. Hsu WC, Bath PA. Development of a patient-oriented tool for evaluating the quality of breast cancer information on the internet. Stud Health Technol Inform. 2008;136:297–302.
    1. Ademiluyi G, Rees CE, Sheard CE. Evaluating the reliability and validity of three tools to assess the quality of health information on the Internet. Patient Educ Couns. 2003 Jun;50(2):151–5.
    1. Eysenbach G, Powell J, Kuss O, Sa ER. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA. 2002;287(20):2691–700.
    1. Craigie M, Loader B, Burrows R, Muncer S. Reliability of health information on the Internet: an examination of experts' ratings. J Med Internet Res. 2002;4(1):e2. doi: 10.2196/jmir.4.1.e2.

Source: PubMed

스폰서 및 공동 작업자

건강 상태

약물 개입

3
구독하다