Three randomized controlled trials evaluating the impact of "spin" in health news stories reporting studies of pharmacologic treatments on patients'/caregivers' interpretation of treatment benefit

Isabelle Boutron, Romana Haneef, Amélie Yavchitz, Gabriel Baron, John Novack, Ivan Oransky, Gary Schwitzer, Philippe Ravaud, Isabelle Boutron, Romana Haneef, Amélie Yavchitz, Gabriel Baron, John Novack, Ivan Oransky, Gary Schwitzer, Philippe Ravaud

Abstract

Background: News stories represent an important source of information. We aimed to evaluate the impact of "spin" (i.e., misrepresentation of study results) in health news stories reporting studies of pharmacologic treatments on patients'/caregivers' interpretation of treatment benefit.

Methods: We conducted three two-arm, parallel-group, Internet-based randomized trials (RCTs) comparing the interpretation of news stories reported with or without spin. Each RCT considered news stories reporting a different type of study: (1) pre-clinical study, (2) phase I/II non-RCT, and (3) phase III/IV RCT. For each type of study, we identified news stories reported with spin that had earned mention in the press. Two versions of the news stories were used: the version with spin and a version rewritten without spin. Participants were patients/caregivers involved in Inspire, a large online community of more than one million patients/caregivers. The primary outcome was participants' interpretation assessed by one specific question "What do you think is the probability that 'treatment X' would be beneficial to patients?" (scale, 0 [very unlikely] to 10 [very likely]).

Results: For each RCT, 300 participants were randomly assigned to assess a news story with spin (n = 150) or without spin (n = 150), and 900 participants assessed a news story. Participants were more likely to consider that the treatment would be beneficial to patients when the news story was reported with spin. The mean (SD) score for the primary outcome for abstracts reported with and without spin for pre-clinical studies was 7.5 (2.2) versus 5.8 (2.8) (mean difference [95% CI] 1.7 [1.0-2.3], p < 0.001); for phase I/II non-randomized trials, 7.6 (2.2) versus 5.8 (2.7) (mean difference 1.8 [1.0-2.5], p < 0.001); and for phase III/IV RCTs, 7.2 (2.3) versus 4.9 (2.8) (mean difference 2.3 [1.4-3.2], p < 0.001).

Conclusions: Spin in health news stories reporting studies of pharmacologic treatments affects patients'/caregivers' interpretation.

Trial registration: ClinicalTrials.gov, NCT03094078 , NCT03094104 , NCT03095586.

Keywords: Detrimental research practices; Distorted interpretation; Randomized trial; Spin.

Conflict of interest statement

IB is an academic editor at BMC Medicine. IO is employed by Medscape, which is part of WebMD. He is also the volunteer co-founder of Retraction Watch, which has been paid writing fees for journalism by Science magazine, STAT News, The Boston Globe, and other publications, and has in the past received grants from the Laura and John Arnold Foundation, the Helmsley Trust, and the John D. and Catherine T. MacArthur Foundation. All other authors declare that they have no competing interests.

Figures

Fig. 1
Fig. 1
Flow of participants in the study
Fig. 2
Fig. 2
Participants’ interpretation of the benefit of treatments when reading a news story reported with or without spin. Scores are based on a numerical rating scale, ranging from 0 (very unlikely) to 10 (very likely). Boxes represent median observations (horizontal rule) with 25th and 75th percentiles of observed data (box edges). The diamonds represent the mean. The error bars represent the minimum and maximum values. RCTs, randomized controlled trials
Fig. 3
Fig. 3
Forest plot of the results for primary and secondary outcomes

References

    1. Sharma V, Dowd MD, Swanson DS, Slaughter AJ, Simon SD. Influence of the news media on diagnostic testing in the emergency department. Arch Pediatr Adolesc Med. 2003;157(3):257–260. doi: 10.1001/archpedi.157.3.257.
    1. Grilli R, Ramsay C, Minozzi S: Mass media interventions: effects on health services utilisation. Cochrane Database Syst Rev 2002(1):CD000389.
    1. Chen X, Siu LL. Impact of the media and the internet on oncology: survey of cancer patients and oncologists in Canada. J Clin Oncol. 2001;19(23):4291–4297. doi: 10.1200/JCO.2001.19.23.4291.
    1. Digital News Fact Sheet. In. Edited by PewResearchCenter. 2018. . Accessed 3 May 2019.
    1. Mitchell A, Shearer JE, Gottfried E, Barthel M: The modern news consumer. In. Edited by PewResearchCenter; 2016. . Accessed 3 May 2019.
    1. Shearer JE, Gottfried E, Lu K: How Americans encounter, recall and act upon digital news. In. Edited by PewResearchCenter. 2017. . Accessed 3 May 2019.
    1. Moynihan R, Bero L, Ross-Degnan D, Henry D, Lee K, Watkins J, Mah C, Soumerai SB. Coverage by the news media of the benefits and risks of medications. N Engl J Med. 2000;342(22):1645–1650. doi: 10.1056/NEJM200006013422206.
    1. Boutron I, Ravaud P. Misrepresentation and distortion of research in biomedical literature. Proc Natl Acad Sci U S A. 2018;115(11):2613–2619. doi: 10.1073/pnas.1710755115.
    1. Yavchitz A, Boutron I, Bafeta A, Marroun I, Charles P, Mantz J, Ravaud P. Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study. PLoS Med. 2012;9(9):e1001308. doi: 10.1371/journal.pmed.1001308.
    1. Hochman M, Hochman S, Bor D, McCormick D. News media coverage of medication research: reporting pharmaceutical company funding and use of generic medication names. Jama. 2008;300(13):1544–1550. doi: 10.1001/jama.300.13.1544.
    1. Schwitzer G. How do US journalists cover treatments, tests, products, and procedures? An evaluation of 500 stories. PLoS Med. 2008;5(5):e95. doi: 10.1371/journal.pmed.0050095.
    1. English Oxford Living Dictionaries []. Accessed 3 May 2019.
    1. Haneef R, Lazarus C, Ravaud P, Yavchitz A, Boutron I. Interpretation of results of studies evaluating an intervention highlighted in Google Health News: a cross-sectional study of news. PLoS One. 2015;10(10):e0140889. doi: 10.1371/journal.pone.0140889.
    1. Sumner P, Vivian-Griffiths S, Boivin J, Williams A, Venetis CA, Davies A, Ogden J, Whelan L, Hughes B, Dalton B, et al. The association between exaggeration in health related science news and academic press releases: retrospective observational study. BMJ. 2014;349:g7015. doi: 10.1136/bmj.g7015.
    1. Sumner P, Vivian-Griffiths S, Boivin J, Williams A, Bott L, Adams R, Venetis CA, Whelan L, Hughes B, Chambers CD. Exaggerations and caveats in press releases and health-related science news. PLoS One. 2016;11(12):e0168217. doi: 10.1371/journal.pone.0168217.
    1. Bubela T. Science communication in transition: genomics hype, public engagement, education and commercialization pressures. Clin Genet. 2006;70(5):445–450. doi: 10.1111/j.1399-0004.2006.00693.x.
    1. Bubela TM, Caulfield TA. Do the print media “hype” genetic research? A comparison of newspaper stories and peer-reviewed research papers. CMAJ. 2004;170(9):1399–1407. doi: 10.1503/cmaj.1030762.
    1. Boutron I, Altman DG, Hopewell S, Vera-Badillo F, Tannock I, Ravaud P. Impact of spin in the abstracts of articles reporting results of randomized controlled trials in the field of cancer: the SPIIN randomized controlled trial. J Clin Oncol. 2014;32(36):4120–4126. doi: 10.1200/JCO.2014.56.7503.
    1. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. Jama. 2010;303(20):2058–2064. doi: 10.1001/jama.2010.651.
    1. Haneef R, Yavchitz A, Ravaud P, Baron G, Oransky I, Schwitzer G, Boutron I. Interpretation of health news items reported with or without spin: protocol for a prospective meta-analysis of 16 randomised controlled trials. BMJ Open. 2017;7(11):e017425. doi: 10.1136/bmjopen-2017-017425.
    1. Ochodo EA, de Haan MC, Reitsma JB, Hooft L, Bossuyt PM, Leeflang MM. Overinterpretation and misreporting of diagnostic accuracy studies: evidence of “spin”. Radiology. 2013;267(2):581–588. doi: 10.1148/radiol.12120527.
    1. Lazarus C, Haneef R, Ravaud P, Boutron I. Classification and prevalence of spin in abstracts of non-randomized studies evaluating an intervention. BMC Med Res Methodol. 2015;15:85. doi: 10.1186/s12874-015-0079-x.
    1. Yavchitz A, Ravaud P, Altman DG, Moher D, Hrobjartsson A, Lasserson T, Boutron I. A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity. J Clin Epidemiol. 2016;75:56–65. doi: 10.1016/j.jclinepi.2016.01.020.
    1. Schwitzer G. Addressing tensions when popular media and evidence-based care collide. BMC Med Inform Decis Mak. 2013;13(Suppl 3):S3. doi: 10.1186/1472-6947-13-S3-S3.
    1. Schwitzer G. A guide to reading health care news stories. JAMA Intern Med. 2014;174(7):1183–1186. doi: 10.1001/jamainternmed.2014.1359.
    1. Schwartz LM, Woloshin S, Andrews A, Stukel TA. Influence of medical journal press releases on the quality of associated newspaper coverage: retrospective cohort study. BMJ. 2012;344:d8164. doi: 10.1136/bmj.d8164.
    1. Bubela T, Nisbet MC, Borchelt R, Brunger F, Critchley C, Einsiedel E, Geller G, Gupta A, Hampel J, Hyde-Lay R, et al. Science communication reconsidered. Nat Biotechnol. 2009;27(6):514–518. doi: 10.1038/nbt0609-514.
    1. Scheufele Dietram A., Krause Nicole M. Science audiences, misinformation, and fake news. Proceedings of the National Academy of Sciences. 2019;116(16):7662–7669. doi: 10.1073/pnas.1805871115.
    1. Caulfield T, Condit C. Science and the sources of hype. Public Health Genomics. 2012;15(3–4):209–217. doi: 10.1159/000336533.
    1. Matthews A, Herrett E, Gasparrini A, Van Staa T, Goldacre B, Smeeth L, Bhaskaran K. Impact of statin related media coverage on use of statins: interrupted time series analysis with UK primary care data. BMJ. 2016;353:i3283. doi: 10.1136/bmj.i3283.
    1. Schwitzer G. Statins, news, and nuance. BMJ. 2016;353:i3379. doi: 10.1136/bmj.i3379.
    1. Fornai F, Longone P, Cafaro L, Kastsiuchenka O, Ferrucci M, Manca ML, Lazzeri G, Spalloni A, Bellio N, Lenzi P, et al. Lithium delays progression of amyotrophic lateral sclerosis. Proc Natl Acad Sci U S A. 2008;105(6):2052–2057. doi: 10.1073/pnas.0708022105.
    1. Wicks P, Vaughan TE, Massagli MP, Heywood J. Accelerated clinical discovery using self-reported patient data collected online and a patient-matching algorithm. Nat Biotechnol. 2011;29(5):411–414. doi: 10.1038/nbt.1837.
    1. The Personal News Cycle. In. Edited by Associated API, Research P-NCfPA . Accessed 3 May 2019.
    1. Oransky I. How publish or perish promotes inaccuracy in science -- and journalism. AMA J Ethics. 2015;17(12):1172–1175. doi: 10.1001/journalofethics.2015.17.12.sect1-1512.
    1. Nisbet MC, Mooney C. Science and society. Framing science. Science. 2007;316(5821):56. doi: 10.1126/science.1142030.
    1. Heathnewsreview Toolkit. In., vol. 2018. . Accessed 3 May 2019.
    1. National Academies of Sciences, Engineering, and Medicine. Communicating Science Effectively: A Research Agenda. Washington, DC: The National Academies Press; 2017. 10.17226/23674.

Source: PubMed

Sponsoren und Mitarbeiter

Krankheiten

Drogeninterventionen

3
Abonnieren