Incentive and Reminder Strategies to Improve Response Rate for Internet-Based Physician Surveys: A Randomized Experiment

David A Cook, Christopher M Wittich, Wendlyn L Daniels, Colin P West, Ann M Harris, Timothy J Beebe, David A Cook, Christopher M Wittich, Wendlyn L Daniels, Colin P West, Ann M Harris, Timothy J Beebe

Abstract

Background: Most research on how to enhance response rates in physician surveys has been done using paper surveys. Uncertainties remain regarding how to enhance response rates in Internet-based surveys.

Objective: To evaluate the impact of a low-cost nonmonetary incentive and paper mail reminders (formal letter and postcard) on response rates in Internet-based physician surveys.

Methods: We executed a factorial-design randomized experiment while conducting a nationally representative Internet-based physician survey. We invited 3966 physicians (randomly selected from a commercial database of all licensed US physicians) via email to complete an Internet-based survey. We used 2 randomly assigned email messages: one message offered a book upon survey completion, whereas the other did not mention the book but was otherwise identical. All nonrespondents received several email reminders. Some physicians were further assigned at random to receive 1 reminder via paper mail (either a postcard or a letter) or no paper reminder. The primary outcome of this study was the survey response rate.

Results: Of the 3966 physicians who were invited, 451 (11.4%) responded to at least one survey question and 336 (8.5%) completed the entire survey. Of those who were offered a book, 345/2973 (11.6%) responded compared with 106/993 (10.7%) who were not offered a book (odds ratio 1.10, 95% CI 0.87-1.38, P=.42). Regarding the paper mail reminder, 168/1572 (10.7%) letter recipients, 148/1561 (9.5%) postcard recipients, and 69/767 (9.0%) email-only recipients responded (P=.35). The response rate for those receiving letters or postcards was similar (odds ratio 1.14, 95% CI 0.91-1.44, P=.26).

Conclusions: Offering a modest nonmonetary incentive and sending a paper reminder did not improve survey response rate. Further research on how to enhance response rates in Internet-based physician surveys is needed.

Keywords: questionnaire design; survey methods; surveys and questionnaires.

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
Participant flow.
Figure 2
Figure 2
Survey response rate with different incentives and reminders. Dark gray bars show response rate by incentive (book offer or none), P=.42. Light gray bars show response rate by paper reminder (letter, postcard, or none), P=.35. Error bars indicate the 95% confidence interval for the binomial proportion.
Figure 3
Figure 3
Timeline of responses for completed surveys.The survey started on day 1. Vertical dashed lines indicate the dates of reminder emails. The paper reminder was mailed on day 7, with the intent that it would arrive on the same day as the first reminder email. Intervals with no dots indicate periods in which no additional responses were received.

References

    1. Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clin Epidemiol. 1997 Oct;50(10):1129–36.
    1. Asch S, Connor SE, Hamilton EG, Fox SA. Problems in recruiting community-based physicians for health services research. J Gen Intern Med. 2000 Aug;15(8):591–9.
    1. McLeod CC, Klabunde CN, Willis GB, Stark D. Health care provider surveys in the United States, 2000-2010: a review. Eval Health Prof. 2013 Mar;36(1):106–26. doi: 10.1177/0163278712474001.
    1. Cho YI, Johnson TP, Vangeest JB. Enhancing surveys of health care professionals: a meta-analysis of techniques to improve response. Eval Health Prof. 2013 Sep;36(3):382–407. doi: 10.1177/0163278713496425.
    1. Davern M. Nonresponse rates are a problematic indicator of nonresponse bias in survey research. Health Serv Res. 2013 Jun;48(3):905–12. doi: 10.1111/1475-6773.12070.
    1. Johnson TP, Wislar JS. Response rates and nonresponse errors in surveys. JAMA. 2012 May 2;307(17):1805–6. doi: 10.1001/jama.2012.3532.
    1. VanGeest JB, Johnson TP, Welch VL. Methodologies for improving response rates in surveys of physicians: a systematic review. Eval Health Prof. 2007 Dec;30(4):303–21. doi: 10.1177/0163278707307899.
    1. Kellerman SE, Herold J. Physician response to surveys. A review of the literature. Am J Prev Med. 2001 Jan;20(1):61–7.
    1. Thorpe C, Ryan B, McLean SL, Burt A, Stewart M, Brown JB, Reid GJ, Harris S. How to obtain excellent response rates when surveying physicians. Fam Pract. 2009 Feb;26(1):65–8. doi: 10.1093/fampra/cmn097.
    1. Klabunde CN, Willis GB, Casalino LP. Facilitators and barriers to survey participation by physicians: a call to action for researchers. Eval Health Prof. 2013 Sep;36(3):279–95. doi: 10.1177/0163278713496426.
    1. Dykema J, Jones NR, Piché T, Stevenson J. Surveying clinicians by web: current issues in design and administration. Eval Health Prof. 2013 Sep;36(3):352–81. doi: 10.1177/0163278713496630.
    1. Dykema J, Stevenson J, Day B, Sellers SL, Bonham VL. Effects of incentives and prenotification on response rates and costs in a national web survey of physicians. Eval Health Prof. 2011 Dec;34(4):434–47. doi: 10.1177/0163278711406113.
    1. Minniear TD, McIntosh EB, Alexander N, Weidle PJ, Fulton J. Using electronic surveys to gather information on physician practices during a response to a local epidemic--Rhode Island, 2011. Ann Epidemiol. 2013 Aug;23(8):521–3. doi: 10.1016/j.annepidem.2013.05.003.
    1. Schleyer TK, Forrest JL. Methods for the design and administration of web-based surveys. J Am Med Inform Assoc. 2000;7(4):416–25.
    1. Braithwaite D, Emery J, De LS, Sutton S. Using the Internet to conduct surveys of health professionals: a valid alternative? Fam Pract. 2003 Oct;20(5):545–51.
    1. Akl EA, Maroun N, Klocke RA, Montori V, Schünemann HJ. Electronic mail was not better than postal mail for surveying residents and faculty. J Clin Epidemiol. 2005 Apr;58(4):425–9. doi: 10.1016/j.jclinepi.2004.10.006.
    1. Crouch S, Robinson P, Pitts M. A comparison of general practitioner response rates to electronic and postal surveys in the setting of the National STI Prevention Program. Aust N Z J Public Health. 2011 Apr;35(2):187–9. doi: 10.1111/j.1753-6405.2011.00687.x.
    1. Leece P, Bhandari M, Sprague S, Swiontkowski MF, Schemitsch EH, Tornetta P, Devereaux PJ, Guyatt GH. Internet versus mailed questionnaires: a controlled comparison (2) J Med Internet Res. 2004 Oct 29;6(4):e39. doi: 10.2196/jmir.6.4.e39.
    1. McMahon SR, Iwamoto M, Massoudi MS, Yusuf HR, Stevenson JM, David F, Chu SY, Pickering LK. Comparison of e-mail, fax, and postal surveys of pediatricians. Pediatrics. 2003 Apr;111(4 Pt 1):e299–303.
    1. Nicholls K, Chapman K, Shaw T, Perkins A, Sullivan MM, Crutchfield S, Reed E. Enhancing response rates in physician surveys: the limited utility of electronic options. Health Serv Res. 2011 Oct;46(5):1675–82. doi: 10.1111/j.1475-6773.2011.01261.x.
    1. Raziano DB, Jayadevappa R, Valenzula D, Weiner M, Lavizzo-Mourey R. E-mail versus conventional postal mail survey of geriatric chiefs. Gerontologist. 2001 Dec;41(6):799–804.
    1. Seguin R, Godwin M, MacDonald S, McCall M. E-mail or snail mail? Randomized controlled trial on which works better for surveys. Can Fam Physician. 2004 Mar;50:414–9.
    1. Thomas M, Rogers R, Maclean R. Collecting data from physicians via web-based surveys: recommendations for improving response rates. The Internet Journal of Medical Informatics. 2002;1(1):1–6.
    1. VanDenKerkhof EG, Parlow JL, Goldstein DH, Milne B. In Canada, anesthesiologists are less likely to respond to an electronic, compared to a paper questionnaire. Can J Anaesth. 2004 May;51(5):449–54.
    1. VanGeest J, Beebe T, Johnson T. Surveys of Physicians. In: Johnson T, editor. Handbook of Survey Methods. Hoboken, NJ: John Wiley & Sons, Inc; 2014. pp. 515–543.
    1. Burns KE, Duffett M, Kho ME, Meade MO, Adhikari NK, Sinuff T, Cook DJ. A guide for the design and conduct of self-administered surveys of clinicians. CMAJ. 2008 Jul 29;179(3):245–52. doi: 10.1503/cmaj.080372.
    1. Field TS, Cadoret CA, Brown ML, Ford M, Greene SM, Hill D, Hornbrook MC, Meenan RT, White MJ, Zapka JM. Surveying physicians: do components of the “Total Design Approach” to optimizing survey response rates apply to physicians? Med Care. 2002 Jul;40(7):596–605.
    1. Klabunde CN, Willis GB, McLeod CC, Dillman DA, Johnson TP, Greene SM, Brown ML. Improving the quality of surveys of physicians and medical groups: a research agenda. Eval Health Prof. 2012 Dec;35(4):477–506. doi: 10.1177/0163278712458283.
    1. Willis GB, Smith T, Lee HJ. Do additional recontacts to increase response rate improve physician survey data quality? Med Care. 2013 Oct;51(10):945–8. doi: 10.1097/MLR.0b013e3182a5023d.
    1. Iglehart JK, Baron RB. Ensuring physicians' competence--is maintenance of certification the answer? N Engl J Med. 2012 Dec 27;367(26):2543–9. doi: 10.1056/NEJMhpr1211043.
    1. Cook DA, Holmboe ES, Sorensen KJ, Berger RA, Wilkinson JM. Getting maintenance of certification to work: a grounded theory study of physicians' perceptions. JAMA Intern Med. 2015 Jan;175(1):35–42. doi: 10.1001/jamainternmed.2014.5437.
    1. Nissen SE. Reforming the continuing medical education system. JAMA. 2015 May 12;313(18):1813–4. doi: 10.1001/jama.2015.4138.
    1. Committee on Planning a Continuing Health Care Professional Education Institute. Board on Health Care Services. Institute of Medicine . Redesigning Continuing Education in the Health Professions. Washington, DC: The National Academies Press; 2010.
    1. American Association for Public Opinion Research. 2016. [2016-08-25]. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys .
    1. Viera AJ, Edwards T. Does an offer for a free on-line continuing medical education (CME) activity increase physician survey response rate? A randomized trial. BMC Res Notes. 2012;5:129. doi: 10.1186/1756-0500-5-129.
    1. Göritz A. Using lotteries, loyalty points,other incentives to increase participant responsecompletion. In: Gosling SD, Johnson JA, editors. Advanced methods for conducting online behavioral research. Washington, DC: American Psychological Association; 2010. pp. 219–33.
    1. Göritz AS. Incentives in Web Studies: Methodological Issues and a Review. International Journal of Internet Science. 2006;1:58–70.
    1. James KM, Ziegenfuss JY, Tilburt JC, Harris AM, Beebe TJ. Getting physicians to respond: the impact of incentive type and timing on physician survey response rates. Health Serv Res. 2011 Feb;46(1 Pt 1):232–42. doi: 10.1111/j.1475-6773.2010.01181.x.
    1. Kanaan RA, Wessely SC, Armstrong D. Differential effects of pre and post-payment on neurologists' response rates to a postal survey. BMC Neurol. 2010;10:100. doi: 10.1186/1471-2377-10-100.
    1. Leung GM, Johnston JM, Saing H, Tin KY, Wong IO, Ho L. Prepayment was superior to postpayment cash incentives in a randomized postal survey among physicians. J Clin Epidemiol. 2004 Aug;57(8):777–84. doi: 10.1016/j.jclinepi.2003.12.021.
    1. Beebe TJ, Locke GR, Barnes SA, Davern ME, Anderson KJ. Mixing web and mail methods in a survey of physicians. Health Serv Res. 2007 Jun;42(3 Pt 1):1219–34. doi: 10.1111/j.1475-6773.2006.00652.x.

Source: PubMed

Sponzoři a spolupracovníci

Zdravotní podmínky

Drogové intervence

3
Předplatit