Does Internet Data Collection Improve Cohort Retention?

Is Internet Data Collection Superior to Traditional Methods in Retaining Participants in Longitudinal Research?

One of the biggest challenges of conducting research where the investigators follow people over a long period of time is keeping the participants involved. One of way that investigators may be able to improve participant involvement in research is to give participants the opportunity to complete their questionnaires online.

Here's how the investigators think the Internet data collection system will help to keep participants involved and ultimately help to answer important questions about the effects of nutrition during pregnancy:

1. It will allow investigators to present complicated questionnaires in a way that doesn't confuse participants. For example, instead of telling participants to skip the remainder of a question if they answered no to the first part of the question, the ePRO system does the skipping for them. The result is less confusion, fewer questions to read, and less of the participant's time.
2. It provides a convenient way for participants to keep track of where to find the questionnaire, and how much they have already completed. Participants sometimes don't have time to answer all of the investigator's questions in one sitting. The ePRO system keeps track of how much participants have already done and it's ready to keep going whenever the participant has time. It also avoids the common problem of losing the questionnaires to the household paperwork pile, or worse to the recycling bin.
3. It provides timely reminders to participants. Study participants, and especially new moms, are busy and they forget to complete the questionnaires. The ePRO system can provide timely reminders encouraging participants to respond.
4. It helps participants complete the questionnaires accurately. The ePRO system has an automatic error detection ability that will alert participants to any questions they missed or completed incorrectly.

Finding ways to keep participants involved in research is something that all researchers are concerned about, and ultimately it's a concern for all Canadians because the results of research are often used to make decisions about health care and the kinds of programs that governments provide. Losing any participant from the study reduces the ability of policy makers to take the best decisions and choices about what health services to fund with limited dollars. What the investigators propose is a study to determine whether Internet data entry really will keep participants involved in a longitudinal study. Previous studies have shown that the internet can be a useful tool for getting people to sign up for research and at least one small study showed that using Internet data entry can actually save money. But no researchers have actually tested whether it really does a better job than paper based questionnaires in terms of keeping participants involved.

In order to answer this question, the investigators will randomly assign the current Alberta Pregnancy Outcomes and Nutrition study participants to either continue receiving the paper-based questionnaire or to start receiving the web-based surveys. The investigators will follow participants over three assessment occasions (over a period of 18 months) and observe any differences between the groups in their involvement. Because the investigators are randomly assigning participants to the two groups, differences in involvement will tell them about usefulness of Internet data entry as a participant retention tool.

Study Overview

• Status: Completed
• Conditions: Internet Data Collection
• Intervention / Treatment: Other: Electronic data collection

Detailed Description

RESEARCH QUESTIONS: In longitudinal research, will participant-initiated internet data entry improve 1) participant retention and 2) data quality? RESEARCH CONTEXT: While longitudinal research may be the best tool for examining the course of human health and illness over the lifespan, the burden of participation over many years often results in loss of participants to follow-up. Such attrition reduces statistical power and threatens the internal validity and generalizability of inferences drawn from longitudinal data (Shadish et al., 2002). Because internet data collection is convenient and simplifies complex questionnaires, this method may reduce participant burden and reduce attrition. While internet data collection is already regarded as a viable tool for participant recruitment (Smith et al., 2007) and usually results in data of equivalent or better quality (e.g., factor loadings, reliabilities) than traditional paper-and-pencil methods (Vergnaud et al., 2011), internet data collection as a tool to increase participant retention within an existing longitudinal cohort has not been evaluated previously. The goal of the current project is to determine whether internet data collection increases participant retention and data quality within the APrON (Alberta Pregnancy Outcomes and Nutrition) study.

APrON is an ongoing longitudinal cohort study conducted in Calgary and Edmonton, involving more than 2000 women, 1200 of their partners, and their 2000 children. It is designed to answer questions about the impact of maternal nutrient intake and status during pregnancy on: a) maternal mental health, b) birth outcomes, and c) child health and development. The data collected from Albertans participating in health research (such as APrON) is a provincial treasure, an investment that needs to be maximized with easy, modern technology to reduce participant burden and promote better long-term follow-up. Although software is available for single questionnaire data entry (e.g., RedCap), the innovative technology the investigators plan to introduce and evaluate is a made-in-Alberta solution that automates and simplifies the collection of complex short- and long-term longitudinal data - a process that other software cannot easily achieve. This technology holds strong potential for broad application to all longitudinal health research.

Currently, APrON collects data using traditional paper questionnaires mailed out at regular intervals throughout pregnancy and as children develop over the first 3 years. At some time points, participants are also asked to attend a clinic where biological samples and physical measures are taken. The questionnaires are long and complex (whether a question needs to be answered is dependent upon responses to previous questions) and have been identified as a major time burden by study participants. In a recent survey of 1763 APrON participants, almost 65%, indicated they would prefer to complete study questionnaires via the internet - most of the remainder had no preference. Completion rate for paper questionnaires is approximately 85% when participants come to a study clinic for an in-person visit. However, when a clinic visit does not accompany the mailed questionnaires (questionnaires are completed at home and returned by postage paid envelope) response rates are ~ 65%.

RESEARCH PLAN/METHODOLOGY: Electronic Patient Reported Outcomes (ePRO) is a web-based software tool developed by the Women & Children's Health Research Institute (WCHRI) at the University of Alberta. Pilot studies have been performed at the University of Alberta, however, this study will be the first use of ePRO in a study population of this size. Questionnaires are built using Checkbox Survey Server. Automatic email invitations to participants are based on the individualized anchor dates and schedules (e.g., the questionnaire for children aged 12 months is linked to the individual child's date of birth). Participants receive links and instructions when a survey is due for completion. Reminders are automatically sent if a questionnaire is not completed within appropriate timelines (e.g., after 2 weeks). Study staff can track late and incomplete questionnaires, re-send invitations, and manually enter data when information is obtained via a paper questionnaire or telephone interview. Importantly, the system facilitates linking of data from different questionnaires and different time points.

The investigators propose a randomized trial of data collection methods. Participants will be allocated using a permuted block randomization of equal block sizes from APrON participants in Calgary to one of two groups: (1) continue with paper-based data collection; and (2) use ePRO for future data collection tasks.

IMPACT/RELEVANCE: In 2008, Alberta invested significant research funds to establish pregnancy and birth cohorts that will help develop knowledge needed to improve the health of mothers and infants. Answering health questions about developmental effects requires continued contact and follow-up with the participants every 3-6 months, which is a significant challenge, and one which most international cohorts are struggling to accomplish with paper questionnaires. What the investigators propose in this grant is to evaluate a novel approach to complex longitudinal research that enables data collection that is convenient, less burdensome, and efficient for participants. The investigators expect that the ePRO system will: 1) increase completion rate by tracking participant progress in the study, notifying them when it is time for another round of questionnaires, administering questionnaires in an efficient manner (skip non-relevant questions), and sending reminders about incomplete questionnaires, and 2) improve data quality by alerting participants to missing items.

• Study Type: Interventional
• Enrollment (Actual): 301
• Phase: Not Applicable

Contacts and Locations

Study Locations

• Canada
  • Alberta
    • Calgary, Alberta, Canada, T3B 6A8
      • Alberta Children's Hospital

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 16 years and older (Child, Adult, Older Adult)
• Accepts Healthy Volunteers: Yes
• Genders Eligible for Study: Female

Description

Inclusion Criteria:

• current participant in the Alberta Pregnancy Outcomes and Nutrition Study
• woman must have an infant less than 1 year of age

Exclusion Criteria:

• women who have formally withdrawn from the Alberta Pregnancy Outcomes and Nutrition Study

Study Plan

How is the study designed?

Design Details

• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
Experimental: Electronic data collection; The group of participants who will complete questionnaires via the internet	Other: Electronic data collection
No Intervention: Paper data collection; The group of participants who will continue to complete paper based questionnaires, as has been the practice for this longitudinal study.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Participant retention 6 months postpartum	participant retention is defined as the number of participants providing data at 6 months postpartum.	postpartum 6 months
Participant retention 12 months postpartum	participant retention is defined as the number of participants providing data at 12 months postpartum.	12 months postpartum

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Completeness of collected data 6 months postpartum	completeness of collected data is defined as the percent of completed items of all items included in the questionnaire package at 6 months postpartum.	postpartum 6 months
Completeness of collected data 12 months postpartum	completeness of collected data is defined as the percent of completed items of all items included in the questionnaire package at 12 months postpartum.	postpartum 12 months

Collaborators and Investigators

• Sponsor: University of Calgary
• Collaborators: M.S.I. Foundation
• Investigators: Principal Investigator: Gerry Giesbrecht, PhD, University of Calgary

Publications and helpful links

General Publications

• Davis LL, Broome ME, Cox RP. Maximizing retention in community-based clinical trials. J Nurs Scholarsh. 2002;34(1):47-53. doi: 10.1111/j.1547-5069.2002.00047.x.
• Vergnaud AC, Touvier M, Mejean C, Kesse-Guyot E, Pollet C, Malon A, Castetbon K, Hercberg S. Agreement between web-based and paper versions of a socio-demographic questionnaire in the NutriNet-Sante study. Int J Public Health. 2011 Aug;56(4):407-17. doi: 10.1007/s00038-011-0257-5. Epub 2011 May 3.
• Smith B, Smith TC, Gray GC, Ryan MA; Millennium Cohort Study Team. When epidemiology meets the Internet: Web-based surveys in the Millennium Cohort Study. Am J Epidemiol. 2007 Dec 1;166(11):1345-54. doi: 10.1093/aje/kwm212. Epub 2007 Aug 28.
• Welker JA. Implementation of electronic data capture systems: barriers and solutions. Contemp Clin Trials. 2007 May;28(3):329-36. doi: 10.1016/j.cct.2007.01.001. Epub 2007 Jan 11.
• Birnbaum MH. Human research and data collection via the internet. Annu Rev Psychol. 2004;55:803-32. doi: 10.1146/annurev.psych.55.090902.141601.

Study record dates

Study Major Dates

• Study Start: December 1, 2012
• Primary Completion (Actual): November 1, 2014
• Study Completion (Actual): November 1, 2014

Study Registration Dates

• First Submitted: November 20, 2012
• First Submitted That Met QC Criteria: November 29, 2012
• First Posted (Estimate): December 4, 2012

Study Record Updates

• Last Update Posted (Estimate): December 8, 2014
• Last Update Submitted That Met QC Criteria: December 4, 2014
• Last Verified: December 1, 2014

More Information

Terms related to this study

• Keywords: internet data collection; longitudinal research; cohort retention
• Other Study ID Numbers: MSI-865