Exemplar: Determining the Best Practices for EMA Studies (Exemplar)
January 9, 2023 updated by: University of Oklahoma
Ecological momentary assessment (EMA) is a measurement methodology that utilizes the repeated collection of real-time data on participants' behavior and experience in their natural environment.
While EMA allows researchers to gain valuable insights into dynamic behavioral processes, the need for frequent self-report can be burdensome and disruptive to participants.
Compliance with EMA protocols is important for accurate, unbiased sampling, yet there is no "gold standard" for EMA study design.
In the present study, the investigators aim to use a factorial design to identify the optimal components, or combinations of components, for achieving the highest compliance rates for smartphone-based ecological momentary assessments (EMAs).
Participants will be randomized into 1 of 32 conditions in a 2x2x2x2x2 design (i.e., number of questions per EMA survey [15 vs. 25] x number of EMAs per day [2 vs. 4] x EMA prompting schedule [random vs. fixed times]) x payment type [per EMA vs. based on percentage of EMAs completed] x order of EMA item question types [i.e., slider type first 2 weeks vs. Likert-type questions first 2 weeks]).
All participants will complete EMAs for 4 weeks.
The effect of each factor on EMA compliance will be examined, as well as the effects of interactions on EMA compliance.
The investigators will also examine within and between-subjects factors that influence response rates, and 2) contextual features such as environment and mood that may be associated with better compliance and participant engagement.
The results of this study will have broad applications for developing best practices guidelines for future studies utilizing EMA methodologies.
Study Overview
Status
Completed
Intervention / Treatment
Detailed Description
Ecological momentary assessment (EMA) is a measurement methodology that utilizes the repeated collection of real-time data on participants' behavior and experience in their natural environment. EMA has been used in behavioral science for years, including research examining the environmental and psychological antecedents of cigarette smoking, substance use disorders, anxiety, eating, and sleep. EMA sampling strategies are typically time-based or event-based. Time-based sampling is solicited based on a schedule; for example, a daily diary prompted every day at the same time, or at random intervals (e.g., every 3-4 hours) each day. In contrast, event- based sampling is centered around a pre-defined event and is typically initiated by the participant. For example, a participant may be asked to initiate an assessment every time the participant smokes a cigarette.
Several studies have examined the impact of EMA study design features on participant compliance. In a recent meta-analysis of compliance with EMA protocols among substance users, there was no evidence that compliance rates were associated with prompt frequency, length of assessment period, or reimbursement. Another study using a pooled dataset of 10 EMA studies found that compliance declined across days, and varied significantly depending on the time of day. In a meta-analysis of EMA studies in mental health research, a higher number of EMAs per day were associated with lower compliance, however the number of days was not. While these studies provide preliminary guidance regarding EMA study design, their results are unfortunately inconsistent, and the varied reporting across studies with respect to the definition of compliance, study design elements, and study populations make it difficult to clearly derive best practice guidelines for EMA design.
In the present study, the investigators use a factorial design to identify the optimal components, or combinations of components, for achieving the highest compliance rates for smartphone-based ecological momentary assessments (EMAs). In addition, the investigators will explore the association between EMA design features and participant compliance/response rates. A factorial design is ideal for exploring these research questions because it is statistically more efficient, as this type of design needs fewer participants to answer questions about each experimental factor of interest.6 In this study, participants will be randomized into 1 of 32 conditions (i.e., number of questions per EMA survey x number of EMAs per day x EMA prompting schedule x EMA item type x payment type) to explore:
- both within and between-subjects factors that influence response rates, and
- contextual features such as environment and mood that may be associated with better compliance and participant engagement. The results of this study will have broad applications for developing best practice guidelines for future studies utilizing EMA methodologies.
Study Type
Interventional
Enrollment (Actual)
416
Phase
- Not Applicable
Contacts and Locations
This section provides the contact details for those conducting the study, and information on where this study is being conducted.
Study Locations
-
-
Oklahoma
-
Oklahoma City, Oklahoma, United States, 73104
- University of Oklahoma Health Sciences Center
-
-
Participation Criteria
Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.
Eligibility Criteria
Ages Eligible for Study
18 years to 99 years (Adult, Older Adult)
Accepts Healthy Volunteers
No
Genders Eligible for Study
All
Description
Inclusion Criteria:
- At least 18 years of age
- Demonstrate > 6th grade English literacy level
- Possess an active Android smartphone that is compatible with the Insight (TM) platform with a data plan and an operating system version 6.0 or higher
- Agree to install the study smartphone application onto their personal phone
- Agree to complete EMAs prompted by and completed through the study smartphone application
Exclusion Criteria:
- Under 18 years of age
- Demonstrates < 6th grade English literacy level
- Does not possess an active Android smartphone that is compatible with the Insight (TM) platform with a data plan and an operating system version 6.0 or higher
- Does not agree to install the study smartphone application onto their personal phone
- Does not agree to complete EMAs prompted by and completed through the study smartphone application
Study Plan
This section provides details of the study plan, including how the study is designed and what the study is measuring.
How is the study designed?
Design Details
- Primary Purpose: Other
- Allocation: Randomized
- Interventional Model: Factorial Assignment
- Masking: None (Open Label)
Arms and Interventions
Participant Group / Arm |
Intervention / Treatment |
|---|---|
|
Experimental: Group 1
$1 Payment per EMA 2 EMAs per day 15 questions per EMA Fixed schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 2
$1 Payment per EMA 2 EMAs per day 15 questions per EMA Fixed schedule Likert first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 3
$1 Payment per EMA 2 EMAs per day 15 questions per EMA Random schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 4
$1 Payment per EMA 2 EMAs per day 15 questions per EMA Random schedule Likert first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 5
$1 Payment per EMA 2 EMAs per day 25 questions per EMA Fixed schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 6
$1 Payment per EMA 2 EMAs per day 25 questions per EMA Fixed schedule Likert first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 7
$1 Payment per EMA 2 EMAs per day 25 questions per EMA Random schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 8
$1 Payment per EMA 2 EMAs per day 25 questions per EMA Random schedule Likert first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 9
$1 Payment per EMA 4 EMAs per day 15 questions per EMA Fixed schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 10
$1 Payment per EMA 4 EMAs per day 15 questions per EMA Fixed schedule Likert first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 11
$1 Payment per EMA 4 EMAs per day 15 questions per EMA Random schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 12
$1 Payment per EMA 4 EMAs per day 15 questions per EMA Random schedule Likert first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 13
$1 Payment per EMA 4 EMAs per day 25 questions per EMA Fixed schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 14
$1 Payment per EMA 4 EMAs per day 25 questions per EMA Fixed schedule Likert first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 15
$1 Payment per EMA 4 EMAs per day 25 questions per EMA Random schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 16
$1 Payment per EMA 4 EMAs per day 25 questions per EMA Random schedule Likert first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 17
Payment by % EMAs 2 EMAs per day 15 questions per EMA Fixed schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 18
Payment by % EMAs 2 EMAs per day 15 questions per EMA Fixed schedule Likert first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 19
Payment by % EMAs 2 EMAs per day 15 questions per EMA Random schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 20
Payment by % EMAs 2 EMAs per day 15 questions per EMA Random schedule Likert first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 21
Payment by % EMAs 2 EMAs per day 25 questions per EMA Fixed schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 22
Payment by % EMAs 2 EMAs per day 25 questions per EMA Fixed schedule Likert first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 23
Payment by % EMAs 2 EMAs per day 25 questions per EMA Random schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 24
Payment by % EMAs 2 EMAs per day 25 questions per EMA Random schedule Likert first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 25
Payment by % EMAs 4 EMAs per day 15 questions per EMA Fixed schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 26
Payment by % EMAs 4 EMAs per day 15 questions per EMA Fixed schedule Likert first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 27
Payment by % EMAs 4 EMAs per day 15 questions per EMA Random schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 28
Payment by % EMAs 4 EMAs per day 15 questions per EMA Random schedule Likert first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 29
Payment by % EMAs 4 EMAs per day 25 questions per EMA Fixed schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 30
Payment by % EMAs 4 EMAs per day 25 questions per EMA Fixed schedule Likert first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 31
Payment by % EMAs 4 EMAs per day 25 questions per EMA Random schedule Slider first
|
Various settings for ecological momentary assessments
|
|
Experimental: Group 32
Payment by % EMAs 4 EMAs per day 25 questions per EMA Random schedule Likert first
|
Various settings for ecological momentary assessments
|
What is the study measuring?
Primary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Proportion of assigned daily smartphone surveys that are completed.
Time Frame: through study completion for each participant, 28 days
|
Proportion of daily smartphone surveys completed will vary based upon group assignment.
|
through study completion for each participant, 28 days
|
Secondary Outcome Measures
Outcome Measure |
Measure Description |
Time Frame |
|---|---|---|
|
Number of minutes required to complete the smartphone-based surveys.
Time Frame: through study completion for each participant, 28 days
|
Time to complete smartphone based surveys will vary based upon group assignment.
|
through study completion for each participant, 28 days
|
Collaborators and Investigators
This is where you will find people and organizations involved with this study.
Sponsor
Publications and helpful links
The person responsible for entering information about the study voluntarily provides these publications. These may be about anything related to the study.
General Publications
- Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol. 2008;4:1-32. doi: 10.1146/annurev.clinpsy.3.022806.091415.
- Stone, A. A., & Shiffman, S. (1994). Ecological momentary assessment (EMA) in behavorial medicine. Annals of Behavioral Medicine, 16(3), 199-202. https://doi.org/10.1093/abm/16.3.199
- Jones A, Remmerswaal D, Verveer I, Robinson E, Franken IHA, Wen CKF, Field M. Compliance with ecological momentary assessment protocols in substance users: a meta-analysis. Addiction. 2019 Apr;114(4):609-619. doi: 10.1111/add.14503. Epub 2018 Dec 21.
- Rintala A, Wampers M, Myin-Germeys I, Viechtbauer W. Response compliance and predictors thereof in studies using the experience sampling method. Psychol Assess. 2019 Feb;31(2):226-235. doi: 10.1037/pas0000662. Epub 2018 Nov 5.
- Vachon H, Viechtbauer W, Rintala A, Myin-Germeys I. Compliance and Retention With the Experience Sampling Method Over the Continuum of Severe Mental Disorders: Meta-Analysis and Recommendations. J Med Internet Res. 2019 Dec 6;21(12):e14475. doi: 10.2196/14475.
- Fletcher DJ, Lewis SM, Matthews JN. Factorial designs for crossover clinical trials. Stat Med. 1990 Oct;9(10):1121-9. doi: 10.1002/sim.4780091002.
Study record dates
These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.
Study Major Dates
Study Start (Actual)
November 30, 2021
Primary Completion (Actual)
October 26, 2022
Study Completion (Actual)
October 26, 2022
Study Registration Dates
First Submitted
November 10, 2021
First Submitted That Met QC Criteria
January 3, 2022
First Posted (Actual)
January 18, 2022
Study Record Updates
Last Update Posted (Estimate)
January 10, 2023
Last Update Submitted That Met QC Criteria
January 9, 2023
Last Verified
January 1, 2023
More Information
Terms related to this study
Other Study ID Numbers
- 13684
Plan for Individual participant data (IPD)
Plan to Share Individual Participant Data (IPD)?
No
Drug and device information, study documents
Studies a U.S. FDA-regulated drug product
No
Studies a U.S. FDA-regulated device product
No
This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.