Comparing the Effectiveness of Different Appointment Reminder Methods

Comparative Effectiveness of Targeted Outreach With an Automated Caller for Increasing Appointment Completion: A Randomized Quality Improvement Initiative

Penn Medicine is continually trying to optimize operations and decrease number of patients who do not show up for their appointments. This has included new changes to text message reminders, implemented as usual care. At a baseline, less than 75% of scheduled outpatient appointments are actually completed. This results in longer wait times and decreased access for patients and operational inefficiencies. The goal of the project is to test whether supplementing standard text message appointment reminders with targeted outreach using an automated phone call to patients with increased risk of not showing up for their appointment (>15% per Epic's Risk of Patient No-Show Model) reduces no show rate (the study's primary outcome) and increases patient appointment completion rate (% of appointments that were completed during scheduled appointment time, a secondary outcome). Participants will be randomized in a 1:1 ratio to receive either the standard text message or standard text message plus the automated caller. Eligible patients have already consented to receiving text message reminders from Penn Medicine and must have an in-person appointment scheduled during the study period. The Access Optimization Group at Penn will be monitoring the randomization and outcomes reporting of whether a patient confirmed, cancelled, or no showed at the scheduled appointment. All eligible outpatient appointments over a two week period will be included in this operational evaluation. The Access Optimization Group will then make a decision on which approach to implement as usual care based on the results of this operational evaluation.

Study Overview

• Status: Completed
• Conditions: Appointment Reminders
• Intervention / Treatment: Behavioral: Text Message + Automated Caller (Intervention)

• Study Type: Interventional
• Enrollment (Actual): 32925
• Phase: Not Applicable

Contacts and Locations

Study Locations

• United States
  • Pennsylvania
    • Philadelphia, Pennsylvania, United States, 19104
      • Penn Medicine

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

Description

Inclusion Criteria:

• in-person appointments
• English or Spanish speakers
• 18 years old or older

Study Plan

How is the study designed?

Design Details

• Primary Purpose: Other
• Allocation: Randomized
• Interventional Model: Parallel Assignment
• Masking: None (Open Label)

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
No Intervention: Text Message (Usual Care); Standard text message appointment reminders to English and Spanish-speaking patients. Text reminders will go out 3-weeks, 3 days, and 24 hours before the appointment.
Active Comparator: Text Message + Automated Caller (Intervention); Standard text message appointment reminders to English and Spanish-speaking patients. Text reminders will go out 3-weeks, 3 days, and 24 hours before the appointment. If patient has not confirmed their appointment after the 3-week and 3-day reminders, they'll receive an automated caller.	Behavioral: Text Message + Automated Caller (Intervention); Standard text message appointment reminders to English and Spanish-speaking patients. Text reminders will go out 3-weeks, 3 days, and 24 hours before the appointment. If patient has not confirmed their appointment after the 3-week and 3-day reminders they will receive an automated caller.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
No Show Rate	Proportion of initially scheduled appointments in which the patient did not show.	Through study completion, on average of 3 weeks

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Appointment Completion Rate	Proportion of initial scheduled appointments completed.	Through study completion, on average of 3 weeks
Appointment Cancelation Rate	Proportion of initially scheduled appointments cancelled.	Through study completion, on average of 3 weeks
Appointment Refill Rate	Proportion of cancelled appointments that were refilled.	Through study completion, on average of 3 weeks
Refilled Appointment Completion Ration	Proportion of refilled appointments that were completed.	Through study completion, on average of 3 weeks
Net Appointment Completion Rate	Proportion of appointment slots resulting in a completed appointment. Includes completed refilled appointments.	Through study completion, on average of 3 weeks

Collaborators and Investigators

• Sponsor: M. Kit Delgado, MD

Publications and helpful links

General Publications

• Tarabichi Y, Higginbotham J, Riley N, Kaelber DC, Watts B. Reducing Disparities in No Show Rates Using Predictive Model-Driven Live Appointment Reminders for At-Risk Patients: a Randomized Controlled Quality Improvement Initiative. J Gen Intern Med. 2023 Oct;38(13):2921-2927. doi: 10.1007/s11606-023-08209-0. Epub 2023 May 1.

Helpful Links

• Helping Patients Make It to Their Appointments Using Predictive Analytics and Personalized Phone Calls

Study record dates

Study Major Dates

• Study Start (Actual): November 5, 2024
• Primary Completion (Actual): December 5, 2024
• Study Completion (Actual): February 5, 2025

Study Registration Dates

• First Submitted: November 13, 2024
• First Submitted That Met QC Criteria: January 5, 2025
• First Posted (Actual): January 9, 2025

Study Record Updates

• Last Update Posted (Actual): April 4, 2025
• Last Update Submitted That Met QC Criteria: April 1, 2025
• Last Verified: April 1, 2025

More Information

Terms related to this study

• Other Study ID Numbers: IRB # 855257

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No