Comparative Effectiveness of Telemedicine in Primary Care

Evaluating the Comparative Effectiveness of Telemedicine in Primary Care: Learning From the COVID-19 Pandemic

Leveraging a natural experiment approach, the investigators will examine rapidly changing telemedicine and in-person models of care during and after the COVID-19 crisis to determine whether certain patients could safely choose to continue telemedicine or telemedicine-supplemented care, rather than return to in-person care.

Study Overview

• Status: Completed
• Conditions: Hypertension; Diabetes; Asthma; Chronic Obstructive Pulmonary Disease (COPD); Congestive Heart Failure
• Intervention / Treatment: Other: Exposure to telemedicine, after the onset of the pandemic

Detailed Description

During the COVID-19 pandemic, telemedicine has quickly emerged as the primary method of providing outpatient care in many regions with shelter-in-place and social distancing policies. It is critical to understand the impact of this rapid and widespread transition from in-person to remote visits on disparities in access to primary care, especially in chronic disease where ongoing communication between providers and patients is essential. Also, these newly developed or expanded telemedicine programs vary widely, raising important questions about the effect of these differences on uptake of telemedicine among different patient populations and on patient-centered outcomes. Leveraging a natural experiment approach, the investigators will examine rapidly changing telemedicine and in-person models of care during and after the COVID-19 crisis to determine whether certain patients could safely choose to continue telemedicine or telemedicine-supplemented care, rather than return to in-person care. The overarching goals of this study are to describe the features of telemedicine programs in primary care during the COVID-19 pandemic and to use natural experiment methods to provide rigorous evidence on the effects of these programs.

PCORI has granted an extension for the final research report to October 1, 2023.

• Study Type: Observational
• Enrollment (Actual): 33100

Contacts and Locations

Study Locations

• United States
  • New York
    • Gainesville, New York, United States, 32610
      • University of Florida
    • New York, New York, United States, 10065
      • Weill Cornell Medicine
    • New York, New York, United States, 10029
      • Mount Sinai
  • North Carolina
    • Chapel Hill, North Carolina, United States, 27599
      • University of North Carolina

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 19 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: N/A

• Sampling Method: Non-Probability Sample

Study Population

The study population encompasses patients that are attributed to primary care clinics in one of the four health systems defined above. Patients are included in the study if they are ages 19 or older and received two or more outpatient visits at a participating practice during a one-year period before the COVID-19 pandemic, and had one or more of five chronic illnesses (asthma, chronic obstructive pulmonary disease (COPD), congestive heart failure (CHF), diabetes, hypertension) as defined by the Medicare Chronic Conditions Warehouse algorithm. For the claims analyses, it will be required that patients are continuously enrolled over the entire study time period.

Description

Inclusion Criteria:

• patients that are attributed to primary care clinics across four health systems in the INSIGHT (Mount Sinai Health System and Weill Cornell Medicine), OneFlorida (University of Florida Health), and STAR (University of North Carolina Health) CRNs.
• Patients received two or more outpatient visits at a participating practice during a one-year period before the COVID-19 pandemic,
• Patients had one or more of five chronic illnesses (asthma, chronic obstructive pulmonary disease (COPD), congestive heart failure (CHF), diabetes, hypertension) as defined by the Medicare Chronic Conditions Warehouse algorithm

Exclusion Criteria:

• Patients who tested COVID-positive
• Patients from hospice and palliative care practices
• Patients from osteopathic medicine practices
• Patients from pediatric practices
• Patients that did not reside in states where the four health systems were located: the New York-Tri State Area (Connecticut, New York, and New Jersey), Florida, and North Carolina.
• Patients that moved out of state (or out of the New York-Tri State Area) or who died during the study period were also excluded.
• Patients who were not continuously enrolled over the entire study period (2019-2021).

Study Plan

How is the study designed?

Design Details

• Observational Models: Cohort
• Time Perspectives: Retrospective

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
High Telemedicine; Patients in practices that had high telemedicine use, based on the percent of visits the practice delivered via telemedicine from April 2020 to December 2021 (the study post-period)	Other: Exposure to telemedicine, after the onset of the pandemic; The exposure of interest was the switch to primary care telemedicine prompted by the COVID-19 epidemic
Low Telemedicine; Patients in practices that had some telemedicine use, based on the percent of visits the practice delivered via telemedicine from April 2020 to December 2021 (the study post-period)	Other: Exposure to telemedicine, after the onset of the pandemic; The exposure of interest was the switch to primary care telemedicine prompted by the COVID-19 epidemic

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Preventable Emergency Department (ED) Admissions	Avoidable emergency department (ED) admissions will be obtained from claims data. The Effect of telemedicine on preventable emergency department admissions will be calculated using difference-in-differences methodology. The estimate coefficient of the difference-in-difference model will be reported.	Assessed per person per quarter for 3 years, data collected encompasses retrospective data from Q1 2019 to Q4 2021
Unplanned Hospital Admissions From the ED	Unplanned hospital admissions from the ED will be obtained from claims data. The effect of telemedicine on unplanned hospital admissions will be calculated using difference-in-difference methodology. The estimate coefficient will be reported.	Assessed at the quarter level for 3 years, data collected encompasses retrospective data from Q1 2019 to Q4 2021
Continuity of Care as Assessed by the Breslau Usual Provider of Care Measure	Continuity of care as assessed by the Breslau Usual Provider of Care measure. The Breslau Usual Provider of Care index is also an indicator of continuity of care, ranging from 0 to 1, where 1 represents continuity of care. The effect of telemedicine on continuity of care using the Breslau Usual Provider of Care measure will be calculated using difference-in-difference methodology. The estimate coefficient will be reported.	Assessed at the quarter level for 3 years, data collected encompasses retrospective data from Q1 2019 to Q4 2021
Number of Unplanned Hospital Admissions From the ED	Unplanned hospital admissions from the ED will be obtained from claims data	60 days after the exposure to one of the comparator arms of clinic-level telemedicine used
Continuity of Care as Assessed by the Bice-Boxerman Continuity of Care Index	Continuity of Care as Assessed by the Bice-Boxerman Continuity of Care Index. The Bice-Boxerman Continuity of Care Index is also an indicator of continuity of care, ranging from 0 to 1, where 1 represents continuity of care. The effect of telemedicine on continuity of care using the Bice-Boxerman Continuity of care index will be calculated using difference-in-difference methodology. The estimate coefficient will be reported.	Assessed at the quarter level for 3 years, data collected encompasses retrospective data from Q1 2019 to Q4 2021
Number of Unplanned Hospital Admissions From the ED	Unplanned hospital admissions from the ED will be obtained from claims data	6 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Number of Unplanned Hospital Admissions From the ED	Unplanned hospital admissions from the ED will be obtained from claims data	12 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Number of Avoidable Emergency Department (ED) Admissions	Avoidable emergency department (ED) admissions will be obtained from claims data	60 days after the exposure to one of the comparator arms of clinic-level telemedicine used
Number of Avoidable Emergency Department (ED) Admissions	Avoidable emergency department (ED) admissions will be obtained from claims data	6 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Number of Avoidable Emergency Department (ED) Admissions	Avoidable emergency department (ED) admissions will be obtained from claims data	12 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Continuity of Care as Assessed by the Bice-Boxerman Continuity of Care Index	Continuity of care will be measured using the Bice-Boxerman Continuity of Care Index. The Bice-Boxerman continuity of care (COC) index reflects the relative share of all of a patient's visits during the year that are billed by distinct providers and/or practices. The index ranges from 0 to 1, where 0 indicates that each visit involved a different provider than all other visits, and 1 that all visits were billed by a single provider, representing continuity of care.	60 days after the exposure to one of the comparator arms of clinic-level telemedicine used
Continuity of Care as Assessed by the Bice-Boxerman Continuity of Care Index	Continuity of care will be measured using the Bice-Boxerman Continuity of Care Index. The Bice-Boxerman continuity of care (COC) index reflects the relative share of all of a patient's visits during the year that are billed by distinct providers and/or practices. The index ranges from 0 to 1, where 0 indicates that each visit involved a different provider than all other visits, and 1 that all visits were billed by a single provider, representing continuity of care.	6 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Continuity of Care as Assessed by the Bice-Boxerman Continuity of Care Index	Continuity of care will be measured using the Bice-Boxerman Continuity of Care Index. The Bice-Boxerman continuity of care (COC) index reflects the relative share of all of a patient's visits during the year that are billed by distinct providers and/or practices. The index ranges from 0 to 1, where 0 indicates that each visit involved a different provider than all other visits, and 1 that all visits were billed by a single provider, representing continuity of care.	12 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Continuity of Care as Assessed by the Breslau Usual Provider of Care Measure	Continuity of care as assessed by the Breslau Usual Provider of Care measure. The Breslau Usual Provider of Care index is also an indicator of continuity of care, ranging from 0 to 1, where 1 represents continuity of care.	60 days after the exposure to one of the comparator arms of clinic-level telemedicine used
Continuity of Care as Assessed by the Breslau Usual Provider of Care Measure	Continuity of care as assessed by the Breslau Usual Provider of Care measure. The Breslau Usual Provider of Care index is also an indicator of continuity of care, ranging from 0 to 1, where 1 represents continuity of care.	6 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Continuity of Care as Assessed by the Breslau Usual Provider of Care Measure	Continuity of care as assessed by the Breslau Usual Provider of Care measure. The Breslau Usual Provider of Care index is also an indicator of continuity of care, ranging from 0 to 1, where 1 represents continuity of care.	12 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Continuity of Care as Assessed by Attendance at Follow-up Appointment	Continuity of care as assessed by attendance at follow-up appointment.	30 days after the exposure to one of the comparator arms of clinic-level telemedicine used
Continuity of Care as Assessed by Attendance at Follow-up Appointment	Continuity of care as assessed by attendance at follow-up appointment.	60 days after the exposure to one of the comparator arms of clinic-level telemedicine used
Continuity of Care as Assessed by Attendance at Follow-up Appointment	Continuity of care as assessed by attendance at follow-up appointment.	6 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Continuity of Care as Assessed by Attendance at Follow-up Appointment	Continuity of care as assessed by attendance at follow-up appointment.	12 months after the exposure to one of the comparator arms of clinic-level telemedicine used

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Evidence of Controlled Disease as Indicated by as Indicated by the National Quality Forum (NQF 0059): Diabetes: Hemoglobin A1c (HbA1c) Poor Control (>9%)	Evidence of controlled disease as indicated by as indicated by the National Quality Forum (NQF 0059): Diabetes: Hemoglobin A1c (HbA1c) Poor Control (>9%), which is the percentage of patients 18 - 75 years of age with diabetes who had hemoglobin A1c > 9.0% during the measurement period	30 days after the exposure to one of the comparator arms of clinic-level telemedicine used
Evidence of Controlled Disease as Indicated by as Indicated by the National Quality Forum (NQF 0059): Diabetes: Hemoglobin A1c (HbA1c) Poor Control (>9%)	Evidence of controlled disease as indicated by as indicated by the National Quality Forum (NQF 0059): Diabetes: Hemoglobin A1c (HbA1c) Poor Control (>9%), which is the percentage of patients 18 - 75 years of age with diabetes who had hemoglobin A1c > 9.0% during the measurement period	60 days after the exposure to one of the comparator arms of clinic-level telemedicine used
Evidence of Controlled Disease as Indicated by as Indicated by the National Quality Forum (NQF 0059): Diabetes: Hemoglobin A1c (HbA1c) Poor Control (>9%)	Evidence of controlled disease as indicated by as indicated by the National Quality Forum (NQF 0059): Diabetes: Hemoglobin A1c (HbA1c) Poor Control (>9%), which is the percentage of patients 18 - 75 years of age with diabetes who had hemoglobin A1c > 9.0% during the measurement period	6 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Evidence of Controlled Disease as Indicated by as Indicated by the National Quality Forum (NQF 0059): Diabetes: Hemoglobin A1c (HbA1c) Poor Control (>9%)	Evidence of controlled disease as indicated by as indicated by the National Quality Forum (NQF 0059): Diabetes: Hemoglobin A1c (HbA1c) Poor Control (>9%), which is the percentage of patients 18 - 75 years of age with diabetes who had hemoglobin A1c > 9.0% during the measurement period	12 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Evidence of Controlled Disease as Indicated by as Indicated by the National Quality Forum (NQF 0018): Controlling High Blood Pressure	Evidence of controlled disease as indicated by as indicated by the National Quality Forum (NQF 0018): Controlling High Blood Pressure, which is the percentage of patients 18 - 85 with hypertension diagnosis and adequate control (< 140/90 mmHg)	30 days after the exposure to one of the comparator arms of clinic-level telemedicine used
Evidence of Controlled Disease as Indicated by as Indicated by the National Quality Forum (NQF 0018): Controlling High Blood Pressure	Evidence of controlled disease as indicated by as indicated by the National Quality Forum (NQF 0018): Controlling High Blood Pressure, which is the percentage of patients 18 - 85 with hypertension diagnosis and adequate control (< 140/90 mmHg)	60 days after the exposure to one of the comparator arms of clinic-level telemedicine used
Evidence of Controlled Disease as Indicated by as Indicated by the National Quality Forum (NQF 0018): Controlling High Blood Pressure	Evidence of controlled disease as indicated by as indicated by the National Quality Forum (NQF 0018): Controlling High Blood Pressure, which is the percentage of patients 18 - 85 with hypertension diagnosis and adequate control (< 140/90 mmHg)	6 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Evidence of Controlled Disease as Indicated by as Indicated by the National Quality Forum (NQF 0018): Controlling High Blood Pressure	Evidence of controlled disease as indicated by as indicated by the National Quality Forum (NQF 0018): Controlling High Blood Pressure, which is the percentage of patients 18 - 85 with hypertension diagnosis and adequate control (< 140/90 mmHg)	12 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Days at Home	Days per month not in hospital or institutional setting	30 days after the exposure to one of the comparator arms of clinic-level telemedicine used
Days at Home	Days per month not in hospital or institutional setting	60 days after the exposure to one of the comparator arms of clinic-level telemedicine used
Days at Home	Days per month not in hospital or institutional setting	6 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Days at Home	Days per month not in hospital or institutional setting	12 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Patient Experiences Based on the Patient Satisfaction Questionnaire (PSQ-18)	Patient experiences based on the Patient Satisfaction Questionnaire (PSQ-18), which is a 5-scale questionnaire including questions on patient satisfaction, communication quality with providers and accessibility/convenience of care.	12 months after the exposure to one of the comparator arms of clinic-level telemedicine used
Ease of Use and Access to Telemedicine Based on Telehealth Usability Questionnaire (TUQ)	For individuals who accessed a telemedicine visit, we will ask questions based on the validated Telehealth Usability Questionnaire (TUQ), including the ease of use and access to the telemedicine service, quality of the interaction with the provider, and satisfaction	12 months after the exposure to one of the comparator arms of clinic-level telemedicine used

Collaborators and Investigators

• Sponsor: Weill Medical College of Cornell University
• Collaborators: Patient-Centered Outcomes Research Institute
• Investigators: Principal Investigator: Jessica Ancker, MPH, PhD, Vanderbilt University Medical Center; Principal Investigator: Rainu Kaushal, MD, MPH, Weill Medical College of Cornell University

Study record dates

Study Major Dates

• Study Start (Actual): March 15, 2021
• Primary Completion (Actual): April 1, 2022
• Study Completion (Actual): April 1, 2022

Study Registration Dates

• First Submitted: December 21, 2020
• First Submitted That Met QC Criteria: December 21, 2020
• First Posted (Actual): December 28, 2020

Study Record Updates

• Last Update Posted (Actual): September 19, 2024
• Last Update Submitted That Met QC Criteria: April 22, 2024
• Last Verified: April 1, 2024

More Information

Terms related to this study

• Keywords: primary care; telemedicine
• Additional Relevant MeSH Terms: Pathologic Processes; Heart Diseases; Cardiovascular Diseases; Respiratory Tract Diseases; Lung Diseases; Disease Attributes; Chronic Disease; Heart Failure; Lung Diseases, Obstructive; Pulmonary Disease, Chronic Obstructive
• Other Study ID Numbers: 20-12023014

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: NO

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No