Resident Supervision Index: Assessing Feasibility and Validity

OBJECTIVES: To develop, assess feasibility, and test the validity of the Resident Supervision Index (RSI), a survey tool for medical residents designed to measure quantitatively the level of supervision the resident received while caring for an outpatient during a patient care encounter.

RESEARCH DESIGN: This is a prospective trial assessing the Residency Supervision Index (Index) applied to outpatient care encounters for content validity, test-retest reliability, and construct validity.

Study Overview

• Status: Completed
• Conditions: Healthy

Detailed Description

OBJECTIVES: To develop, assess feasibility, and test the validity of the Resident Supervision Index (RSI), a survey tool for medical residents designed to measure quantitatively the level of supervision the resident received while caring for an outpatient during a patient care encounter.

RESEARCH DESIGN: This is a prospective trial assessing the Residency Supervision Index (Index) applied to outpatient care encounters for test-retest reliability and construct validity.

METHODOLOGY: Trained interviewers administered the Index during face-to-face and in-clinic interviews with 60 consenting resident physicians and their 37 consenting attending physicians to descsribe the care they provided to 143 patients at the outpatient clinics involving 148 clinical encounters at the Loma Linda VA Medical Center.

For each encounter, data comes from administering the Resident Supervision Index to the resident and attending. Baseline data describing each subject (attending physicians and resident physicians) came from face-to-face interviews.

Test-retest reliability is assessed by re-administering the Index to residents for within 24 hours of the encounter. Concurrent validity is assessed by re-administering the Index to the attending physician responsible for the patient's care.

CLINICAL RELATIONSHIPS: The study will help our understanding of how residents at VA medical centers receive training and are supervised for the purpose of both education and patient outcomes.

IMPACT/SIGNIFICANCE: The instrument is planned for future studies to assess the association between resident supervision and training outcomes, clinical workload, patient outcomes, quality of care, and care costs.

• Study Type: Observational
• Enrollment (Actual): 97

Contacts and Locations

Study Locations

• United States
  • California
    • Loma Linda, California, United States, 92357
      • VA Loma Linda Healthcare System, Loma Linda, CA

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: Yes
• Genders Eligible for Study: All

• Sampling Method: Non-Probability Sample

Study Population

Residents

Description

Inclusion Criteria:

• Males and females
• Ages 18 and older

Exclusion Criteria:

• None

Study Plan

How is the study designed?

Number of groups / cohorts

1

Cohorts and Interventions

Group / Cohort

Residents and Attending Physicians; This group contains the residents and the attending physicians who consented to participate in the study. They participated in a survey administered by a research assistant. Both physician resident and attending physicians were administered the survey twice within 24-hours and their test-retest responses were compared for reliability. Responses to attending and resident physicians covering the care for the patient and clinical care encounter were compared for accuracy.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Intraclass Correlation Coefficient Between Physician Resident and Attending Physician.	Describes agreement between physician resident and attending physician assessment of total resident supervision time for a given patient and patient care clinical encounter.	24-hours

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Intraclass Correlation Coefficient Assessing Test-retest Reliability for Resident Physicians	Assesses the test-retest reliability of repeated responses for resident physicians only to report the total time spent during resident supervision.	24 hours
Intraclass Correlation Coefficient for Test-retest Reliability for Attending Physicians	Assesses the test-retest reliability among repeated responses to surveys administered to attending physicians only.	24 hours

Collaborators and Investigators

• Sponsor: US Department of Veterans Affairs
• Investigators: Principal Investigator: Terrell Michael Kashner, PhD JD MPH, VA Loma Linda Healthcare System, Loma Linda, CA; Principal Investigator: Dale S. Cannon, PhD, VA Salt Lake City Health Care System, Salt Lake City, UT

Publications and helpful links

General Publications

• Cannon GW, Keitz SA, Holland GJ, Chang BK, Byrne JM, Tomolo A, Aron DC, Wicker AB, Kashner TM. Factors determining medical students' and residents' satisfaction during VA-based training: findings from the VA Learners' Perceptions Survey. Acad Med. 2008 Jun;83(6):611-20. doi: 10.1097/ACM.0b013e3181722e97.
• Kashner TM, Henley SS, Golden RM, Byrne JM, Keitz SA, Cannon GW, Chang BK, Holland GJ, Aron DC, Muchmore EA, Wicker A, White H. Studying the effects of ACGME duty hours limits on resident satisfaction: results from VA learners' perceptions survey. Acad Med. 2010 Jul;85(7):1130-9. doi: 10.1097/ACM.0b013e3181e1d7e3.
• Byrne JM, Kashner M, Gilman SC, Aron DC, Cannon GW, Chang BK, Godleski L, Golden RM, Henley SS, Holland GJ, Kaminetzky CP, Keitz SA, Kirsh S, Muchmore EA, Wicker AB. Measuring the intensity of resident supervision in the department of veterans affairs: the resident supervision index. Acad Med. 2010 Jul;85(7):1171-81. doi: 10.1097/ACM.0b013e3181d5a954.
• Kashner TM, Byrne JM, Henley SS, Golden RM, Aron DC, Cannon GW, Chang BK, Gilman SC, Holland GJ, Kaminetzky CP, Keitz SA, Muchmore EA, Kashner TK, Wicker AB. Measuring progressive independence with the resident supervision index: theoretical approach. J Grad Med Educ. 2010 Mar;2(1):8-16. doi: 10.4300/JGME-D-09-00083.1.
• Kashner TM, Byrne JM, Chang BK, Henley SS, Golden RM, Aron DC, Cannon GW, Gilman SC, Holland GJ, Kaminetzky CP, Keitz SA, Muchmore EA, Kashner TK, Wicker AB. Measuring progressive independence with the resident supervision index: empirical approach. J Grad Med Educ. 2010 Mar;2(1):17-30. doi: 10.4300/1949-8357-2.1.17.

Study record dates

Study Major Dates

• Study Start: March 1, 2008
• Primary Completion (Actual): December 1, 2008
• Study Completion (Actual): December 1, 2008

Study Registration Dates

• First Submitted: May 16, 2008
• First Submitted That Met QC Criteria: May 19, 2008
• First Posted (Estimate): May 20, 2008

Study Record Updates

• Last Update Posted (Estimate): April 28, 2015
• Last Update Submitted That Met QC Criteria: April 6, 2015
• Last Verified: August 1, 2014

More Information

Terms related to this study

• Keywords: Feasibility; Supervision; Assessing
• Other Study ID Numbers: SHP 08-164