Faculty Development to Improve Mini-CEX Ratings

April 25, 2008 updated by: Mayo Clinic

Effect of Faculty Development on Mini-CEX Score Reliability and Accuracy: A Randomized, Controlled Trial

We will conduct a faculty development workshop with intent to improve interrater reliability and rater accuracy. Outcomes will include interrater reliability, accuracy, and workshop evaluations.

Study Overview

Status

Completed

Conditions

Intervention / Treatment

Detailed Description

Hypothesis: An intervention for training faculty preceptors on the use of the mini-CEX will improve inter-rater reliability and classification accuracy.

Methods. We will develop a faculty development workshop aimed at improving the inter-rater reliability and accuracy of mini-CEX scores. To evaluate the workshop we will conduct a randomized, controlled trial. Preceptors will be randomly assigned to immediate or delayed workshop. The primary outcome will be inter-rater reliability of mini-CEX scores as preceptors rate videotaped resident-patient encounters before and four weeks after the workshop. We will also evaluate the accuracy of performance classifications. We will also collect mini-CEX ratings of actual resident-patient interactions one year preceding and one year following the workshop, and calculate interrater reliability.

Statistical methods. Inter-rater reliability will be determined using the intraclass correlation coefficient (ICC) and corresponding confidence intervals. ICC will be used to determine chance-corrected agreement (accuracy). Discrimination (differences in mean scores among scripted levels of performance) will also be evaluated as a measure of accuracy.

Scientific basis. Clinical performance assessments constitute a key element of resident education. The effect of rater training on the inter-rater reliability of mini-CEX scores is unknown. This workshop will utilize methods (rater error and field of reference training) shown to be effective in improving rater accuracy in non-medical fields.

Inclusion criteria. All internal medicine residency continuity clinic preceptors are eligible.

Study Type

Interventional

Enrollment (Actual)

52

Phase

  • Not Applicable

Contacts and Locations

This section provides the contact details for those conducting the study, and information on where this study is being conducted.

Study Locations

  • United States
    • Minnesota
      • Rochester, Minnesota, United States, 55905
        • Mayo Clinic

Participation Criteria

Researchers look for people who fit a certain description, called eligibility criteria. Some examples of these criteria are a person's general health condition or prior treatments.

Eligibility Criteria

Ages Eligible for Study

  • Child
  • Adult
  • Older Adult

Accepts Healthy Volunteers

No

Genders Eligible for Study

All

Description

Inclusion Criteria:

  • Internal medicine residency continuity clinic preceptor at Mayo Clinic Rochester Minnesota

Exclusion Criteria:

  • None

Study Plan

This section provides details of the study plan, including how the study is designed and what the study is measuring.

How is the study designed?

Design Details

  • Primary Purpose: Treatment
  • Allocation: Randomized
  • Interventional Model: Parallel Assignment
  • Masking: None (Open Label)

Arms and Interventions

Participant Group / Arm
Intervention / Treatment
Experimental: Workshop
Rater training workshop: rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion
Behavioral: Rater training workshop
Rater training workshop: rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion
No Intervention: Delayed workshop
No specific intervention until after follow-up assessment, then receives same workshop.

What is the study measuring?

Primary Outcome Measures

Outcome Measure
Time Frame
Interrater reliability (videotaped resident-patient encounters)
Time Frame: April-June 2006
April-June 2006

Secondary Outcome Measures

Outcome Measure
Time Frame
Rater accuracy (agreement with scripted levels of performance on videotaped resident-patient encounters): percent agreement, chance-corrected agreement, mean scores
Time Frame: April-June 2006
April-June 2006
Interrater reliability (actual resident-patient encounters)
Time Frame: March 2005 (historical) to June 2007
March 2005 (historical) to June 2007

Collaborators and Investigators

This is where you will find people and organizations involved with this study.

Sponsor

Mayo Clinic

Investigators

  • Principal Investigator: David A Cook, MD, MHPE, Mayo Clinic

Publications and helpful links

The person responsible for entering information about the study voluntarily provides these publications. These may be about anything related to the study.

General Publications

Helpful Links

Study record dates

These dates track the progress of study record and summary results submissions to ClinicalTrials.gov. Study records and reported results are reviewed by the National Library of Medicine (NLM) to make sure they meet specific quality control standards before being posted on the public website.

Study Major Dates

Study Start

March 1, 2006

Primary Completion (Actual)

June 1, 2006

Study Completion (Actual)

June 1, 2007

Study Registration Dates

First Submitted

April 23, 2008

First Submitted That Met QC Criteria

April 25, 2008

First Posted (Estimate)

April 28, 2008

Study Record Updates

Last Update Posted (Estimate)

April 28, 2008

Last Update Submitted That Met QC Criteria

April 25, 2008

Last Verified

April 1, 2008

More Information

Terms related to this study

Keywords

Other Study ID Numbers

  • 06-002730

This information was retrieved directly from the website clinicaltrials.gov without any changes. If you have any requests to change, remove or update your study details, please contact register@clinicaltrials.gov. As soon as a change is implemented on clinicaltrials.gov, this will be updated automatically on our website as well.

Subscribe