Development of the Motivational Interviewing Coach Rating Scale (MI-CRS) for health equity implementation contexts

Abstract

Objective: The field of implementation science emphasizes efficient and effective fidelity measurement for research outcomes and feedback to support quality improvement. This paper reports on such a measure for motivational interviewing (MI), developed with rigorous methodology and with diverse samples.

Method: Using item response theory (IRT) methods and Rasch modeling, we analyzed coded (a) recordings (n = 99) of intervention sessions in a clinical trial of African American adolescents with obesity; (b) standard patient interactions (n = 370) in an implementation science study with youth living with HIV; and (c) standard patient interactions (n = 172) in a diverse community sample.

Results: These methods yielded a reliable and valid 12-item scale on several indicators using Rausch modeling including single construct dimensionality, strong item-session maps, good rating scale functionality, and item fit after revisions. However, absolute agreement was modest. The 12 items yielded thresholds for 4 categories: beginner, novice, intermediate and advanced.

Conclusions: The 12-item Motivational Interviewing Coach Rating Scale is the first efficient and effective fidelity measure appropriate with diverse ethnic groups, with interventions that are MI only or interventions that integrate MI with other interventions, and with adolescents and families as well as adults. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Trial registration: ClinicalTrials.gov NCT01350531.

Figures

Figure 1

Rating scale category probability curves from the Rasch measurement model based on the Fit Families sample Note. For each category, the curve reflects the probability of endorsement (y-axis) at each level of the MI competency construct (x-axis). The x-axis reflects the difference between the level of competence for a session and the difficulty of the competence component being rated, from left to right, with more basic MI components and an increasing level of MI competence. If a session has a high level of competence and a basic component of MI is being rated, the probability of a rating in the highest category approaches 100%. Conversely, if a session has low competence and an advanced component of MI is being rated, the probability of the rating in the lowest category approaches 100%.

Figure 2

Item-session map from the Rasch measurement model based on the Fit Families sample. Note. The item-session map illustrates the distribution of session measures (i.e., “scores”) relative to the distribution of item measures. On the left side of the center dividing line are markers for a mean session (M) and sessions 1 and 2 SDs above and below the mean. On the right side of this line are markers for the mean item difficulty location and items 1 and 2 SDs above and below the mean. This leads to several criteria for evaluating performance: (1) The session and items distributions cover a range of approximately four or more logits, (2) the session mean and item mean are closely aligned, (3) there are no significant vertical “gaps” in the item distribution (i.e., a >0.50 logit range of the session distribution that is not assessed by any items at the same location), (4) there are items targeted to the full distribution of sessions, and (5) the ordering of items, from least to most difficult, should match theoretical expectations. a The session panel illustrates the distribution of session measures, with each circle representing the Rasch logit-based “score” for a single coded session. Sessions with the highest observed competence are at the top, and those with the lowest levels are at the bottom. b The item panel illustrates the distribution of item measures, with less commonly endorsed items at the top and more commonly endorsed items at the bottom. The bolded center column reflects the location of the Rasch item difficulty estimate, the Poor column is the difficulty estimate for a 50% probability of a rating in the lowest category, and the Excellent column is the estimate for a 50% probability of a rating in the highest category.