Evaluating the reliability of multi-item scales: a non-parametric approach to the ordered categorical structure of data collected with the Swedish version of the Tampa Scale for Kinesiophobia and the Self-Efficacy Scale

Abstract

Objective: To compare the ability of a rank-invariant non-parametric method with that of kappa statistics to evaluate the reliability of the Swedish version of the Tampa Scale for Kinesiophobia and the Self-Efficacy Scale by identifying systematic and random disagreement. The aim was, further, to compare 2 different statistical approaches to obtain a global value from multi-item scales.

Design: A test-retest study.

Subjects: A total of 46 patients with whiplash-associated disorders were enrolled and 39 (85%) completed the test-retest assessment.

Methods: Data from the multi-item scales were summarized using both sum and median scores. Paired data were evaluated with a rank-invariant statistical method to identify systematic and random disagreement. Data were also evaluated with kappa statistics.

Results: The non-parametric approach demonstrated that the Swedish version of the Tampa Scale for Kinesiophobia and the Self-Efficacy Scale are reliable for patients with whiplash-associated disorders. In contrast to the rank-invariant method, kappa statistics provided no information on disagreement between the 2 test occasions. Median scoring improved reliability due to lack of disagreement while the sum scores method was characterized by random individual disagreement.

Conclusion: This study has increased understanding of the advantages and limitations of 2 non-parametric statistical methods and, it is hoped, will contribute to the development of reliable measurements.