Inter and intra-rater repeatability of the scoring of foot pain drawings

Benjamin D Chatterton, Sara Muller, Martin J Thomas, Hylton B Menz, Keith Rome, Edward Roddy, Benjamin D Chatterton, Sara Muller, Martin J Thomas, Hylton B Menz, Keith Rome, Edward Roddy

Abstract

Background: Foot pain drawings (manikins) are commonly used to describe foot pain location in self-report health surveys. Respondents shade the manikin where they experience pain. The manikin is then scored via a transparent overlay that divides the drawings into areas. In large population based studies they are often scored by multiple raters. A difference in how different raters score manikins (inter-rater repeatability), or in how an individual rater scores manikins over time (intra-rater repeatability) can therefore affect data quality. This study aimed to assess inter- and intra-rater repeatability of scoring of the foot manikin.

Methods: A random sample was generated of 50 respondents to a large population based survey of adults aged 50 years and older who experienced foot pain and completed a foot manikin. Manikins were initially scored by any one of six administrative staff (Rating 1). These manikins were re-scored by a second rater (Rating 2). The second rater then re-scored the manikins one week later (Rating 3). The following scores were compared: Rating 1 versus Rating 2 (inter-rater repeatability), and Rating 2 versus Rating 3 (intra-rater repeatability). A novel set of clinically relevant foot pain regions made up of one or more individual areas on the foot manikin were developed, and assessed for inter- and intra-rater repeatability.

Results: Scoring agreement of 100% (all 50 manikins) was seen in 69% (40 out of 58) of individual areas for inter-rater scoring (range 94 to 100%), and 81% (47 out of 58) of areas for intra-rater scoring (range 96 to 100%). All areas had a kappa value of ≥0.70 for inter- and intra-rater scoring. Scoring agreement of 100% was seen in 50% (10 out of 20) of pain regions for inter-rater scoring (range 96 to 100%), and 95% (19 out of 20) of regions for intra-rater scoring (range 98 to 100%). All regions had a kappa value of >0.70 for inter- and intra-rater scoring.

Conclusions: Individual and multiple raters can reliably score the foot pain manikin. In addition, our proposed regions may be used to reliably classify different patterns of foot pain using the foot manikin.

Figures

Figure 1
Figure 1
The blank foot pain manikin. Garrow AP, Silman AJ, Macfarlane GJ: The Cheshire Foot Pain and Disability Survey: a population survey assessing prevalence and associations. Pain 2004, 110:378-84. This figure has been reproduced with permission of the International Association for the Study of Pain® (IASP). The figure may NOT be reproduced for any other purpose without permission.
Figure 2
Figure 2
The areas used for scoring of the foot pain manikin.

References

    1. Hill CL, Gill TK, Menz HB, Taylor AW. Prevalence and correlates of foot pain in a population-based study: the North West Adelaide health study. J Foot Ankle Res. 2008;1:2. doi: 10.1186/1757-1146-1-2.
    1. Garrow AP, Silman AJ, Macfarlane GJ. The Cheshire Foot Pain and Disability Survey: a population survey assessing prevalence and associations. Pain. 2004;110:378–384. doi: 10.1016/j.pain.2004.04.019.
    1. Menz HB, Morris ME, Lord SR. Foot and ankle risk factors for falls in older people: a prospective study. J Gerontol A Biol Sci Med Sci. 2006;61:866–870. doi: 10.1093/gerona/61.8.866.
    1. Benvenuti F, Ferrucci L, Guralnik JM, Gangemi S, Baroni A. Foot pain and disability in older persons: an epidemiologic survey. J Am Geriatr Soc. 1995;43:479–484.
    1. Peat G, Thomas E, Wilkie R, Croft P. Multiple joint pain and lower extremity disability in middle and old age. Disabil Rehabil. 2006;28:1543–1549. doi: 10.1080/09638280600646250.
    1. Bowling A, Grundy E. Activities of daily living: changes in functional ability in three samples of elderly and very elderly people. Age Ageing. 1997;26:107–114. doi: 10.1093/ageing/26.2.107.
    1. Menz HB, Jordan KP, Roddy E, Croft PR. Characteristics of primary care consultations for musculoskeletal foot and ankle problems in the UK. Rheumatology (Oxford) 2010;49:1391–1398. doi: 10.1093/rheumatology/keq092.
    1. Gorter K, de Poel S, de Melker R, Kuyvenhoven M. Variation in diagnosis and management of common foot problems by GPs. Fam Pract. 2001;18:569–573. doi: 10.1093/fampra/18.6.569.
    1. McCarthy EM, Sheane BJ, Cunnane G. Greater focus on clinical rheumatology is required for training in internal medicine. Clin Rheumatol. 2009;28:139–143. doi: 10.1007/s10067-008-0997-7.
    1. Waller R, Manuel P, Williamson L. The Swindon foot and ankle questionnaire: is a picture worth a thousand words? ISRN Rheumatol. 2012;2012:105479.
    1. Lacey RJ, Lewis M, Jordan K, Jinks C, Sim J. Interrater reliability of scoring of pain drawings in a self-report health survey. Spine (Phila Pa 1976) 2005;30:E455–E458. doi: 10.1097/.
    1. Roddy E, Myers H, Thomas MJ, Marshall M, D’Cruz D, Menz HB, Belcher J, Muller S, Peat G. The clinical assessment study of the foot (CASF): study protocol for a prospective observational study of foot pain and foot osteoarthritis in the general population. J Foot Ankle Res. 2011;4:22. doi: 10.1186/1757-1146-4-22.
    1. Jinks C, Lewis M, Ong BN, Croft P. A brief screening tool for knee pain in primary care. 1. Validity and reliability. Rheumatology (Oxford) 2001;40:528–536. doi: 10.1093/rheumatology/40.5.528.
    1. Persson AL, Garametsos S, Pedersen J. Computer-aided surface estimation of pain drawings - intra- and inter-rater reliability. J Pain Res. 2011;4:135–141.
    1. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, Bouter LM, de Vet HC. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34–42. doi: 10.1016/j.jclinepi.2006.03.012.
    1. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–428.
    1. Margolis RB, Tait RC, Krause SJ. A rating system for use with patient pain drawings. Pain. 1986;24:57–65. doi: 10.1016/0304-3959(86)90026-6.
    1. Chan CW, Goldman S, Ilstrup DM, Kunselman AR, O’Neill PI. The pain drawing and Waddell’s nonorganic physical signs in chronic low-back pain. Spine (Phila Pa 1976) 1993;18:1717–1722. doi: 10.1097/00007632-199310000-00001.
    1. Udén A, Aström M, Bergenudd H. Pain drawings in chronic back pain. Spine (Phila Pa 1976) 1988;13:389–392. doi: 10.1097/00007632-198804000-00002.
    1. Ohnmeiss DD, Vanharanta H, Ekholm J. Relationship of pain drawings to invasive tests assessing intervertebral disc pathology. Eur Spine J. 1999;8:126–131. doi: 10.1007/s005860050141.
    1. Ohnmeiss DD, Vanharanta H, Ekholm J. Relation between pain location and disc pathology: a study of pain drawings and CT/discography. Clin J Pain. 1999;15:210–217. doi: 10.1097/00002508-199909000-00008.
    1. Roach KE, Brown MD, Dunigan KM, Kusek CL, Walas M. Test-retest reliability of patient reports of low back pain. J Orthop Sports Phys Ther. 1997;26:253–259. doi: 10.2519/jospt.1997.26.5.253.
    1. Ohnmeiss DD. Repeatability of pain drawings in a low back pain population. Spine (Phila Pa 1976) 2000;25:980–988. doi: 10.1097/00007632-200004150-00014.
    1. Margolis RB, Chibnall JT, Tait RC. Test-retest reliability of the pain drawing instrument. Pain. 1988;33:49–51. doi: 10.1016/0304-3959(88)90202-3.

Source: PubMed

Sponsorer och medarbetare

Medicinska tillstånd

Läkemedelsinterventioner

3
Prenumerera