Multiple Sclerosis Performance Test: Technical Development and Usability

Jane K Rhodes, David Schindler, Stephen M Rao, Fernando Venegas, Efrosini T Bruzik, Wendy Gabel, James R Williams, Glenn A Phillips, Colleen C Mullen, Jaime L Freiburger, Lyla Mourany, Christine Reece, Deborah M Miller, Francois Bethoux, Robert A Bermel, Lauren B Krupp, Ellen M Mowry, Jay Alberts, Richard A Rudick, Jane K Rhodes, David Schindler, Stephen M Rao, Fernando Venegas, Efrosini T Bruzik, Wendy Gabel, James R Williams, Glenn A Phillips, Colleen C Mullen, Jaime L Freiburger, Lyla Mourany, Christine Reece, Deborah M Miller, Francois Bethoux, Robert A Bermel, Lauren B Krupp, Ellen M Mowry, Jay Alberts, Richard A Rudick

Abstract

Introduction: In the clinic, the assessment of patients with multiple sclerosis (MS) is typically qualitative and non-standardized.

Objectives: To describe the MS Performance Test (MSPT), an iPad Air® 2 (Apple, Cupertino, CA, USA)-based neurological assessment platform allowing patients to input relevant information without the aid of a medical technician, creating a longitudinal, clinically meaningful, digital medical record. To report results from human factor (HF) and usability studies, and the initial large-scale implementation in a practice setting.

Methods: The HF study examined use-error patterns in small groups of MS patients and healthy controls (n = 14), the usability study assessed the effectiveness of patient interaction with the tool by patients with a range of MS disability (n = 60) in a clinical setting, and the implementation study deployed the MSPT across a diverse population of patients (n = 1000) in a large MS center for routine clinical care.

Results: MSPT assessments were completed by all users in the HF study; minor changes to design were recommended. In the usability study, 73% of patients with MS completed the MSPT, with an average administration time of 32 min; 85% described their experience with the tool as satisfactory. In the initial implementation for routine care, 84% of patients with MS completed the MSPT, with an average administration time of 28 min.

Conclusion: Patients with MS with varying disability levels completed the MSPT with minimal or no supervision, resulting in comprehensive, efficient, standardized, quantitative, clinically meaningful data collection as part of routine medical care, thus allowing for large-scale, real-world evidence generation.

Funding: Biogen.

Trial registration: NCT02664324.

Keywords: Digital assessment; Functional performance outcome measures; Multiple sclerosis; Multiple sclerosis functional composite; Neuro-QoL; Neurology; PerfOs.

Figures

Fig. 1
Fig. 1
MSPT assessment tool. Upper panel iPad Air® 2 contained within the hardware case, with grid overlay that also functions as a kickstand (a), shown over the screen (b). (c) Bluetooth remote for walking speed test; (d) aluminum pegs for manual dexterity test; (e) magnetized cover for (c) and (d); (f) headphones for audio instructions; (g) power cord. Lower left panel grid overlay in the kickstand position used for all modules except the manual dexterity test. Lower right panel aluminum pegs inserted into the grid overlay for the manual dexterity test. MSPT Multiple Sclerosis Performance Test
Fig. 2
Fig. 2
MSPT PerfO modules presented to user. MSPT Multiple Sclerosis Performance Test, PerfO functional performance outcome measure
Fig. 3
Fig. 3
Diagrammatic representation of MSPT routine clinical care (careflow). MSPT Multiple Sclerosis Performance Test
Fig. 4
Fig. 4
MSPT graphic user interface for health care professional (HCP) use. A clinic staff member can log a patient into the MSPT system using the administration wrapper (step 2 in the careflow). Each patient has a unique ID and information from prior visits can be recalled so that data input is reduced on subsequent visits. Data collected from the patient are aggregated and presented to the HCP at the point of care (step 5 in the careflow). A summary dashboard was designed to provide the HCP with a comprehensive overview of the patient’s functional status. MSPT Multiple Sclerosis Performance Test
Fig. 5
Fig. 5
MSPT cloud structure and data flow. Patient (1) inputs data using MSPT software application graphic user interface (2) and files are instantaneously uploaded to the MSPT cloud, a HIPAA compliant, secure AWS environment (3). Files are transferred in JavaScript Object Notation (JSON) format. Files can be transferred to a web server, a cloud-based database or exported via a gateway to allow integration into the medical record (4). The HCP (5) can access patient data via the MSPT software application (6) or a secure web portal (7). API application programming interface, EMR electronic medical record, AWS Amazon Web Services, HCP health care professional, HIPAA Health Insurance Portability and Accountability Act, HL7 Health Level-7, MSPT Multiple Sclerosis Performance Test, SOAP Simple Object Access Protocol
Fig. 6
Fig. 6
MSPT usability study satisfaction survey. Fifty-nine participants were given a short survey at the end of usability testing, probing overall satisfaction with the MSPT tool. Responses were scored and averages are presented as percentages. CST contrast sensitivity test, MDT manual dexterity test, MSPT Multiple Sclerosis Performance Test

References

    1. Inusah S, Sormani MP, Cofield SS, et al. Assessing changes in relapse rates in multiple sclerosis. Mult Scler. 2010;16(12):1414–1421. doi: 10.1177/1352458510379246.
    1. Karabudak R, Dahdaleh M, Aljumah M, et al. Functional clinical outcomes in multiple sclerosis: current status and future prospects. Mult Scler Relat Disord. 2015;4(3):192–201. doi: 10.1016/j.msard.2015.03.004.
    1. Meyer-Moock S, Feng YS, Maeurer M, Dippel FW, Kohlmann T. Systematic literature review and validity evaluation of the Expanded Disability Status Scale (EDSS) and the Multiple Sclerosis Functional Composite (MSFC) in patients with multiple sclerosis. BMC Neurol. 2014;14:58. doi: 10.1186/1471-2377-14-58.
    1. Lublin FD, Reingold SC, Cohen JA, et al. Defining the clinical course of multiple sclerosis: the 2013 revisions. Neurology. 2014;83(3):278–286. doi: 10.1212/WNL.0000000000000560.
    1. Rudick RA, Cutter G, Reingold S. The multiple sclerosis functional composite: a new clinical outcome measure for multiple sclerosis trials. Mult Scler. 2002;8(5):359–365. doi: 10.1191/1352458502ms845oa.
    1. Ontaneda D, LaRocca N, Coetzee T, Rudick R, NMSS MSFC Task Force Revisiting the multiple sclerosis functional composite: proceedings from the National Multiple Sclerosis Society (NMSS) task force on clinical disability measures. Mult Scler. 2012;18(8):1074–1080. doi: 10.1177/1352458512451512.
    1. Balcer LJ, Raynowska J, Nolan R, et al. Validity of low-contrast letter acuity as a visual performance outcome measure for multiple sclerosis. Mult Scler. 2017;23(5):734–747. doi: 10.1177/1352458517690822.
    1. Rudick RA, Miller D, Bethoux F, et al. The Multiple Sclerosis Performance Test (MSPT): an iPad-based disability assessment tool. J Vis Exp. 2014;88:e51318.
    1. Motl RW, Cohen JA, Benedict R, et al. Validity of the timed 25-foot walk as an ambulatory performance outcome measure for multiple sclerosis. Mult Scler. 2017;23(5):704–710. doi: 10.1177/1352458517690823.
    1. Benedict RH, DeLuca J, Phillips G, et al. Validity of the Symbol Digit Modalities Test as a cognition performance outcome measure for multiple sclerosis. Mult Scler. 2017;23(5):721–733. doi: 10.1177/1352458517690821.
    1. Feys P, Lamers I, Francis G, et al. The Nine-Hole Peg Test as a manual dexterity performance measure for multiple sclerosis. Mult Scler. 2017;23(5):711–720. doi: 10.1177/1352458517690824.
    1. National Multiple Sclerosis Society. Multiple sclerosis functional composite (MSFC) scoring and training manual, revised. 2001. . Accessed 18 Dec 2018.
    1. Rosato R, Testa S, Oggero A, Molinengo G, Bertolotto A. Quality of life and patient preferences: identification of subgroups of multiple sclerosis patients. Qual Life Res. 2015;24(9):2173–2182. doi: 10.1007/s11136-015-0952-4.
    1. Miller DM, Allen R. Quality of life in multiple sclerosis: determinants, measurement, and use in clinical practice. Curr Neurol Neurosci Rep. 2010;10(5):397–406. doi: 10.1007/s11910-010-0132-4.
    1. Smith A. Symbol digit modalities test. Los Angeles: Western Psychological Services; 1973.
    1. Rao SM, NMSS Cognitive Function Study Group . A manual for the brief repeatable batter of neuropsychological test in multiple sclerosis. New York: National Multiple Sclerosis Society; 1990.
    1. Baier ML, Cutter GR, Rudick RA, et al. Low-contrast letter acuity testing captures visual dysfunction in patients with multiple sclerosis. Neurology. 2005;64(6):992–995. doi: 10.1212/01.WNL.0000154521.40686.63.
    1. Kraft GH, Amtmann D, Bennett SE, et al. Assessment of upper extremity function in multiple sclerosis: review and opinion. Postgrad Med. 2014;126(5):102–108. doi: 10.3810/pgm.2014.09.2803.
    1. Kaufman M, Moyer D, Norton J. The significant change for the Timed 25-foot Walk in the multiple sclerosis functional composite. Mult Scler. 2000;6(4):286–290. doi: 10.1177/135245850000600411.
    1. Rao SM, Losinski G, Mourany L, et al. Processing speed test: validation of a self-administered, iPad®-based tool for screening cognitive dysfunction in a clinic setting. Mult Scler. 2017;23(14):1929–1937. doi: 10.1177/1352458516688955.
    1. Cella D, Lai JS, Nowinski CJ, et al. Neuro-QOL: brief measures of health-related quality of life for clinical research in neurology. Neurology. 2012;78(23):1860–1867. doi: 10.1212/WNL.0b013e318258f744.
    1. Miller DM, Bethoux F, Victorson D, et al. Validating neuro-QoL short forms and targeted scales with people who have multiple sclerosis. Mult Scler. 2016;22(6):830–841. doi: 10.1177/1352458515599450.
    1. Cella D. Development and initial validation of patient-reported item banks for use in neurological research and practice. 2015. . Accessed 18 Dec 2018.
    1. International Organization for Standardization. ISO13485:2016: medical devices—quality management systems—requirements for regulatory purposes. 2016. . Accessed 18 Dec 2018.
    1. Svanaes D, Das A, Alos OA. The contextual nature of usability and its relevance to medical informatics. In: Andersen SK, Klein MCA, Schulz S, Aarts J, Mazzoleni MC, editors. eHealth beyond the horizon—get IT there. Clifton: IOS; 2008.

Source: PubMed

Upcoming Clinical Trials

Sponsors and Collaborators

Medical Conditions

Drug Interventions

Subscribe