Automated Assessment of Balance Rehabilitation Exercises With a Data-Driven Scoring Model: Algorithm Development and Validation Study

Abstract

Background: Balance rehabilitation programs represent the most common treatments for balance disorders. Nonetheless, lack of resources and lack of highly expert physiotherapists are barriers for patients to undergo individualized rehabilitation sessions. Therefore, balance rehabilitation programs are often transferred to the home environment, with a considerable risk of the patient misperforming the exercises or failing to follow the program at all. Holobalance is a persuasive coaching system with the capacity to offer full-scale rehabilitation services at home. Holobalance involves several modules, from rehabilitation program management to augmented reality coach presentation.

Objective: The aim of this study was to design, implement, test, and evaluate a scoring model for the accurate assessment of balance rehabilitation exercises, based on data-driven techniques.

Methods: The data-driven scoring module is based on an extensive data set (approximately 1300 rehabilitation exercise sessions) collected during the Holobalance pilot study. It can be used as a training and testing data set for training machine learning (ML) models, which can infer the scoring components of all physical rehabilitation exercises. In that direction, for creating the data set, 2 independent experts monitored (in the clinic) 19 patients performing 1313 balance rehabilitation exercises and scored their performance based on a predefined scoring rubric. On the collected data, preprocessing, data cleansing, and normalization techniques were applied before deploying feature selection techniques. Finally, a wide set of ML algorithms, like random forests and neural networks, were used to identify the most suitable model for each scoring component.

Results: The results of the trained model improved the performance of the scoring module in terms of more accurate assessment of a performed exercise, when compared with a rule-based scoring model deployed at an early phase of the system (k-statistic value of 15.9% for sitting exercises, 20.8% for standing exercises, and 26.8% for walking exercises). Finally, the resulting performance of the model resembled the threshold of the interobserver variability, enabling trustworthy usage of the scoring module in the closed-loop chain of the Holobalance coaching system.

Conclusions: The proposed set of ML models can effectively score the balance rehabilitation exercises of the Holobalance system. The models had similar accuracy in terms of Cohen kappa analysis, with interobserver variability, enabling the scoring module to infer the score of an exercise based on the collected signals from sensing devices. More specifically, for sitting exercises, the scoring model had high classification accuracy, ranging from 0.86 to 0.90. Similarly, for standing exercises, the classification accuracy ranged from 0.85 to 0.92, while for walking exercises, it ranged from 0.81 to 0.90.

Trial registration: ClinicalTrials.gov NCT04053829; https://ichgcp.net/clinical-trials-registry/NCT04053829.

Keywords: balance rehabilitation exercises; exercise evaluation; persuasive coaching system; scoring model.

Conflict of interest statement

Conflicts of Interest: None declared.

©Vassilios Tsakanikas, Dimitris Gatsios, Athanasios Pardalis, Kostas M Tsiouris, Eleni Georga, Doris-Eva Bamiou, Marousa Pavlou, Christos Nikitas, Dimitrios Kikidis, Isabelle Walz, Christoph Maurer, Dimitrios Fotiadis. Originally published in JMIR Rehabilitation and Assistive Technology (https://rehab.jmir.org), 31.08.2022.

Figures

Figure 1

Virtual coaching closed-loop interaction. The proposed model is integrated into the “intelligent” module of the virtual coaching system.

Figure 2

The Holobalance system. (A) Sensor positioning in the Holobalance system. (B) Devices of the Holobalance system. IMU: inertial measurement unit.

Figure 3

The scoring model.

Figure 4

Machine learning (ML) model training approach. kNN: k-nearest neighbors; ROC: receiver operating characteristic; SVM: support vector machine.

Figure 5

Confusion matrix. All types of exercises (N=665) in the annotation process of 2 observers.