Exploratory Outlier Detection for Acceleromyographic Neuromuscular Monitoring: Machine Learning Approach

Abstract

Background: Perioperative quantitative monitoring of neuromuscular function in patients receiving neuromuscular blockers has become internationally recognized as an absolute and core necessity in modern anesthesia care. Because of their kinetic nature, artifactual recordings of acceleromyography-based neuromuscular monitoring devices are not unusual. These generate a great deal of cynicism among anesthesiologists, constituting an obstacle toward their widespread adoption. Through outlier analysis techniques, monitoring devices can learn to detect and flag signal abnormalities. Outlier analysis (or anomaly detection) refers to the problem of finding patterns in data that do not conform to expected behavior.

Objective: This study was motivated by the development of a smartphone app intended for neuromuscular monitoring based on combined accelerometric and angular hand movement data. During the paired comparison stage of this app against existing acceleromyography monitoring devices, it was noted that the results from both devices did not always concur. This study aims to engineer a set of features that enable the detection of outliers in the form of erroneous train-of-four (TOF) measurements from an acceleromyographic-based device. These features are tested for their potential in the detection of erroneous TOF measurements by developing an outlier detection algorithm.

Methods: A data set encompassing 533 high-sensitivity TOF measurements from 35 patients was created based on a multicentric open label trial of a purpose-built accelero- and gyroscopic-based neuromuscular monitoring app. A basic set of features was extracted based on raw data while a second set of features was purpose engineered based on TOF pattern characteristics. Two cost-sensitive logistic regression (CSLR) models were deployed to evaluate the performance of these features. The final output of the developed models was a binary classification, indicating if a TOF measurement was an outlier or not.

Results: A total of 7 basic features were extracted based on raw data, while another 8 features were engineered based on TOF pattern characteristics. The model training and testing were based on separate data sets: one with 319 measurements (18 outliers) and a second with 214 measurements (12 outliers). The F1 score (95% CI) was 0.86 (0.48-0.97) for the CSLR model with engineered features, significantly larger than the CSLR model with the basic features (0.29 [0.17-0.53]; P<.001).

Conclusions: The set of engineered features and their corresponding incorporation in an outlier detection algorithm have the potential to increase overall neuromuscular monitoring data consistency. Integrating outlier flagging algorithms within neuromuscular monitors could potentially reduce overall acceleromyography-based reliability issues.

Trial registration: ClinicalTrials.gov NCT03605225; https://ichgcp.net/clinical-trials-registry/NCT03605225.

Keywords: acceleromyography; anesthesiology; machine learning; monitoring devices; monitors; neuromuscular; neuromuscular monitoring; outlier analysis; postoperative residual curarization; train-of-four.

Conflict of interest statement

Conflicts of Interest: None declared.

©Michaël Verdonck, Hugo Carvalho, Johan Berghmans, Patrice Forget, Jan Poelaert. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 06.06.2021.

Figures

Figure 1

Normal and anomalous train-of-four (TOF) patterns; upper figures display normal TOF observations while bottom figures represent anomalies. Normality recordings are illustrated by the upper 2 patterns, where 4 clear peaks can be detected and that follow one another in a fixed time interval. The lower 2 patterns represent 2 simulated anomalies, where the 4 peaks cannot be clearly recognized from the TOF pattern; or where a wide gap in terms of time interval exists between peaks.

Figure 2

Basic and engineered feature illustration (x axis: time, y axis: combined angulation and acceleration). Each color represents an individual train-of-four (TOF) twitch. Each individual twitch (T1, T2, T3, and T4) is composed of multiple acceleration/angulation points during the corresponding contraction, and not solely by the highest value. The TOF recording is obtained by the summation of each individual twitch.

Figure 3

Overview of data set partitioning and model training.

Figure 4

Panel A displays a scatter plot of train-of-four ratio (TOFR) and T1. Panel B displays a scatter plot of the features deltaT4_T1 and ratioT1. Scatter plot displays 1 as outlier, and 0 as a normal observation. TOFR in absolute units. DeltaT4_T1 in milliseconds. T1 in root-mean square angulation and acceleration.

Figure 5

Learning curves of the cost-sensitive logistic regression models of the basic feature (above) set and the engineering feature set (below). Training and validation metrics are the F1-score and the ROC-AUC score (y-axis). X-axis represents the number of training instances.

Figure 6

Receiver-operating characteristic curves displaying the ability of the algorithms to classify a train-of-four measurement as an outlier. AUC: area under the curve.

Figure 7

(A) Short 4-peak burst—rebound phenomenon TOF-like pattern after a single movement, short intertwitch distance; (B) T3 less than T2 and T4; (C) Almost equidistant oscillations; (D) Crescendo pattern; (E) Decrescendo but gross oscillations and variable interpeak distance. Some of the detected patterns have implications. y axis: muscle acceleration (m/s2), x axis: time (centiseconds). TOF: train-of-four.