Intelligent wearable allows out-of-the-lab tracking of developing motor abilities in infants

Abstract

Background: Early neurodevelopmental care needs better, effective and objective solutions for assessing infants' motor abilities. Novel wearable technology opens possibilities for characterizing spontaneous movement behavior. This work seeks to construct and validate a generalizable, scalable, and effective method to measure infants' spontaneous motor abilities across all motor milestones from lying supine to fluent walking.

Methods: A multi-sensor infant wearable was constructed, and 59 infants (age 5-19 months) were recorded during their spontaneous play. A novel gross motor description scheme was used for human visual classification of postures and movements at a second-level time resolution. A deep learning -based classifier was then trained to mimic human annotations, and aggregated recording-level outputs were used to provide posture- and movement-specific developmental trajectories, which enabled more holistic assessments of motor maturity.

Results: Recordings were technically successful in all infants, and the algorithmic analysis showed human-equivalent-level accuracy in quantifying the observed postures and movements. The aggregated recordings were used to train an algorithm for predicting a novel neurodevelopmental measure, Baba Infant Motor Score (BIMS). This index estimates maturity of infants' motor abilities, and it correlates very strongly (Pearson's r = 0.89, p < 1e-20) to the chronological age of the infant.

Conclusions: The results show that out-of-hospital assessment of infants' motor ability is possible using a multi-sensor wearable. The algorithmic analysis provides metrics of motility that are transparent, objective, intuitively interpretable, and they link strongly to infants' age. Such a solution could be automated and scaled to a global extent, holding promise for functional benchmarking in individualized patient care or early intervention trials.

Keywords: Biomarkers; Paediatric research.

Conflict of interest statement

Competing interestsThe authors declare the following competing interests: E.I. is the founder and shareholder of Planno Ltd, which consults in technical textile design and manufacturing. The remaining authors declare no competing interests.

© The Author(s) 2022.

Figures

Fig. 1. Overview of the MAIJU wearable, infant cohort, and recording data.

a A 10-month-old subject crawling at home with the MAIJU jumpsuit, equipped with movement sensors in the proximal pockets of each limb. The photograph has been published with informed parental consent. b Summary of the infant cohort (N = 59 infants, N = 64 recordings) recorded in the present study. Bars depict a monthly breakdown of the numbers of infants with MAIJU recordings with vs without synchronized annotated video recordings, as well as the total length of data available for each age. c An example recording in the annotation software showing 20 s of the raw 24-channel data obtained from the four MAIJU sensors, as well as the respective human annotations for postures and movements shown in the bars above the signals, colored according to the motor ability categories shown in Fig. 5a. Note the frequent transitions in posture and movement categories.

Fig. 2. Study design and the infant motor ability description scheme.

a Flowchart depicting the overall study design. Coloring of the classification comparisons between humans (red) and human vs algorithm (blue) correspond to the same colors in section B. b Illustrations of the posture and movement categories identified in our motor ability description scheme. Numbers in each cell depict the proportion of each category within the annotated dataset (black), and the Fleiss’ kappa agreement between human observers (red) or between the algorithmic analysis and human observers (blue) in the classification of 2.3-s signal frames. c Correlation between infant age and the proportions of motor ability types (N = 42) identified from the video recordings by the human observers (individual points; the line indicates a quadratic regression model with 95% confidence intervals; r represents the Pearson’s correlation coefficient). Note a robust age-related decrease in prone posture, increases in standing and fluent movement, as well as the bell-shaped developmentally transient occurrence of the crawling posture.

Fig. 3. Block diagram of the deep-learning-based motor ability classifier architecture.

Abbreviations: activation function (act), average (AVG), channels (ch), convolution operation (conv), dilation (dil), filter size (fw), leaky rectified linear unit (lrelu), padding (pad). The encoder module performs frame-level sensor fusion to obtain a 160-dimensional latent expression of the raw accelerometer and gyroscope signals. The classifier module models the frame-to-frame time dynamics of these features and outputs softmax probabilities for each category separately for each of the classification tracks (posture, movement, and carrying). The training was performed with minibatch gradient descent using the ADAM algorithm (batch size 100 consecutive frames, learning rate 10−4, beta1 = 0.9, beta2 = 0.999, epsilon = 10−8) with a weighted categorical cross-entropy loss. In the loss function, each frame’s error was weighted with the inverse probability of the target class’s occurrence in the training data to mitigate the effects of unbalanced category distributions within the training data. Sample dropout (p = 0.3), as well as sensor dropout (p = 0.3), was also applied randomly to the input signals during training to ensure the robustness of the trained models. The training was run for 200 epochs and held out validation data (20% of training data) was used to select the best performing model in terms of the unweighted average F1 score. The code for the motor ability classifier was implemented with Tensorflow (v.1.12.0) and Python (v.3.6.9). The code is available at request.

Fig. 4. Classifier development and computational analyses of MAIJU recordings.

a t-SNE plots obtained from self-supervised feature embeddings (CPC) with color codings for posture (top) and movement (bottom). Note the clear clustering of posture categories, while the movement categories show relatively more dispersion. b Confusion matrices showing recall values (in %) of the algorithm output (“Predicted class”) and the compounded human expert annotations (“Target class”). Note the high numbers in the diagonal line indicating high agreement. c Comparison of quantified motor ability between the classifier and human annotations (N = 42). In the upper graphs, the scatter plots show the proportion of time spent in the given postures or movements as estimated by the classifier algorithm (Y-axis) and the human annotations (X-axis). The Pearson’s r (and its p value) denotes the linear correlation between the proportion values. Below, the Bland–Altman plots of annotations vs classification errors are shown for assessing whether the classification errors have a systematic bias and/or are dependent on the amount of posture/movement identified by the algorithm. The stippled lines depict an average one-month developmental change (percentage points per month) as taken from a linear regression model fitted between the age (in months) and the given motor ability occurrence (cf. Fig. 2c). Note that 100 and 88% of the measurements in posture and fluent movement categories, respectively, are within these stippled lines. The shaded zone depicts the 95% confidence interval (in percentage points) of the classifier error. The t value depicts the two-tailed t-test result (with 40 degrees of freedom) on the null hypothesis that the error has a mean of zero; this shows that the proportional estimates are unbiased.

Fig. 5. Assessing maturation of infant motor ability with MAIJU.

a Graphs showing the occurrence of each posture (left) and motor ability class (right) as a function of infants’ prematurity-corrected age (N = 60). The black lines denote the interquartile range (IQR) of the age-related occurrence, and the red cross depicts the median age for the occurrence. The measures combine all analyzed 2.3 s time frames of the recording session, and all infants exhibit motor ability in several classes, which show clear developmental trajectories. Note also the clear developmental sequence in the movement categories within each posture. b Scatter plot showing a correlation between infants’ (N = 60) chronological age and the age prediction from the BIMS algorithm. c Dependence of BIMS estimate on the length of recordings between 10 and 100 min of data. Data were taken as randomly sampled segments from N = 12 recordings whose length was over 120 min (range 121–150 min). The findings in the Y-axis are expressed as the mean absolute error (MAE) in the age prediction as in b) (bars show the median, IQR, and the range). Note how the MAE stabilizes with recording lengths over one hour. d Correlation between BIMS and AIMS score (purple) compared to the correlation between true age and AIMS score (green) (N = 28). The result indicates that the BIMS score is biased towards the actual developmental level, as the correlation is significantly higher (cocor tests; p < 0.05; N = 28) compared to the chronological age correlation. e Comparison between a parental estimate of infant’s time spent in various postures and the MAIJU-derived corresponding measures (N = 20).