GEARing smart environments for pediatric motor rehabilitation
Elena Kokkoni, Effrosyni Mavroudi, Ashkan Zehfroosh, James C Galloway, Renè Vidal, Jeffrey Heinz, Herbert G Tanner, Elena Kokkoni, Effrosyni Mavroudi, Ashkan Zehfroosh, James C Galloway, Renè Vidal, Jeffrey Heinz, Herbert G Tanner
Abstract
Background: There is a lack of early (infant) mobility rehabilitation approaches that incorporate natural and complex environments and have the potential to concurrently advance motor, cognitive, and social development. The Grounded Early Adaptive Rehabilitation (GEAR) system is a pediatric learning environment designed to provide motor interventions that are grounded in social theory and can be applied in early life. Within a perceptively complex and behaviorally natural setting, GEAR utilizes novel body-weight support technology and socially-assistive robots to both ease and encourage mobility in young children through play-based, child-robot interaction. This methodology article reports on the development and integration of the different system components and presents preliminary evidence on the feasibility of the system.
Methods: GEAR consists of the physical and cyber components. The physical component includes the playground equipment to enrich the environment, an open-area body weight support (BWS) device to assist children by partially counter-acting gravity, two mobile robots to engage children into motor activity through social interaction, and a synchronized camera network to monitor the sessions. The cyber component consists of the interface to collect human movement and video data, the algorithms to identify the children's actions from the video stream, and the behavioral models for the child-robot interaction that suggest the most appropriate robot action in support of given motor training goals for the child. The feasibility of both components was assessed via preliminary testing. Three very young children (with and without Down syndrome) used the system in eight sessions within a 4-week period.
Results: All subjects completed the 8-session protocol, participated in all tasks involving the selected objects of the enriched environment, used the BWS device and interacted with the robots in all eight sessions. Action classification algorithms to identify early child behaviors in a complex naturalistic setting were tested and validated using the video data. Decision making algorithms specific to the type of interactions seen in the GEAR system were developed to be used for robot automation.
Conclusions: Preliminary results from this study support the feasibility of both the physical and cyber components of the GEAR system and demonstrate its potential for use in future studies to assess the effects on the co-development of the motor, cognitive, and social systems of very young children with mobility challenges.
Keywords: Activity recognition; Body weight support; Decision making; Human-robot interaction; Pediatric rehabilitation.
Conflict of interest statement
The authors declare that they have no competing interests.
Figures
References
- Campos JJ, Anderson DI, Barbu-Roth MA, Hubbard EM, Hertenstein MJ, Witherington D. Travel broadens the mind. Infancy. 2000;1:149–219. doi: 10.1207/S15327078IN0102_1.
- Diamond A. Close interrelation of motor development and cognitive development and of the cerebellum and prefrontal cortex. Child Dev. 2000;71:44–56. doi: 10.1111/1467-8624.00117.
- Iverson JM. Developing language in a developing body: the relationship between motor development and language development. J Child Lang. 2010;37:229–261. doi: 10.1017/S0305000909990432.
- Hitzert MM, Roze E, Van Braeckel KNJA, Bos AF. Motor development in 3-month-old healthy term-born infants is associated with cognitive and behavioural outcomes at early school age. Dev Med Child Neurol. 2014;56:869–876. doi: 10.1111/dmcn.12468.
- Cioni G, Inguaggiato E, Sgandurra G. Early intervention in neurodevelopmental disorders: underlying neural mechanisms. Dev Med Child Neurol. 2016;58:61–66. doi: 10.1111/dmcn.13050.
- Rosenzweig MR. Environmental complexity, cerebral change, and behavior. Am Psychol. 1966;21:321–332. doi: 10.1037/h0023555.
- Baroncelli L, Braschi C, Spolidoro M, Begenisic T, Sale A, Maffei L. Nurturing brain plasticity: impact of environmental enrichment. Cell Death Differ. 2010;17:1092–1103. doi: 10.1038/cdd.2009.193.
- Morgan C, Novak I, Badawi N. Enriched environments and motor outcomes in cerebral palsy: systematic review and meta-analysis. Pediatrics. 2013;132:e735–e746. doi: 10.1542/peds.2012-3985.
- Fox SE, Levitt P, Nelson CA. How the timing and quality of early experiences influence the development of brain architecture. Child Dev. 2010;81:28–40. doi: 10.1111/j.1467-8624.2009.01380.x.
- Johnston MV, Ishida A, Ishida WN, Matsushita HB, Nishimura A, Tsuji M. Plasticity and injury in the developing brain. Brain Dev. 2009;31:1–10. doi: 10.1016/j.braindev.2008.03.014.
- Bayley N. Bayley scales of infant and toddler development: Bayley-III. San Antonio: Harcourt Assessment, Psychological Corporation; 2006.
- Van Praag H, Kempermann G, Gage FH. Neural consequences of environmental enrichment. Nat Rev Neurosci. 2000;1:191–198. doi: 10.1038/35044558.
- Gannotti ME. Coupling timing of interventions with dose to optimize plasticity and participation in pediatric neurologic populations. Pediatr Phys Ther. 2017;29:S37–S47. doi: 10.1097/PEP.0000000000000383.
- Johansson BB, Ohlsson AL. Environment, social interaction, and physical activity as determinants of functional outcome after cerebral infarction in the rat. Exp Neurol. 1996;139:322–327. doi: 10.1006/exnr.1996.0106.
- Berger SE, Theuring C, Adolph KE. How and when infants learn to climb stairs. Infant Behav Dev. 2007;30:36–49. doi: 10.1016/j.infbeh.2006.11.002.
- Adolph KE. Psychophysical assessment of toddlers’ ability to cope with slopes. J Exp Psychol Hum Percept Perform. 1995;21:734–750. doi: 10.1037/0096-1523.21.4.734.
- Lobo MA, Harbourne RT, Dusing SC, McCoy SW. Grounding early intervention: physical therapy cannot just be about motor skills anymore. Phys Ther. 2013;93:94–103. doi: 10.2522/ptj.20120158.
- Fetters L. Perspective on variability in the development of human action. Phys Ther. 2010;90:1860–1867. doi: 10.2522/ptj.2010090.
- Harbourne RT, Berger SE. Embodied cognition in practice: exploring effects of a motor-based problem-solving intervention. Phys Ther. 2019;99:786–796. doi: 10.1093/ptj/pzz031.
- Von Hofsten C. Action, the foundation for cognitive development. Scand J Psychol. 2009;50:617–623. doi: 10.1111/j.1467-9450.2009.00780.x.
- Smith L, Gasser M. The development of embodied cognition: six lessons from babies. Artif Life. 2005;11:13–29. doi: 10.1162/1064546053278973.
- Thelen E. Grounded in the world: developmental origins of the embodied mind. Infancy. 2000;1:3–28. doi: 10.1207/S15327078IN0101_02.
- Hidler Joseph, Brennan David, Black iian, Nichols Diane, Brady Kathy, Nef Tobias. ZeroG: Overground gait and balance training system. The Journal of Rehabilitation Research and Development. 2011;48(4):287. doi: 10.1682/JRRD.2010.05.0098.
- Prosser LA, Ohlrich LB, Curatalo LA, Alter KE, Damiano DL. Feasibility and preliminary effectiveness of a novel mobility training intervention in infants and toddlers with cerebral palsy. Dev Neurorehabil. 2012;15:259–266. doi: 10.3109/17518423.2012.687782.
- Kokkoni E, Galloway JC. User-centred assistive technology assessment of a portable open-area body weight support system for in-home use. Disabil Rehabil Assist Technol. 2019;0:1–8. Available from: 10.1080/17483107.2019.1683236.
- Kokkoni E, Logan SW, Stoner T, Peffley T, Galloway JC. Use of an in-home body weight support system by a child with Spina bifida. Pediatr Phys Ther. 2018;30:E1–E6. doi: 10.1097/PEP.0000000000000516.
- Feil-Seifer D, Matarić MJ. Defining socially assistive robotics. Proc IEEE 9th Int Conf Rehabil robot. IEEE. 2005;2005:465–8.
- Scassellati B, Admoni H, Matarić M. Robots for Use in Autism Research. Annu Rev Biomed Eng. 2012;14:275–294. doi: 10.1146/annurev-bioeng-071811-150036.
- Kim ES, Berkovits LD, Bernier EP, Leyzberg D, Shic F, Paul R, et al. Social robots as embedded reinforcers of social behavior in children with autism. J Autism Dev Disord. 2013;43:1038–1049. doi: 10.1007/s10803-012-1645-2.
- Kaur Maninderjit, Gifford Timothy, Marsh Kerry L., Bhat Anjana. Effect of Robot–Child Interactions on Bilateral Coordination Skills of Typically Developing Children and a Child With Autism Spectrum Disorder: A Preliminary Study. Journal of Motor Learning and Development. 2013;1(2):31–37. doi: 10.1123/jmld.1.2.31.
- Carlos J, José P, González C, Bandera A, Bustos P, Fernández F. Evaluating the child – robot interaction of the NAOTherapist platform in pediatric rehabilitation. Int J Soc Robot Springer Netherlands. 2017;9:343–358. doi: 10.1007/s12369-017-0402-2.
- Fridin Marina, Belokopytov Mark. Robotics Agent Coacher for CP motor Function (RAC CP Fun) Robotica. 2014;32(8):1265–1279. doi: 10.1017/S026357471400174X.
- Fitter NT, Funke R, Pulido C, Eisenman LE, Deng W, Rosales MR, et al. Using a Socially Assistive Humanoid Robot to Encourage Infant Leg Motion Training. Robot Autom Mag. 2019.
- Nithianantharajah Jess, Hannan Anthony J. Enriched environments, experience-dependent plasticity and disorders of the nervous system. Nature Reviews Neuroscience. 2006;7(9):697–709. doi: 10.1038/nrn1970.
- Haehl Victoria, Vardaxis Vassilios, Ulrich Beverly. Learning to cruise: Bernstein's theory applied to skill acquisition during infancy. Human Movement Science. 2000;19(5):685–715. doi: 10.1016/S0167-9457(00)00034-8.
- Adolph KE, Cole WG, Komati M, Garciaguirre JS, Badaly D, Lingeman JM, et al. How do you learn to walk? Thousands of steps and dozens of falls per day. Psychol Sci. 2012;23:1387–1394. doi: 10.1177/0956797612446346.
- Richard A, Gall J. Temporal action detection using a statistical language model. IEEE Conf Comput Vis Pattern Recognit. 2016:3131–40.
- Lea C, Flynn MD, Vidal R, Reiter A, Hager GD. Temporal Convolutional Networks for Action Segmentation and Detection. IEEE Conf Comput Vis Pattern Recognit. 2017:1003–12.
- Chen SF, Goodman JT. An empirical study of smoothing techniques for language modeling. Comput Speech Lang. 1999;13:359–394. doi: 10.1006/csla.1999.0128.
- Cardoso Aline Christine das Neves, de Campos Ana Carolina, dos Santos Mariana Martins, Santos Denise Castilho Cabrera, Rocha Nelci Adriana Cicuto Ferreira. Motor Performance of Children With Down Syndrome and Typical Development at 2 to 4 and 26 Months. Pediatric Physical Therapy. 2015;27(2):135–141. doi: 10.1097/PEP.0000000000000120.
- Pereira K, Basso RP, Lindquist ARR, Da Silva LGP, Tudella E. Infants with Down syndrome: percentage and age for acquisition of gross motor skills. Res Dev Disabil Elsevier Ltd. 2013;34:894–901. doi: 10.1016/j.ridd.2012.11.021.
- Mahoney G, Robinson C, Perales F. Early motor intervention; the need for new treatment paradigms. Infants Young Child. 2004;17:291–300. doi: 10.1097/00001163-200410000-00003.
- Virji-Babul N, Kerns K, Zhou E, Kapur A, Shiffrar M. Perceptual-motor deficits in children with Down syndrome: implications for intervention. Down Syndr Res Pract. 2006;10:74–82. doi: 10.3104/reports.308.
- Karasik LB, Tamis-Lemonda CS, Adolph KE. Transition from crawling to walking and infants’ actions with objects and people. Child Dev. 2011;82:1199–1209. doi: 10.1111/j.1467-8624.2011.01595.x.
- Dosso JA, Boudreau JP. Crawling and walking infants encounter objects differently in a multi-target environment. Exp Brain Res. 2014;232:3047–3054. doi: 10.1007/s00221-014-3984-z.
- Wang H, Schmid C. Action recognition with improved trajectories. Proc IEEE Int Conf Comput Vis. 2013:3551–8.
- Andrews S, Tsochantaridis I, Hofmann T. Support Vector Machines for Multiple-Instance Learning. Adv Neural Inf Process Syst. 2003;577–584. Available from: .
- Zehfroosh A, Kokkoni E, Tanner HG, Heinz J. Learning models of human-robot interaction from small data. Proc Mediterr Conf Control Autom. 2017:223–8.
- Salter T, Werry I, Michaud F. Going into the wild in child-robot interaction studies: issues in social robotic development. Intell Serv Robot. 2008;1:93–108. doi: 10.1007/s11370-007-0009-9.
- Kennedy James, Baxter Paul, Belpaeme Tony. Nonverbal Immediacy as a Characterisation of Social Behaviour for Human–Robot Interaction. International Journal of Social Robotics. 2016;9(1):109–128. doi: 10.1007/s12369-016-0378-3.
- Mead R, Mataric MJ. Robots have needs too: people adapt their Proxemic preferences to improve autonomous robot recognition of human social signals. Human-Robot Interact. 2016;5:48–68. doi: 10.5898/JHRI.5.2.Mead.
Source: PubMed