MinIAttention - Attention Management in Minimal Invasive Surgery

Laparoscopic surgical suboptimal outcomes in patient safety measures are correlated with (i) cognitive load / level of attention of the operating surgeon, (ii) the frequency and degree of disruptions to the surgical workflow, and (iii) the misalignment of visual and motor axes in laparoscopic equipment / setting (eye-hand coordination).

This project will create the foundational, design and operational principles of future, surgeon-friendly minimal invasive surgery operating room information technologies (MIS-IT), which -given the ever growing complexity in surgical workflows, as well as instrument and equipment settings- will have to build on human attention as a scarce resource.

On the formal model's and methods' side, MinIAttention will identify types of human attention, as well as cognitive and physiological mechanisms revealing its relation to perception, memory, decision making, and learning. Starting with established theories of individual attention (Capacity Theory, Multiple Resource Theory, Feature Integration Theory) and the respective attention models (Broadbent, Kahneman, Wickens), the investigators will characterize aspects of attention of surgeons during MIS operations. MinIAttention will empirically evidence its models on the dynamics of a surgeon's attention along the workflow of a MIS operation. A multi-sensor attention recognition reference framework will be implementation, involving externalized signals of a surgeons attention (eye gaze, head and hand gesture, head and full body pose, physiological signals, as well as communication and social interaction). Evidenced MinIAttention attention models will represent the core body of aware surgeon assistance systems, covering (i) sensory assistance, enhancing the surgeons perceptual capacities, (ii) motor assistance, enhancing the surgeons motor and manipulative capacities (iii) decision making assistance, guiding the surgeon towards informed, evidence based, rational, transparent and timely decisions during operation, and (iv) cognitive assistance, enhancing the surgeons memory management capacities with background digital memory systems. On the purpose of supporting this research proposal with an early impression of the feasibility of the proposed research method and approach, the proposing partners have voluntarily set up prototypical attention capturing system. With this preliminary proposal support study the investigators have evidenced that ensembles of sensors together with our attention models can serve as a nonobtrusive, yet potentially effective means to determine indicators of a surgeon's attention during live surgeries. Above that, the pattern recognition methods an machine learning techniques appear viable for the task of automated attention and cognitive load diagnosis, and the proposed assistance and mulitmodal feedback system reveals feasible. The investigators can thus say, that the research method and approach which MinIAttention will build upon is solid, promising, and preliminarily evidenced beforehand.

Aside the MIS-IT focus of this project, MinIAttention will serve as a reference to a very general, observably upcoming information society dilemma: information overload and attention scarcity. In today's information-rich world, where people are overflooded with signals and messages at all levels of perception and modalities (visual, auditory, tactile, olfactory), the need to allocate attention efficiently among the overabundance of information sources appears to be among the most demanding challenges for ICT mediated communication today. For the design and implementation of novel, future ICT systems of all kinds, it is of high interest to understand how spontaneous, local, individual attention to novel information items occurs. Some two decades of HCI and pervasive/ubiquitous computing research have clearly revealed that out of the many indicative design factors for modern ICT, human attention is the first source of perception, consequently also awareness towards information and other individuals. MinIAttention will create the foundational basis for attention-aware ICT, i.e. develop (i) formal models of human attention along with (ii) multisensory recognition architectures and reasoning algorithms to estimate and assess levels of human attention, together with their (iii) embedding into ICT systems of everyday use.

Five international groups will collaborate to develop MinIAttention. JKU IPC has introduced attention aware ICT in the Pervasive Computing scientific community, developed pionieering methods and systems, and also promoted cognitive ICT to become a European research priority (H2020). AKH KUK runs Austrias most advanced laparoscopic operation theater with cutting edge technological equipment, Karl STORZ ENDOSKOPE is a worldwide leading surgical instrument supplier. SUSSEX is among the worlds most renowned institutions in machine learning related to wearable computing, as is FRI for vital state recognition.