Surgical Process Modeling for Open Spinal Surgeries

Fabio Carrillo, Hooman Esfandiari, Sandro Müller, Marco von Atzigen, Aidana Massalimova, Daniel Suter, Christoph J Laux, José M Spirig, Mazda Farshad, Philipp Fürnstahl, Fabio Carrillo, Hooman Esfandiari, Sandro Müller, Marco von Atzigen, Aidana Massalimova, Daniel Suter, Christoph J Laux, José M Spirig, Mazda Farshad, Philipp Fürnstahl

Abstract

Modern operating rooms are becoming increasingly advanced thanks to the emerging medical technologies and cutting-edge surgical techniques. Current surgeries are transitioning into complex processes that involve information and actions from multiple resources. When designing context-aware medical technologies for a given intervention, it is of utmost importance to have a deep understanding of the underlying surgical process. This is essential to develop technologies that can correctly address the clinical needs and can adapt to the existing workflow. Surgical Process Modeling (SPM) is a relatively recent discipline that focuses on achieving a profound understanding of the surgical workflow and providing a model that explains the elements of a given surgery as well as their sequence and hierarchy, both in quantitative and qualitative manner. To date, a significant body of work has been dedicated to the development of comprehensive SPMs for minimally invasive baroscopic and endoscopic surgeries, while such models are missing for open spinal surgeries. In this paper, we provide SPMs common open spinal interventions in orthopedics. Direct video observations of surgeries conducted in our institution were used to derive temporal and transitional information about the surgical activities. This information was later used to develop detailed SPMs that modeled different primary surgical steps and highlighted the frequency of transitions between the surgical activities made within each step. Given the recent emersion of advanced techniques that are tailored to open spinal surgeries (e.g., artificial intelligence methods for intraoperative guidance and navigation), we believe that the SPMs provided in this study can serve as the basis for further advancement of next-generation algorithms dedicated to open spinal interventions that require a profound understanding of the surgical workflow (e.g., automatic surgical activity recognition and surgical skill evaluation). Furthermore, the models provided in this study can potentially benefit the clinical community through standardization of the surgery, which is essential for surgical training.

Keywords: bottom-up modeling; open spinal surgery; pedicle screw; spinal fusion; spinal instrumentation; surgical process modeling; top-down modeling.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2022 Carrillo, Esfandiari, Müller, von Atzigen, Massalimova, Suter, Laux, Spirig, Farshad and Fürnstahl.

Figures

Figure 1
Figure 1
Nomenclature for the Granularity and Hierarchy Levels used in our SPMs. Each elements is depicted with a distintive geometrical shape throughout the manuscript. We show here an example case for each of the given granularity levels. The higher granularity level (μ = 5) correspond to the surgical procedure itself and is depicted as a marked rectangle. μ = 4 and μ = 3 correspond to steps and substeps and are denoted by a filled and a non-filled rectangle, respectively. μ = 2 and μ = 1 correspond to tasks and sub-tasks and are denoted by a filled and non-filled rounded rectangle, respectively. Actions (μ = 0) are denoted by hexagon and events are represented by a dashed rectangle.
Figure 2
Figure 2
Deductive SPM for open spinal surgery at the step level (μ = 4), showing the 5 main surgical steps identified for open spinal surgery: (I) superficial incision, (II) deep incision, (III) implantation of pedicle screws, (IV) rod insertion, and (V) wound closure. Additional pathology-specific steps are depicted in orange and are usually located between steps (IV) and (V).
Figure 3
Figure 3
Hierarchical decomposition for the 5 identified surgical steps of the open spinal surgery using the nomenclature defined in Figure 1. Events are represented by a dashed square and are not linked to the rest of the tree, but they are located directly below the corresponding surgical steps where they are expected to happen. Level of granularity is indicated in orange on the right-hand side of the figure.
Figure 4
Figure 4
SPM for the Superficial Incision step. A snapshot of the specific part of the surgery involving this step is shown on the left. Transitions are indicated with an arrow and the number of transitions between the elements are indicated above the arrow with an integer number. Dashed arrows indicate repetitions or deviations from the standard workflow.
Figure 5
Figure 5
SPM for the Deep Incision step. A snapshot of the specific part of the surgery involving this step is shown on the upper left corner. Transitions are indicated with an arrow and the number of transitions between the elements are indicated above the arrow with an integer number. Dashed arrows indicate repetitions or deviations from the standard workflow.
Figure 6
Figure 6
SPM for the Implantation of Pedicle Screws step. A snapshot of the specific part of the surgery involving this step is shown on the upper left corner. Transitions are indicated with an arrow and the number of transitions between the elements are indicated above the arrow with an integer number. Dashed arrows indicate repetitions or deviations from the standard workflow.
Figure 7
Figure 7
SPM for the Rod Insertion step. A snapshot illustrating the specific surgical step is shown on the upper left corner. Transitions are indicated with an arrow and the number of transitions between the elements are indicated above the arrow with an integer number. Dashed arrows indicate repetitions or deviations from the standard workflow.
Figure 8
Figure 8
SPM for the Wound Closure step. A snapshot of the specific part of the surgery illustrating this step is shown on the upper left corner. Transitions are indicated with an arrow and the number of transitions between the elements are indicated above the arrow with an integer number. Dashed arrows indicate repetitions or deviations from the standard workflow.

References

    1. Stitzenberg KB, Sheldon GF. Progressive specialization within general surgery: adding to the complexity of workforce planning. J Am Coll Surg. (2005) 201:925–32. 10.1016/j.jamcollsurg.2005.06.253
    1. Cleary K, Kinsella A, Mun SK. OR 2020 workshop report: operating room of the future. Int Congr Ser. (2005) 1281:832–8. 10.1016/j.ics.2005.03.279
    1. Joskowicz L. Computer-aided surgery meets predictive, preventive, and personalized medicine. EPMA J. (2017) 8:1–4. 10.1007/s13167-017-0084-8
    1. Herrell SD. Robotic surgery: past, present, and future. In: Su L-M, editor. Atlas of Robotic Urologic Surgery. Cham: Springer International Publishing; (2017). p. 459–72.
    1. Negrillo-Cárdenas J, Jiménez-Pérez JR, Feito FR. The role of virtual and augmented reality in orthopedic trauma surgery: from diagnosis to rehabilitation. Comput Methods Programs Biomed. (2020) 191:105407. 10.1016/j.cmpb.2020.105407
    1. Hashimoto DA, Ward TM, Meireles OR. The role of artificial intelligence in surgery. Adv Surg. (2020) 54:89–101. 10.1016/j.yasu.2020.05.010
    1. Maier-Hein L, Vedula SS, Speidel S, Navab N, Kikinis R, Park A, et al. . Surgical data science for next-generation interventions. Nat Biomed Eng. (2017) 1:691–6. 10.1038/s41551-017-0132-7
    1. Padoy N, Blum T, Feussner H, Berger M-O, Navab N. On-line recognition of surgical activity for monitoring in the operating room. In: Proceedings of the 20th national conference on Innovative applications of artificial intelligence, Vol.3. Chicago, IL: AAAI Press; (2008).
    1. Padoy N, Blum T, Ahmadi SA, Feussner H, Berger MO, Navab N. Statistical modeling and recognition of surgical workflow. Med Image Anal. (2012) 16:632–41. 10.1016/j.media.2010.10.001
    1. Riffaud L, Neumuth T, Morandi X, Trantakis C, Meixensberger J, Burgert O, et al. . Recording of surgical processes: a study comparing senior and junior neurosurgeons during lumbar disc herniation surgery. Neurosurgery. (2010) 67(2 Suppl Operative):325–32. 10.1227/NEU.0b013e3181f741d7
    1. Nakawala H, Ferrigno G, De Momi E. Development of an intelligent surgical training system for thoracentesis. Artif Intell Med. (2018) 84:50–63. 10.1016/j.artmed.2017.10.004
    1. Liebmann P, Neumuth T. Model driven design of workflow schemata for the operating room of the future. In: INFORMATIK 2010 Service Science–Neue Perspektiven für die Informatik. Bonn: GI; (2010). p. 419.
    1. Driessen SRC, Van Zwet EW, Haazebroek P, Sandberg EM, Blikkendaal MD, Twijnstra ARH, et al. . A dynamic quality assessment tool for laparoscopic hysterectomy to measure surgical outcomes. Am J Obstet Gynecol. (2016) 215:754.e1–754.e8. 10.1016/j.ajog.2016.07.004
    1. Gholinejad M, Loeve AJ, Dankelman J. Surgical process modelling strategies: which method to choose for determining workflow? Minim Invasive Ther Allied Technol. (2019) 28:15. 10.1080/13645706.2019.1591457
    1. Neumuth T. Surgical process modeling. Innov Surg Sci. (2017) 2:123–37. 10.1515/iss-2017-0005
    1. Padoy N, Blum T, Essa I, Feussner H, Berger MO, Navab N. A boosted segmentation method for surgical workflow analysis. Med Image Comput Comput Assist Interv. (2007) 10(Pt 1):102–9. 10.1007/978-3-540-75757-3_13
    1. Ahmadi SA, Sielhorst T, Stauder R, Horn M, Feussner H, Navab N. Recovery of Surgical Workflow Without Explicit Models. In: Larsen R, Nielsen M, Sporring J, editors. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2006. Berlin, Heidelberg: Springer; (2006). p. 420–8. (Lecture Notes in Computer Science)
    1. Zappella L, Béjar B, Hager G, Vidal R. Surgical gesture classification from video and kinematic data. Med Image Anal. (2013) 17:732–45. 10.1016/j.media.2013.04.007
    1. Twinanda AP, Shehata S, Mutter D, Marescaux J, Mathelin M de, Padoy N. EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging. (2017) 36:86–97. 10.1109/TMI.2016.2593957
    1. Bodenstedt S, Wagner M, Katić D, Mietkowski P, Mayer B, Kenngott H, et al. . Unsupervised Temporal Context Learning Using Convolutional Neural Networks for Laparoscopic Workflow Analysis. ArXiv170203684 Cs. (2017). Available online at: (accessed January 12, 2021).
    1. Si C, Chen W, Wang W, Wang L, Tan T. An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition. ArXiv190209130 Cs. (2019). Available online at: (accessed January 12, 2021)
    1. Esfandiari H, Newell R, Anglin C, Street J, Hodgson AJ. A deep learning framework for segmentation and pose estimation of pedicle screw implants based on C-arm fluoroscopy. Int J Comput Assist Radiol Surg. (2018) 13:1269–82. 10.1007/s11548-018-1776-9
    1. Liebmann F, Roner S, von Atzigen M, Scaramuzza D, Sutter R, Snedeker J, et al. . Pedicle screw navigation using surface digitization on the Microsoft HoloLens. Int J Comput Assist Radiol Surg. (2019) 14:1157–65. 10.1007/s11548-019-01973-7
    1. Maffulli N, Rodriguez HC, Stone IW, Nam A, Song A, Gupta M, et al. . Artificial intelligence and machine learning in orthopedic surgery: a systematic review protocol. J Orthop Surg. (2020) 15:478. 10.1186/s13018-020-02002-z
    1. Deib G, Johnson A, Unberath M, Yu K, Andress S, Qian L, et al. . Image guided percutaneous spine procedures using an optical see-through head mounted display: proof of concept and rationale. J NeuroInterventional Surg. (2018) 10:1187–91. 10.1136/neurintsurg-2017-013649
    1. Lipson SJ. Spinal-fusion surgery – advances and concerns. N Engl J Med. (2004) 350:643–4. 10.1056/NEJMp038162
    1. Vos T, Barber RM, Bell B, Bertozzi-Villa A, Biryukov S, Bolliger I, et al. . Global, regional, and national incidence, prevalence, and years lived with disability for 301 acute and chronic diseases and injuries in 188 countries, 1990–2013: a systematic analysis for the global burden of disease study 2013. Lancet. (2015) 386:743–800. 10.1016/S0140-6736(15)60692-4
    1. Picavet HSJ, Schouten JSAG. Musculoskeletal pain in the netherlands: prevalences, consequences and risk groups, the DMC3-study. Pain. (2003) 102:167–78. 10.1016/s0304-3959(02)00372-x
    1. Murray CJL, Vos T, Lozano R, Naghavi M, Flaxman AD, Michaud C, et al. . Disability-Adjusted Life Years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: a systematic analysis for the global burden of disease study 2010. Lancet. (2012) 380:2197–223. 10.1016/S0140-6736(12)61690-0
    1. Raciborski F, Gasik R, Kłak A. Disorders of the spine. A Major Health and Social Problem. Reumatologia. (2016) 4:196–200. 10.5114/reum.2016.62474
    1. Bailly F, Foltz V, Rozenberg S, Fautrel B, Gossec L. The impact of chronic low back pain is partly related to loss of social role: a qualitative study. Joint Bone Spine. (2015) 82:437–41. 10.1016/j.jbspin.2015.02.019
    1. Schofield DJ, Shrestha RN, Percival R, Callander EJ, Kelly SJ, Passey ME. Early retirement and the fnancial assets of individuals with back problems. Eur Spine J. (2011) 20:731–6. 10.1007/s00586-010-1647-8
    1. Koes BW, van Tulder MW, Thomas S. Diagnosis and treatment of low back pain. BMJ. (2006) 332:1430–4. 10.1136/bmj.332.7555.1430
    1. van Tulder MW, Koes BW, Bouter LM. Conservative treatment of acute and chronic nonspecific low back pain: a systematic review of randomized controlled trials of the most common interventions. Spine. (1997) 22:2128–56. 10.1097/00007632-199709150-00012
    1. Deyo RA, Nachemson A, Mirza SK. Spinal-fusion surgery—the case for restraint. Spine J. (2004) 4:S138–42. 10.1016/j.spinee.2004.08.001
    1. Heini PF. The current treatment—a survey of osteoporotic fracture treatment. osteoporotic spine fractures: the spine surgeon's perspective. Osteoporos Int. (2005) 16:S85–92. 10.1007/s00198-004-1723-1
    1. Hee HT, Majd ME, Holt RT, Pienkowski D. Better treatment of vertebral osteomyelitis using posterior stabilization and titanium mesh cages. J Spinal Disord Tech. (2002) 15:149–56; discussion 156. 10.1097/00024720-200204000-00010
    1. Maruyama T, Takeshita K. Surgical treatment of scoliosis: a review of techniques currently applied. Scoliosis. (2008) 3:6. 10.1186/1748-7161-3-6
    1. Mackel CE, Jada A, Samdani AF, Stephen JH, Bennett JT, Baaj AA, et al. . A comprehensive review of the diagnosis and management of congenital scoliosis. Childs Nerv Syst. (2018) 34:2155–71. 10.1007/s00381-018-3915-6
    1. Esfandiari H. An Intraoperative Position Assessment System for Pedicle Screw Insertion Surgeries (PhD Thesis). The University of British Columbia, Vancouver, Canada (2020).
    1. Katonis P, Christoforakis J, Aligizakis AC, Papadopoulos C, Sapkas G, Hadjipavlou A. Complications and problems related to pedicle screw fixation of the Spine. Clin Orthop. (2003) 411:86–94. 10.1097/01.blo.0000068761.86536.1d
    1. Gautschi OP, Schatlo B, Schaller K, Tessitore E. Clinically relevant complications related to pedicle screw placement in thoracolumbar surgery and their management: a literature review of 35,630 pedicle screws. Neurosurg Focus. (2011) 31:E8. 10.3171/2011.7.FOCUS11168
    1. Nevzati E, Marbacher S, Soleman J, Perrig WN, Diepers M, Khamis A, et al. . Accuracy of pedicle screw placement in the thoracic and lumbosacral spine using a conventional intraoperative fluoroscopy-guided technique: a national neurosurgical education and training center analysis of 1236 consecutive screws. World Neurosurg. (2014) 82:866–71.e2. 10.1016/j.wneu.2014.06.023
    1. Mummaneni PV, Dhall SS, Ondra SL, Mummaneni VP, Berven S. Pedicle subtraction osteotomy. Neurosurgery. (2008) 63(suppl_3):A171–6. 10.1227/01.NEU.0000325680.32776.82
    1. Hartl R, Lam KS, Wang J, Korge A, Kandziora F, Audigé L. Worldwide survey on the use of navigation in spine surgery. World Neurosurg. (2013) 79:162–72. 10.1016/j.wneu.2012.03.011
    1. Kosmopoulos V, Schizas C. Pedicle screw placement accuracy: a meta-analysis. Spine. (2007) 32:E111–20. 10.1097/01.brs.0000254048.79024.8b
    1. Gibaud B, Forestier G, Feldmann C, Ferrigno G, Gonçalves P, Haidegger T, et al. . Toward a standard ontology of surgical process models. Int J Comput Assist Radiol Surg. (2018) 13:1397–408. 10.1007/s11548-018-1824-5
    1. Jannin P, Morandi X. Surgical models for computer-assisted neurosurgery. Neuroimage. (2007) 37:783–91. 10.1016/j.neuroimage.2007.05.034
    1. Katić D, Julliard C, Wekerle AL, Kenngott H, Müller-Stich BP, Dillmann R, et al. . LapOntoSPM: an ontology for laparoscopic surgeries and its application to surgical phase recognition. Int J Comput Assist Radiol Surg. (2015) 10:1427–34. 10.1007/s11548-015-1222-1
    1. Perrone R, Nessi F, Momi E, Boriero F, Capiluppi M, Fiorini P, et al. . Ontology-based modular architecture for surgical autonomous robots. In: The Hamlyn Symposium on Medical Robotics. London, UK: Imperial College; (2014).
    1. Nagy TD, Haidegger T. A DVRK-based framework for surgical subtask automation. Acta Polytech Hung. (2019) 16:61. 10.12700/APH.16.8.2019.8.5
    1. MacKenzie L, Ibbotson JA, Cao CGL, Lomax AJ. Hierarchical decomposition of laparoscopic surgery: a human factors approach to investigating the operating room environment. Minim Invasive Ther Allied Technol. (2001) 10:121–7. 10.1080/136457001753192222
    1. Neumuth T. Surgical Process Modeling Theory, Methods, and Applications (Habilitation Thesis: ). University of Leipzig, Leipzig, Germany (2012).
    1. Stein G, Eysel P, Scheyerer MJ. Expertise orthopädie und unfallchirurgie wirbelsäule. Georg Thieme Verlag. (2019) 1:2216. 10.1055/b-006-149533
    1. Wang J, Chen Y, Hao S, Peng X, Hu L. Deep learning for sensor-based activity recognition: a survey. Pattern Recognit Lett. (2019) 119:3–11. 10.1016/j.patrec.2018.02.010
    1. Uemura M, Jannin P, Yamashita M, Tomikawa M, Akahoshi T, Obata S, et al. . Procedural surgical skill assessment in laparoscopic training environments. Int J Comput Assist Radiol Surg. (2016) 11:543–52. 10.1007/s11548-015-1274-2
    1. Forestier G, Petitjean F, Riffaud L, Jannin P. Automatic matching of surgeries to predict surgeons' next actions. Artif Intell Med. (2017) 81:3–11. 10.1016/j.artmed.2017.03.007
    1. Franke S, Meixensberger J, Neumuth T. Intervention time prediction from surgical low-level tasks. J Biomed Inform. (2013) 46:152–9. 10.1016/j.jbi.2012.10.002
    1. Neumuth T, Mudunuri R, Jannin P, Meixensberger J, Burgert O. SWAN-Suite: the tool landscape for surgical workflow analysis. In Paris; Berlin; Boston, MA: De Gruyter; (2007). p. 199–204.
    1. Lalys F, Riffaud L, Morandi X, Jannin P. Automatic phases recognition in pituitary surgeries by microscope images classification. In: Navab N, Jannin P, editors. Information Processing in Computer-Assisted Interventions. Berlin, Heidelberg: Springer; (2010). p. 34–44. (Lecture Notes in Computer Science).
    1. Bhatia B, Oates T, Xiao Y, Hu P. Real-time identification of operating room state from video. In: Proceedings of the 19th national conference on Innovative applications of artificial intelligence - Volume 2. Vancouver, British Columbia, Canada: AAAI Press; (2007). p. 1761–6. (IAAI'07).
    1. Nagy DÁ, Rudas IJ, Haidegger T. OntoFlow, a software tool for surgical workflow recording. In: 2018 IEEE 16th World Symposium on Applied Machine Intelligence and Informatics (SAMI: ) Herlany, Slovakia (2018). p. 000119–24.
    1. Katić D, Wekerle AL, Görtler J, Spengler P, Bodenstedt S, Röhl S, et al. . Context-aware augmented reality in laparoscopic surgery. Comput Med Imaging Graph. (2013) 37:174–82. 10.1016/j.compmedimag.2013.03.003

Source: PubMed

赞助者和合作者

医疗条件

药物干预

3
订阅