FIRST LOOK: Surgical Fingerprints: Real-time Analysis of Intraoperative Events

2222 e1540841541605 1

Intraoperative adverse events (IAEs), such as bowel/vascular injury, are estimated to occur in 2% of operations. IAEs can exact a toll on patient quality of life and costs, with average admission charges estimated to be 41% higher for patients who experience IAEs. Up to two-thirds of surgical errors occur intraoperatively, and 86% of these are secondary to cognitive factors such as failures in judgment, memory, or vigilance that lead to poor decisions. Currently, analysis of the intraoperative phase of care is limited to review of dictated operative reports that are notoriously incomplete. While video has been shown to be more accurate for identification of IAEs, manual review of video is costly and time-consuming.

Click here to watch Dr. Hashimoto’s First Look presentation.

We developed a machine learning approach to analyze laparoscopic video and generate identification and segmentation of operative steps. Using annotated video, we trained a support vector machine and a neural network to classify video frames into their respective operative steps. Hidden Markov models were used for segmentation of videos into operative steps with >90% accuracy (Fig. 1). We used coresets to leverage semantic summarization of video to increase the efficiency of segmentation. The cumulative log probability for each frame allowed for real-time estimation of deviation from an expected operative path and resulted in a “surgical fingerprint” that visually summarized potential areas of unexpected operative events (Fig. 2).

Real-time analysis of surgical video provides the foundation for intra- and post-operative clinical decision support to augment surgical decision-making. We plan to link population-based pre-procedure risk scores with patient-specific surgical fingerprints with the goal of preventing IAEs and identifying optimal, patient-specific post-procedural management, with the potential to recognize rare patient scenarios and seemingly common patients who may not fit into standard protocols. By unlocking intraoperative care as a quantitative data source to predict IAEs, complications, and readmissions, we expect to have an impact on daily clinical care by providing clinicians with an effective, data-driven tool for perioperative management.

For more information about Dr. Hashimoto’s research, please contact Partners HealthCare Innovation by clicking here.

Guy Rosman2, Daniela Rus2, Ozanan R. Meireles1

1Department of Surgery, Massachusetts General Hospital

2Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology


Fig 1. Over 90% accuracy of machine segmentation (blue) vs. human segmentation (green)

Fig 2. Fingerprints comparing A) routine sleeve gastrectomy vs. B) sleeve + lysis of adhesions


Most Recent Posts:

MGB GRAPHICS CILP Program 220927 0912 150 POWERPOINT 2

Are you an Innovator? Applications for MGB Commercialization and Inclusive Leadership Program now open

The MGB Commercialization and Inclusive Leadership Program (CILP) is a one-day in-person program on commercial application and…

Read More
whalen kimberly 003502672 high res

Finding a Safer Way to Draw Pediatric Blood Led This Nurse to Innovate

The Blood Shield Transfer device allows safe transfer of pediatric blood to microcontainers and blood gas syringes…

Read More
Blog MGB Implements DexCare image

Mass General Brigham Implements DexCare to Support Care Integration and Advance Digital Health

Collaboration demonstrates Mass General Brigham’s commitment to digital transformation and enhancing patient experience   SEATTLE, Oct. 31, 2022…

Read More