FIRST LOOK: Surgical Fingerprints: Real-time Analysis of Intraoperative Events

2222 e1540841541605 1

Intraoperative adverse events (IAEs), such as bowel/vascular injury, are estimated to occur in 2% of operations. IAEs can exact a toll on patient quality of life and costs, with average admission charges estimated to be 41% higher for patients who experience IAEs. Up to two-thirds of surgical errors occur intraoperatively, and 86% of these are secondary to cognitive factors such as failures in judgment, memory, or vigilance that lead to poor decisions. Currently, analysis of the intraoperative phase of care is limited to review of dictated operative reports that are notoriously incomplete. While video has been shown to be more accurate for identification of IAEs, manual review of video is costly and time-consuming.

Click here to watch Dr. Hashimoto’s First Look presentation.

We developed a machine learning approach to analyze laparoscopic video and generate identification and segmentation of operative steps. Using annotated video, we trained a support vector machine and a neural network to classify video frames into their respective operative steps. Hidden Markov models were used for segmentation of videos into operative steps with >90% accuracy (Fig. 1). We used coresets to leverage semantic summarization of video to increase the efficiency of segmentation. The cumulative log probability for each frame allowed for real-time estimation of deviation from an expected operative path and resulted in a “surgical fingerprint” that visually summarized potential areas of unexpected operative events (Fig. 2).

Real-time analysis of surgical video provides the foundation for intra- and post-operative clinical decision support to augment surgical decision-making. We plan to link population-based pre-procedure risk scores with patient-specific surgical fingerprints with the goal of preventing IAEs and identifying optimal, patient-specific post-procedural management, with the potential to recognize rare patient scenarios and seemingly common patients who may not fit into standard protocols. By unlocking intraoperative care as a quantitative data source to predict IAEs, complications, and readmissions, we expect to have an impact on daily clinical care by providing clinicians with an effective, data-driven tool for perioperative management.

For more information about Dr. Hashimoto’s research, please contact Partners HealthCare Innovation by clicking here.

Guy Rosman2, Daniela Rus2, Ozanan R. Meireles1

1Department of Surgery, Massachusetts General Hospital

2Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology


Fig 1. Over 90% accuracy of machine segmentation (blue) vs. human segmentation (green)

Fig 2. Fingerprints comparing A) routine sleeve gastrectomy vs. B) sleeve + lysis of adhesions


Most Recent Posts:

Blog CILP 2023 Cohort image

Meet the 2023 Commercialization and Inclusive Leadership Program Participants

Congratulations to all the participants chosen to attend the 2023 Mass General Brigham Commercialization and Inclusive Leadership…

Read More
Blog 2023 WMIF image

Connect with global leaders in healthcare innovation at the 2023 World Medical Innovation Forum June 12-14

The annual World Medical Innovation Forum is being held in person at the Westin Seaport District in…

Read More
Blog Employee Service Recognition image

Innovation Employees Celebrate Years of Service Milestones

More than 500 Mass General Brigham employees were honored in the recent 26th annual Enterprise Services Employee…

Read More