Video-based feedback is an educational and informative learning approach that offers a unique method to assign qualitative and quantitative metrics for surgeons and their teams. These metrics inform the entire operating room (OR), as well as ancillary services that function alongside the OR system.
Surgical teams worldwide are striving to develop algorithms for real time surgical skill assessment, team performance enhancement, error prevention during surgery, and overall enhancement of patient outcomes. This last aim—improved patient care—is a vital objective, as aligning surgeons’ competencies with patient results is pivotal for ensuring healthcare excellence.
Employing objective quantitative and qualitative parameters alongside outcomes-driven data is poised to offer the most precise and dependable indicators of surgical proficiency and OR efficiency. These metrics have the potential to enhance training, education, and practice, ultimately elevating the quality of patient care.
Video-based, peer-reviewed education—typically evaluated by human assessors and soon to be supported by artificial intelligence (AI) and augmented reality—represents the forthcoming paradigm in surgical education. Given the ongoing transition to competency-based medical education, a more systematic approach to assessment is imperative to ascertain trainees’ proficiency in requisite skills and to provide constructive feedback for competence development.1
In 2023, Wong and colleagues introduced a classification system for intraoperative feedback during live surgical procedures.2 Concurrently, the American Board of Surgery (ABS) conducted an examination of video-based assessment (VBA) as a supplementary tool for appraising technical skills in certification, determining it a promising adjunct to the existing board certification process and warranting further exploration by the ABS.3
Given the rapid recognition and inevitable integration of AI and VBA into surgical training, developing more reliable methods of assessing performance in surgical practice and surgical trainees is imperative. Video-based feedback in a nonpunitive approach offers a way to assign qualitative and quantitative metrics that are reproducible and educational.
This article examines 10 current methods for VBA of surgical skills in the OR and outlines evaluation criteria for intraoperative camera devices.
Apella.io
Founded in 2019, in San Francisco, California, Apella Technology was established with the primary objective of enhancing hospital operations, elevating surgical quality, augmenting staff training, and advancing overall surgical care.
Apella employs in-OR sensors to gather data and, in conjunction with a diverse range of AI algorithms, generates metrics that specifically target the objectives.
Apella aspires to be integrated into all types of surgical procedures, fostering a unified perspective among healthcare teams, and striving to empower surgeons, OR personnel, and administrative staff. In 2021, the startup initiated the adoption of its technology within ORs, including at institutions like Houston Methodist in Texas.
Avail Medsystems
Avail Medsystems was founded in 2018, in Santa Clara, California. This technology aims to build a remote network from which every procedure room may have access to, and collaboration with, clinical experts.
The system—which includes both audio/visual hardware and software, with options for set-up of a console or, alternatively, installation of a remote app for iPad or laptop use—is marketed as compatible with all surgery types and allows for live procedures to be streamed, imaging to be shared, and communication between those in the OR and those remotely located.
A member portal is offered from which healthcare providers and industry experts may schedule collaborative procedures. Most recently, a partnership with Medtronic Neurovascular for a remote connectivity program was established.
Caresyntax
Caresyntax®, as it is known currently, was originally founded in 2013 in Germany, and it expanded to the US in 2017.
This version of Caresyntax resulted from relocation and unification of the former medical technology company S-CAPTE GmbH and is a vendor-neutral platform marketed to gain insight into patient outcomes, operational efficiency, and profitability. The company’s High-Fidelity Surgical Record™ works across the preoperative, intraoperative, and postoperative continuum. This includes electronic health records, hospital information systems, supply/implant data, device data, imaging and surgical video, all with anonymization and privacy controls.
Currently, general/visceral, orthopaedic, and cardiovascular surgeries are targeted, though many surgeries are compatible with the technology. The goal is to make surgery smarter and safer.
According to the company, as of 2023, approximately 2,800 ORs and 32,000 surgical teams employ this technology. The University Hospital Virgen del Rocio in Seville, Spain, streams procedures for students with the technology in real-time,4 while Saint Thomas Hospital in Nashville, Tennessee, was able to mitigate the start times of delayed cases and have a 40% decrease in after-hour cases based on Caresyntax analytics.5
Similar collaborations have taken place with the University of California (UC) San Diego Medical Center and the University Hospital Dresden in Germany to improve OR efficiency and patient care. A 2023 publication showed the potential of Caresyntax to reduce costs and improve patient quality of life among colorectal surgeries.6
Caresyntax recently developed an app for surgeons, InfluenceOR™, to allow for board-certified review of individual surgeon’s procedures, analyzing surgical technique and decision-making through video-based assessments to promote data-driven surgery.
C-SATS
Crowd-Sourced Assessment of Technical Skills (C-SATS™) is a video capture platform aimed at improving surgical proficiency through performance analytics, self-learning opportunities, and peer-to-peer interactions. C-SATS was founded in 2014, at the University of Washington and was acquired by Johnson & Johnson in 2018.
The technology functions through video capture from any minimally invasive surgery (MIS) platform (agnostic) to a cloud-based HITRUST CSF®-certified private case library. The data are subjected to video-based, AI-driven actionable and objective insight. The platform also includes access to 19,000 procedure videos and an expert case series across 10+ specialties, including bariatric, gynecology, urology, thoracic, and colorectal surgery.
The surgical library serves as a resource for reviewing diverse surgical approaches, observing procedures conducted by accredited surgeons, accessing insights and strategies for navigating challenging surgical maneuvers, and participating in a surgeon-centric community for inquiries and discourse. The community fosters mentorship opportunities, catering to both experienced surgeons pursuing ongoing refinement and feedback and trainees in remote regions seeking guidance. Additionally, surgeons who contribute as case reviewers receive compensation for their time.
A 2022 systematic review compared crowd-sourced reviews to traditional expert reviews, revealing that C-SATS consistently delivered accurate evaluations of surgeons’ technical proficiency in less time than expert evaluations.7
In 2017, UC Irvine’s Urology Department employed C-SATS for assessing incoming residents’ surgical technique, achieving favorable inter-rater agreement with expert physician assessors.8 This technology holds promise for enhancing the efficiency and applicability of expert evaluations across various surgical domains.
OR Black Box
OR Black Box (ORBB) technology, invented by surgeon Teodor Grantcharov, MD, PhD, FACS, in 2007, aims to enhance patient surgery throughout the perioperative process. It has been implemented in various hospitals worldwide; in the US, this technology currently is being used at Stanford University in California, Duke University in Durham, North Carolina, The University of Texas Southwestern in Dallas, and Northwell Health Long Island Jewish Medical Center in New Hyde Park, New York.
ORBB records diverse data during surgery such as OR audio-video recordings, patient vitals, and surgical instrument usage feedback, fostering transparency and a culture of accountability in the OR.
The collected surgical data are transmitted to computer servers and the ORBB server, where a blend of AI and trained analysts assess the information to identify performance patterns, OR efficiency, safety checklist adherence, and team communication trends. The data then can be used to furnish feedback to individual surgeons and support teams, enabling them to learn from errors and enhance their skills over time.
A 2022 study used ORBB to track checklist compliance, engagement, and quality.9 The study identified weaknesses in checklist quality, and using the data, the hospital had solid evidence to enforce policy change and track follow-up results that showed improvement in checklist application. A prospective cohort study of laparoscopic surgeries using ORBB at the University of Toronto in Canada revealed frequent disruption and error from various sources, with auditory distractions reported a median 138 times per case.10 Equipment, such as surgical instruments and laparoscopic consoles, were either absent or malfunctioning in one-third of the cases, and technical error was recorded a median of 20 times per case.10 These deficiencies led to a median of eight adverse events per case.
Orpheus Medical
Orpheus Medical, founded in 2010 by Tovi Carmon, was later acquired by Intuitive Surgical in 2020. The technology integrates with the da Vinci robotic surgical system and other video integration systems to capture, stream, and view, surgical video using a touch panel or a remote application.
The videos can be stored on the data cloud and reviewed from a mobile device or computer by the surgical team with patients postoperatively or for more accurate postoperative documentation. The platform allows for real-time collaboration, consultation, or conferencing, broadening the walls of the OR. Additionally, the video, audio, and images may be edited and annotated, demonstrating practical benefits for quality improvement and education.
Proximie
Proximie was founded in 2016 by Nadine Hachach-Haram, MD, FACS, a consultant plastic surgeon, as a tool to allow surgeons to connect virtually to any OR, effectively widening access to surgical expertise in areas with low access.
According to Dr. Hachach-Haram, the vision of this platform is to “democratize surgery through better data by connecting every OR and Cath Lab in the world.” The platform is virtually interactive and is currently used for telesurgery and telementoring purposes. Learners can either virtually observe in real time or via the surgical case library. Surgeons can request intraoperative remote consultations, reducing access barriers in rural settings.
The surgical case library can be used to review personal surgeries, pinpoint weak points, and obtain data-driven feedback. The collective library creates a transparent environment to compare procedures performed by different surgeons in different hospitals to reveal patterns and trends with the overall goal of reducing adverse events and driving productivity. Proximie is currently used within 500 hospitals around the world in more than 50 countries and by 16,000+ users. This technology is for all surgical specialties.
The technology works through a broadcast of four simultaneous live video feeds to multiple users, including from any medical device (imaging, cameras, navigation, fluoro, robotics, scopes, ultrasound, electrocardiogram, and any other device with a video output), as well as regular cameras (webcams, room cameras, overhead cameras, wearables, etc.).
In practical use, the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) used the platform during the COVID-19 pandemic for remote training to teach the Acquisition of Data for Outcomes and Procedure Transfer hernia program. In post course surveys, participants reported that the course was effective in meeting their educational goals.11
Sony NUCLeUS
Sony NUCLeUS™ is a digital imaging platform that streamlines the collection, management, and distribution of medical video content across a hospital’s network.12 NUCLeUS came to the US in 2019 after a successful launch in Belgium and Sweden. It is a scalable, vendor-neutral platform that can accommodate virtually any video source, from a wide range of modality manufacturers.
The platform also can capture and record images from endoscopes, robotic-assisted surgical devices, microscopes, angiography systems, C-arms, ultrasound scanners, and more. NUCLeUS is a powerful digital imaging platform that can help hospitals improve the efficiency and effectiveness of their surgical workflows. It also can help improve patient care by providing remote access to medical video content.
NUCLeUS enables real-time surgical live streaming to remote specialists, students, and colleagues, facilitating immediate collaboration, consultation, and educational opportunities. This service enhances healthcare providers’ competencies and knowledge, thereby potentially enhancing patient outcomes.
Furthermore, NUCLeUS serves as a valuable tool for the acquisition and analysis of medical video data for clinical research endeavors, contributing to the advancement of disease understanding, treatment modalities, and the development of improved therapies.12
There is a growing body of research on the use of AI to measure OR efficiency. These programs aim to optimize various aspects of OR management, resource allocation, and workflow. However, there is currently no single AI program that can reliably measure all objective quantitative and qualitative metrics in the OR.
Theator
Theator, an Israeli startup based in Palo Alto, California, was founded by Tamir Wolf, MD, PhD. After experiences with seeing family and coworkers undergo the same procedure—but with drastically different outcomes due to variability in surgery and clinical management—this trauma surgeon wanted to reduce variability in treatment approaches. Thus, he founded Theator, a technology currently implemented in laparoscopic and robotic surgeries.
The software uses machine learning and computer vision to structure and compare surgical video footage, connecting procedural outcomes with specific steps and events to identify successful characteristics as well as deviations from best practices. Real-time analysis generates actionable insights shared with physicians, teams, and institutions to enhance patient care and inform future procedures.
The Theator library contains 30,000+ hours of surgical data with a billion frames, offering reliable and efficient AI-driven video review and metric analysis for skill improvement among trainees and experienced surgeons.
Theator technology has been incorporated into multiple large-scale settings, including by Tel Aviv Sourasky Medical Center (Ichilov Hospital) in Israel, the Mayo Clinic in Rochester, Minnesota, the Canadian Association of General Surgeons, McGill University in Montreal, Quebec, as well as SAGES as part of its VBA program to measure competency for the relevant procedures.
However, even with an increasing number of centers adopting this kind of technology, an article published in 2020 by researchers from Stanford University found that while AI is a promising quality assurance tool, there is still room for refinement and a need for physician oversight, especially in complex cases.13
Touch Surgery
Touch Surgery, founded in 2013, was designed by four surgeons based in London, UK. These surgeons had the goal of taking surgery education to the next level and began conceptual development in 2010.
The way the surgeons reached their goal is twofold: the Touch Surgery Enterprise and Touch Surgery Simulations. Enterprise is marketed as an AI-powered surgical video and analytics platform to help train and discover new techniques. Simulations is available as an iPhone and iPad app designed to help trainee surgeons.
This technology allows for AI to automatically segment videos from the OR into key procedural steps, with automatic uploads to secure cloud storage. Surgeons have the option to add annotations to these videos both during and after surgery to facilitate navigation through the library.
Proprietary technology blurs the video recording outside of the body to allow for patient and staff privacy. Actionable information can be obtained by comparing individual cases to a historical database of cases and by examining cross-departmental reports to identify variations in techniques. The repository of cases is designed to promote best practices at both the individual and higher levels, enabling feedback from peers. This technology is applicable across various specialties and is compatible with numerous laparoscopic and surgical robotic systems.
