Back to 2026 Abstracts
Real-Time Mixed-Reality Overlays of AI-Segmented Ultrasound for Operator-Agnostic Pleural and Chest Wall Guidance
Philipp Feodorovici1, An-Nhien Vo1, Dhanakumaresh Subramani1, Donatas Zalepugas1, Stefan Senkel1, Hruy Menghesha1, Benedetta Bedetti2, Philipp Schnorrr2, Jan Christian Arensmeyer1, Joachim Schmidt1.
1University Hospital Bonn, Bonn, Germany, 2Helios Hospital Bonn/Rhein-Sieg, Bonn, Germany.
BACKGROUND: Thoracic procedures (e.g., thoracentesis, chest tube placement, chest wall targeting operations) demand accurate spatial orientation. Although ultrasound (US) is real-time and radiation-free, its 2D views are difficult to correlate with anatomy and remain operator dependent. In addition, affordable, anatomically realistic, US-compatible thoracic phantoms are needed for repeatable training and for validating image-analysis algorithms. We aimed to integrate a realistic thoracic phantom with mixed reality (MR) overlays of AI-segmented US to support real-time guidance and reproducible evaluation.
METHODS: Patient CT datasets were segmented with an open-source pipeline to generate printable skeleton and soft-tissue molds. Multi-material phantoms reproducing ribs, intercostal spaces, diaphragm, lung, and configurable pleural effusions were fabricated. Real-time US from an optically tracked probe was streamed to a workstation for inference by a computer vision model that segmented ribs, pleural fluid, lung surface, and diaphragm. Segmentation masks were registered to the phantom and rendered on a head-mounted MR display as holographic overlays. Simulated thoracentesis, small-bore chest tube placement, and chest wall targeting were performed to assess end-to-end latency, overlay stability during probe and needle motion, and qualitative alignment with ground-truth anatomy.
RESULTS: The fabrication workflow yielded consistent anatomical and sonographic landmarks. The guidance pipeline produced stable, anatomically plausible segmentations with minimal latency, enabling smooth MR visualization. Overlays, shown in Figure 1, remained aligned during dynamic probe manipulation and needle advancement, supporting intuitive trajectory planning, depth estimation, and safe intercostal space selection in simulated tasks.
CONCLUSIONS: An integrated MR-US platform coupled to a realistic thoracic phantom is technically feasible and directly oriented to clinical use cases, including bedside thoracentesis, chest tube insertion and chest wall targeting operations. Anticipated clinical benefits include fewer needle passes, improved first-attempt success, and reduced complications through enhanced spatial understanding. Prospective evaluation in human subjects is warranted, with endpoints such as procedure time, first-pass success, number of needle redirections and complication rates. The same platform supports training, competency assessment, and serves as a physical ground truth for advancing imaging algorithms.
LEGEND: Figure 1: Mixed reality headset view showing ultrasound imaging (A) with three-dimensional overlays (B) and needle entry into the thoracic phantom during simulated guidance (C,D).
Back to 2026 Abstracts
