[论文] GRAC: Graph-Based Retrieval-Augmented Captioning for Surgical Scene De...

论文概要

研究领域: CV
作者: Rochak Dhakal, Chen Zhao, Soup Cobb, Xin Qian, Xiaofeng Yang
发布时间: 2026-11-06
arXiv: 2611.09866

中文摘要

目的：外科手术报告支持手术记录、外科培训、绩效评估和工作流程审查，但提取有意义的信息通常需要大量人工审查和临床专业知识。自动化手术场景描述可以通过利用重复出现的手术活动和先验知识来减轻这一负担。方法：本文提出 GRAC（基于图的检索增强字幕），将手术场景表示为捕获器械、动作和解剖结构之间关系的图。给定输入场景图，GRAC 学习从训练数据中检索相关图-字幕对，比较检索示例与输入场景的器械-组织交互，使用比较特征在生成过程中条件化检索字幕表示。结果：在 EndoVis-18 基准上，GRAC 取得 5.88 的 CIDEr-D（SGT 为4.23，最强传统 Transformer 基线为3.36）；器械-动作-目标三元组集合精确准确率达52.6%（SGT为32.7%，最强传统基线为26.2%）。在 TORS 上，GRAC 取得 5.06 的 CIDEr-D（最强基线为3.80）。结论：结合结构化手术场景表示与检索示例能改善手术场景字幕并产生更具上下文意义的描述。

原文摘要

Purpose: Surgical operative reports support procedural documentation, surgical training, performance assessment, and workflow review. However, extracting meaningful information often requires extensive manual review and clinical expertise. Automated surgical scene descriptions could reduce this burden by leveraging recurring surgical activities and prior knowledge. Methods: We propose GRAC (Graph-Based Retrieval-Augmented Captioning), a framework that represents surgical scenes as graphs capturing relationships among instruments, actions, and anatomical structures. Given an input scene graph, GRAC learns to retrieve relevant graph-caption pairs from training data. It compares instrument-tissue interactions between retrieved examples and the input scene, using comparison features to conditi...

自动采集于 2026-11-10

#论文 #arXiv #CV #小凯