TEDi: Temporal Memory-Enhanced and Denoising Transformer for Surgical Instrument Segmentation
Jiahong Yuan, Weiming Mi, Tao Zhang, Haoyin Zhou
cs.CV
Quick verdict
TEDi adds query memory and temporal denoising to Mask2Former for instrument segmentation; EndoVis SOTA credible, but gains modest and lack statistical tests.
Summary
TEDi is a query-based surgical instrument segmentation framework built on Mask2Former that injects temporal information at the query level rather than the pixel level. In a two-stage design, a Swin-S/Mask2Former segmenter (N=10 queries) is fine-tuned and frozen; then a Memory Search Enhancement (MSE) module encodes predicted masks into query space, fuses them with queries via learnable gating, stores the last K=3 frames in a FIFO memory bank, and enhances current queries through stacked memory-attention blocks. A Temporal Consistency Denoising (TCD) decoder aligns current queries to the previous frame's denoised queries via Hungarian matching and refines them with cascaded cross-attention blocks, resetting the reference every T=3 frames. On EndoVis 2017/2018, TEDi reports 80.71/78.30/52.11 and 86.69/84.82/57.41 (challenge/ISINet/mean-class IoU): best on all three EV18 metrics, best Ch_IoU and mc_IoU on EV17, with the largest gains on rare, confusable classes (EV18 clip applier 4.27 to 14.82 over baseline). Ablations attribute +1.47 (MSE) and +1.41 (TCD) Ch_IoU alone, +2.51 combined. Limitations: gains over the authors' own tuned Mask2Former(N=10) baseline are modest (~2.5 points); EV17 ISI_IoU trails QPD despite the 'consistently outperforms' claim; no significance tests or variance are reported; evaluation is limited to two EndoVis benchmarks; code availability statements contradict each other (abstract URL vs 'released after acceptance').
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A query-level memory bank with gated mask encoding and memory attention (MSE) retrieves cross-frame semantic priors that enrich current-frame queries.
- TCD builds an identity-aligned temporal reference via Hungarian matching and cascaded denoising cross-attention, suppressing temporally unstable query predictions.
- TEDi outperforms state-of-the-art methods on EndoVis 2017/2018 (EV18: 86.69/84.82/57.41 Ch/ISI/mc IoU; EV17: 80.71/78.30/52.11), with the largest gains on rare, confusable categories.
- Both modules contribute independently and jointly (ablation on EV18: +1.47 and +1.41 Ch_IoU alone, +2.51 combined), with best performance at K=3, T=3.
Strengths
Targets a documented failure mode with a query-space design
Sec. 1 and 3.3 motivate the work by QBS models' category flicker and confusion among similar instruments; MSE stores compact instance-aware queries with mask-aware gating (Eqs. 1-2) instead of dense pixel memory, explicitly arguing against the redundancy of pixel-level banks (citing XMem, RMEM).
Gains concentrate exactly where the paper claims: category discrimination
On EV18, mc_IoU jumps 48.91 to 57.41 over the authors' tuned M2F(N=10), far more than Ch_IoU (+2.51); rare/confusable classes improve sharply: Clip Applier 4.27 to 14.82, SI 54.00 to 69.96, Ultrasound Probe 0.00 to 9.59 (Table 1 and 2).
Causal, streaming-friendly design with honest efficiency accounting
Clean module-level ablation and hyperparameter sweep
Mechanistic diagnostic linking modules to behavior
Weaknesses
Modest margin over the authors' own tuned baseline
'Consistently outperforms' is overstated
No uncertainty quantification
Reproducibility gaps
Short temporal horizons and narrow evaluation scope
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is TEDi and how does it improve surgical instrument segmentation?
TEDi is a transformer model that segments and classifies surgical instruments in endoscopic video. It starts from a Mask2Former-style query-based segmenter and adds two temporal modules: a query-level memory bank that retrieves semantic information from the previous 3 frames (Memory Search Enhancement), and a denoising decoder that aligns current predictions with the previous frame's stabilized predictions to reduce category flickering (Temporal Consistency Denoising).
How accurate is TEDi on the EndoVis 2017 and 2018 benchmarks?
The paper reports on EndoVis 2017: 80.71 challenge IoU, 78.30 ISINet IoU, and 52.11 mean class IoU; on EndoVis 2018: 86.69, 84.82, and 57.41 respectively. This is the best reported result on all three EndoVis 2018 metrics and the best challenge/mean-class IoU on EndoVis 2017, though on EndoVis 2017 the ISINet IoU (78.30) is slightly below the QPD method (79.58).
Is the TEDi code publicly available?
The paper is inconsistent: the abstract links to github.com/argon-xixi/TEDi, but the main text states the code will be released after acceptance. Readers should check the repository directly; as of the paper's writing its completeness is unverified.
Is TEDi fast enough for real-time surgical video processing?
The paper reports 57.96 ms per clip at 512x512 resolution on an NVIDIA A100 GPU, with the memory bank adding only about 5.5 ms and peak GPU memory of 666 MB. This is close to real-time speed, though whether it qualifies depends on whether the latency is per window or per output frame, which the paper leaves ambiguous.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-3 (architecture, qualitative comparison, query-confidence plots) cannot be assessed from the extracted text; claims resting on them are taken as stated.
- Tables 1-2 were extracted as concatenated text; per-category numbers were parsed carefully but column-alignment errors are possible (e.g., Table 2 K/T columns).
- Accessibility of github.com/argon-xixi/TEDi could not be verified (no network access).
- Abstract and body contradict on code availability; this review reports the discrepancy rather than resolving it (codeAvailable reflects the abstract's stated URL).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.