BS: Take the Hint - Interactive Multitracer PET/CT Lesion Segmentation with a Scribble-Conditioned ResEnc U-Net
Marven Sherif, Amgad Elmasry, Youssef Ghazal, Ayman Elghotni
cs.CV · cs.AI
Quick verdict
Scribble-conditioned ResEnc U-Net for autoPET/CT V raises Dice 0.554→0.751 over five interactive rounds; honest work, but lacks ablations and baselines.
Summary
This paper documents the Brightskies team's submission to the autoPET/CT V challenge on interactive, multitracer (FDG and PSMA) whole-body PET/CT lesion segmentation. The method is a 3D residual encoder U-Net (nnU-Net ResEncL) with four input channels: CT, aorta blood-pool-normalized PET, and sparse foreground/background scribble maps. Weights are initialized from the autoPET-III winning solution, with the input stem extended from two to four channels via zero-initialized scribble convolutions so the pretrained function is preserved exactly at initialization; each fold starts from the matching fold checkpoint to avoid validation leakage. On the challenge's five-fold split, mean Dice rises from 0.5539 unaided to 0.7512 and lesion-level F1 from 0.5281 to 0.7326 after five simulated correction rounds, with about 85% of the gain delivered by the first scribble and fold-to-fold spread compressing five-fold. The authors candidly report that the background scribble channel suffers a train/test semantic mismatch and is nearly inert, that only tumour-bearing cases are scored (571 studies excluded), that the 0.65 threshold was tuned on fold 0, and that the simulated reader is optimistic. Main weaknesses: no component ablations, no comparison against other submissions or the official baseline, and no confidence quantification.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A scribble-conditioned ResEnc U-Net initialised from the autoPET-III winner raises mean five-fold Dice from 0.5539 to 0.7512 and lesion-level F1 from 0.5281 to 0.7326 over five correction rounds.
- About 85% of the interactive gain arrives with the first scribble, with rounds r2–r5 adding only ~0.03 to each metric.
- Interaction compresses between-fold performance spread five-fold (Dice spread 0.1729→0.0344) and reorders the fold ranking, largely compensating for unaided model quality.
- The gain comes almost entirely from recovering missed lesions (829 FN→TP conversions), while background scribbles — 69% of those drawn — are nearly inert due to a train/test semantic mismatch.
Strengths
Leakage-aware pretraining protocol
Fold i is initialised from the matching autoPET-III fold checkpoint (Zenodo [2]) so no validation case enters pretraining (Sec 2.1), and zero-initialised scribble stem convolutions make the extended four-channel network functionally identical to the pretrained two-channel model at step zero (Sec 2.3) — both stated and verifiable.
Candid error accounting
The Discussion quantifies 829 FN→TP conversions versus an FP increase from 6869 to 6998, concedes that background scribbles (69% of the 5426 drawn) are 'close to inert' due to a train/test semantic mismatch, and names the fix — an honest self-audit rarely seen in submission reports.
Label-free, tracer-agnostic PET normalisation
High reproducibility detail
Interesting behavioural finding
Weaknesses
No component ablations
No external baselines
Headline averages exclude 571 lesion-free studies
No uncertainty quantification
Threshold tuned on fold 0 and reused everywhere
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a challenge submission report for autoPET/CT V (interactive, multitracer FDG/PSMA whole-body PET/CT lesion segmentation), not a study claiming a novel architecture. Four central claims, all backed by Tables 1–2: (1) a scribble-conditioned ResEnc U-Net initialised from the autoPET-III winner raises mean five-fold Dice from 0.5539 (r0) to 0.7512 (r5) and lesion-level F1 from 0.5281 to 0.7326; (2) ~85% of the gain arrives with the first scribble; (3) interaction compresses between-fold spread five-fold (0.1729→0.0344 Dice) and reorders the folds; (4) the gain is almost entirely FN→TP recovery (829 conversions), while background scribbles (69% of the 5426 drawn) are 'close to inert' owing to a train/test semantic mismatch. The paper is unusually candid about what its numbers do not show.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the autoPET/CT V challenge task?
It is an interactive lesion-segmentation challenge on whole-body PET/CT covering two tracers (18F-FDG and PSMA). For each scan the algorithm receives the images plus sparse user scribbles marking lesion and non-lesion voxels, and must output a lesion mask; performance is scored over successive correction rounds using volumetric Dice and lesion-level F1.
How much does a single scribble improve this model's segmentation?
In the paper's five-fold cross-validation, one scribble lifts mean Dice from 0.5539 to 0.7223 and lesion-level F1 from 0.5281 to 0.7036 — about 85% of the total five-round gain — while the remaining four rounds add only about 0.03 more to each metric.
Is the code and data available so I can reproduce this?
The paper links a GitHub repository (amged-elmasry-bs/bs_autopetV_submission), the pretrained fold checkpoints are on a public Zenodo record, and the method relies on the challenge's public PET/CT cohort, split file (splits_final.json) and official MetricEvaluator — so reproduction is plausible, though training needs a 32 GB GPU per fold.
Why do background scribbles help so little in this system?
Because of a semantic mismatch: during training the background channel held a thin ring hugging true lesion boundaries, but at evaluation it marks false-positive regions the model itself produced — a pattern the network never saw. The authors report this channel is nearly inert and note that 69% of the simulated scribbles were background.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Mathematical notation is OCR-garbled in places (the Tversky loss equation, '192³' rendered as '1923'); meanings were reconstructed from context and are not affected.
- The extracted text ends inside Table 3 (algorithm details), which appears to be the final page; no further content is visibly missing.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.