Fragment-Aware Vision Transformers for Fresco-Fragment Style Classification
Sara Miketek, Biagio Barchielli, Nadeem Iqbal Kajla, Sinem Aslan
cs.CV · cs.LG
Quick verdict
Masking+inpainting+KL-contrastive+logit ensemble for fresco fragments; promising gains (CLEOPATRA 0.604→0.656 acc) but single-run, no significance tests.
Summary
The paper tackles style classification of fresco fragments, where only incomplete, irregular pieces of a wall painting survive. On ViT-B/16 the authors build a progressive framework: a baseline; a foreground-guided masked variant that suppresses background-only patch tokens via attention masks; an inpainted variant trained on fragments extrapolated to full squares with Criminisi or LaMa inpainting; a KL-based supervised contrastive loss computed on predictive distributions; and a learnable logit ensemble fusing the three branches. A graph-attention fusion variant is evaluated but deliberately demoted. On CLEOPATRA (11 classes), the ensemble raises accuracy from 0.604 to 0.656 and macro-F1 from 0.596 to 0.648 over the baseline, with the KL term improving every single branch; graph fusion reaches 0.681/0.677 but is reported only as a comparison. On POMPAAF (4 Pompeian styles, six fragmentation settings), the ensemble beats the best single branch in four of six settings and ties graph fusion (differences ≤0.004). An interpretability study (attention rollout, SHAP-BPT, region-level attribution, controlled input ablations) shows all branches place at least 72% of positive attribution on real painted surface, while the inpainted branch draws 0.284 from the synthesized background versus roughly 0.10 for the others. Key limitations: single-run results without significance tests, only eight CLEOPATRA test artworks, no POMPAAF validation split, incompletely specified hyperparameters, and an unexplained failure to reproduce Elkin et al.'s published POMPAAF numbers.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Fragment-aware modeling improves over a standard ViT-B/16 baseline: the learnable logit ensemble raises CLEOPATRA accuracy from 0.604 to 0.656 and macro-F1 from 0.596 to 0.648 (Table 2).
- The KL distribution-level contrastive objective consistently improves all three single branches (e.g., CLEOPATRA baseline accuracy 0.604→0.611, Table 2).
- A simple learnable logit ensemble matches the far more complex graph-fusion variant on POMPAAF (differences ≤0.004, Table 3), so graph fusion is treated as a secondary component.
- The inpainted branch is the strongest single model but draws 0.284 of its positive attribution from the synthesized background versus 0.096/0.106 for baseline/masked (Table 4), a quantified caution for heritage use.
- Attribution and controlled ablation analyses show the models rely on internal painted evidence rather than fragment silhouette or background.
Strengths
Honest reporting of negative and cautionary results
The paper openly reports failing to reproduce Elkin et al.'s POMPAAF numbers under the same splits and training strategy (Sec. 4.2), quantifies the inpainting branch's reliance on synthesized pixels (Table 4: 0.284 background attribution vs 0.096/0.106 for baseline/masked), and demotes its own graph-fusion variant after finding no consistent gain.
Clean progressive ablation design
Baseline → masked → inpainted → ensemble, each component isolable against the ViT-B/16 reference; the KL contrastive term improves all three single branches on CLEOPATRA (Table 2: 0.604→0.611, 0.616→0.624, 0.623→0.636 accuracy), supporting the 'consistently improves' claim on that dataset.
Interpretability that tests rather than assumes
Two benchmarks with six fragmentation protocols and leakage-aware splits
Weaknesses
Single runs, no statistical uncertainty anywhere
The 'small, dataset-specific' framing of graph fusion is contestable
Per-branch results hidden on POMPAAF; quantitative interpretability single-dataset
Hyperparameters and selection protocol under-specified
Unresolved reproduction failure of the POMPAAF baseline
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper addresses style classification of archaeological fresco fragments, where the model sees only an incomplete, irregularly shaped piece on a square canvas rather than a complete artwork. The framework is progressive: a ViT-B/16 baseline (Sec. 3.2); a foreground-guided masked variant whose attention mask forbids interaction with background-only tokens (Sec. 3.3); an 'inpainted' branch trained on fragments extrapolated to full squares via Criminisi exemplar inpainting and LaMa, framed strictly as geometric regularisation rather than reconstruction (Sec. 3.4); a KL-based supervised contrastive loss over predictive distributions (Sec. 3.5); and a learnable logit ensemble, with a graph-attention fusion variant kept as a secondary comparison (Sec. 3.6). The central claims are that fragment-aware modelling beats the standard ViT, that the KL objective consistently helps every branch, that the simple ensemble matches the complex graph fusion, and that the models rely on genuine painted evidence — with the quantified caveat that the inpainted branch partly leans on synthesized pixels. The claims are largely supported by the presented tables, though the fusion claim rests on a defensible but contestable judgment call (see below).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What are the CLEOPATRA and POMPAAF datasets?
CLEOPATRA is a benchmark of 11 artistic-style classes built by fragmenting complete paintings (18,458 training and 5,713 test fragments, split at the artwork level so all fragments of a painting stay in one partition). POMPAAF is a benchmark of Pompeian wall-painting fragments in 4 style categories (structural, architectural, ornamental, intricate) from 311 frescoes, generated under several fragmentation protocols: crossing cuts (5 or 20 cuts), square grids (12 or 160 fragments), and eroded Voronoi fragments (12 or 160).
How much does the proposed ensemble improve accuracy?
On CLEOPATRA it raises a ViT-B/16 baseline from 0.604 accuracy / 0.596 macro-F1 to 0.656 / 0.648, beating every single branch. On POMPAAF it beats the best single branch in 4 of 6 fragmentation settings (largest gains at Square-12: 0.714 vs 0.683) and stays competitive in the other two, though the single best branch slightly wins on Cross-20 (0.525 vs 0.516) and Eroded-160 (0.606 vs 0.600).
Does the inpainting branch of the model rely on fake (synthesized) evidence?
Partly. The paper quantifies that the inpainted branch assigns 0.284 of its positive attribution (SHAP-BPT, 100 CLEOPATRA test fragments) to the LaMa-synthesized surround, roughly three times the 0.096-0.106 assigned by the baseline and masked branches. All branches still place the majority of attribution (about 72% or more) on the real painted surface. The authors flag the synthesized-pixel reliance as a caution for heritage use.
Is code available for this paper?
Yes, the paper links a public GitHub repository (fragment_classification) in a footnote, and both datasets are public benchmarks. However, exact hyperparameters (contrastive temperature, loss weights, warm-up epochs, learning rates) are not numerically specified in the text, and results are single runs without seeds.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures (tokenization, masking, inpainting examples, attribution maps in Fig. 5) could not be analyzed from the extracted text; judgments rest on captions and prose only.
- Exact hyperparameter values (τ, α, β, warm-up epochs, learning rates) are absent from the extracted text, so the training protocol could not be fully assessed.
- The contributions list in Sec. 1 contains a garbled bullet ('report that a more complex graph-fusion variant does not consistently improve on it...'), likely a typesetting or extraction artifact; the abstract is also truncated mid-sentence in the source.
- Whether the missing hyperparameters appear in an appendix or the code repository could not be verified from this text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.