RelightFormer: Feed-forward Generative Transformer for Multiview Object Relighting
Hejun Wang, Jinxi Li, Junwei Jiang, Shiwei Mao, Hu Cheng, Shouwang Huang, Bo Yang
cs.CV · cs.AI · cs.GR · cs.LG · cs.MM
Quick verdict
Feed-forward video-DiT relighting with strong synthetic results and a large open dataset; SOTA claims rest mainly on a self-built benchmark.
Summary
RelightFormer reformulates object relighting as direct feed-forward generation. Building on the Wan2.1 video diffusion Transformer (rectified flow), it concatenates reference and noise tokens and injects the target environment map through a latent illumination cross-attention module, motivated by an analogy to the rendering equation; frame-indexed RoPE is replaced with permutation-invariant camera-based encodings (PRope/GTA). The authors also release the Laval Objaverse Dataset: 90,545 Objaverse objects rendered under 39,008 Laval HDR illuminations (16x rotation-augmented), split by object, lighting, and viewpoint. Trained at 256x256 for 80K steps on 4 H200s, it reports sPSNR 23.80/24.83/25.07 for 1/16/32 views on LOD, beating fine-tuned Neural Gaffer and zero-shot LightSwitch, DiLightNet, and Reli3D; zero-shot transfers to TensoIR and real OLATverse are narrower (Neural Gaffer wins sPSNR/LPIPS in point-light relighting) and RelightFormer-Post roughly matches Reli3D on Stanford-ORB. Main limitations: most quantitative superiority is shown on the authors' own benchmark whose test pairs include source-illumination identity cases, baselines are not retrained on LOD, and ablations use a small 2,989-object subset.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 3
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A latent illumination module injecting the target environment map via cross-attention (Sec. 3.2, Table 5) outperforms channel-wise concatenation for conditioning generative relighting.
- Permutation-invariant positional encodings (PRope/GTA) are essential for symmetric processing of unordered multi-view inputs (Sec. 3.3; 32-view sPSNR drops 23.51 to 17.53 without them).
- LOD — 90,545 objects under 39,008 unique Laval HDR illuminations — is the first large-scale, fully open-source multi-view relighting dataset (Sec. 4, Table 7).
- RelightFormer achieves state-of-the-art visual quality across single-view, multi-view, and novel-view relighting, with monotonic gains as reference views increase (Tables 1, 4).
Strengths
Well-motivated, ablation-backed architectural choices
The latent illumination cross-attention module (Sec. 3.2) is justified by the rendering-equation/attention analogy, and Table 5 quantifies each design: removing permutation invariance drops 32-view sPSNR from 23.51 to 17.53, fixed single-view training from 23.51 to 16.07, and channel-wise concatenation instead of cross-attention from 23.51 to 23.27.
Large, fully open multi-view relighting dataset
LOD (Sec. 4, Appendix B) renders 90,545 Objaverse objects under 39,008 unique Laval HDR illuminations (16x rotation augmentation) with strict object/light/viewpoint splits; Table 7 argues it is the only large-scale, synthetic, publicly accessible resource of its kind.
Unusually broad task coverage
Careful evaluation protocol and honest negative spots
Competitive novel-view relighting at minutes-scale cost
Weaknesses
Headline superiority shown mainly on the authors' own synthetic benchmark
Test protocol contains identity-relighting pairs
Baselines not retrained on LOD, with possible train/test object overlap
Ablations trained on a small subset
No significance testing; several details under-specified
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
RelightFormer — Preliminary Review
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is RelightFormer?
A feed-forward generative Transformer, adapted from the Wan2.1 video diffusion model, that relights objects from one or more input images under a target environment map without any inverse-rendering step. It was accepted to SIGGRAPH Asia 2026 and is accompanied by the open Laval Objaverse Dataset (about 90K objects, 39K illuminations).
How does RelightFormer inject target lighting into the model?
The environment map is tone-mapped, VAE-encoded into tokens, and injected into image features through a dedicated cross-attention module at every Transformer block, motivated by an analogy between attention weights and the rendering equation's material/lighting product.
How well does RelightFormer perform compared to baselines?
On the authors' synthetic LOD benchmark it reports the best scores across 1, 16, and 32 input views (e.g., 25.07 sPSNR at 32 views vs 22.34 for fine-tuned Neural Gaffer). On the real OLATverse dataset gains are narrower, and under rotating point-light relighting Neural Gaffer still wins sPSNR and LPIPS. On Stanford-ORB its post-trained variant roughly matches Reli3D, while slower per-scene optimization methods remain ahead.
Is the code and dataset available?
The paper states code and data are released at github.com/vLAR-group/RelightFormer and describes LOD as fully open-source; this review is based on the paper text and cannot verify the actual repository or download availability.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are known only through their captions; the visual quality of qualitative comparisons (Figs. 1, 3-8) could not be assessed from the extracted text.
- Near the end of Appendix C, the sPSNR passage interpolates between the scaling equation (Eq. 12-13) and its surrounding sentences — a PDF-extraction artifact, not missing content.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.