JEDI: JEPA-to-Edge Distillation for Efficient Cropland Segmentation from Satellite Imagery
Kishor Kumar Bhaumik, Nicolas Roque dos Santos, Jia Chen, Evangelos E. Papalexakis
cs.CV · cs.LG
Quick verdict
Two-stage JEPA-to-SegFormer distillation reaches 68.0 mIoU on CalCROP21 (teacher 70.0); gains well-ablated but single-dataset, binary-task, GPU-only evidence.
Summary
The paper proposes JEDI, a two-stage knowledge-distillation framework that compresses a 639M-parameter I-JEPA ViT-H/16 teacher into compact SegFormer encoders for binary cropland segmentation on CalCROP21. Stage 1 aligns the student's terminal-stage features to the teacher's 784x1280 token space via a linear projection plus bilinear upsampling, trained as pure feature reconstruction with no labels. Stage 2 jointly optimizes supervised cross-entropy, temperature-scaled (tau=2) response distillation, and a persistent feature-alignment loss retained throughout task adaptation. A preprocessing step selects one NDVI-maximizing Sentinel-2 acquisition per tile, cutting inference to a single pass. On CalCROP21, JEDI-B0 (4.04M parameters) reports 68.0±0.4 mIoU versus 52.0 for the standalone student, 65.9 for the strongest compared baseline (CIRKD), and 70.0 for the teacher; B1 and B2 reach 69.0 and 69.5. Ablations attribute the largest single-component drop to removing Stage-1 alignment (−5.0), followed by persistent alignment (−3.0); teacher-swap experiments show I-JEPA (68.0) far outperforming DINO (36.0) and supervised ViT (54.5) teachers. Reported efficiency: 5.8 GMACs and 7.1 ms on an A6000 for B0 versus 557.5 GMACs for the teacher. Limitations acknowledged by the authors: binary task only, a single region/dataset, and loss of temporal information from NDVI selection; efficiency is measured on a datacenter GPU rather than edge hardware. Five-seed means±std are reported without significance tests.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A two-stage framework can transfer an isotropic I-JEPA ViT-H teacher's representations (784x1280 tokens) into a hierarchical SegFormer student despite simultaneous mismatches in embedding dimension, spatial resolution, and topology, using only a linear projection plus spatial resampling.
- Persisting the feature-alignment objective throughout task adaptation is the key design: γ=0 drops JEDI-B0 from 68.0 to 65.0 mIoU, and JEDI beats KD, SKD, CWD, and CIRKD at all three student capacities (Table II).
- JEDI-B0 reaches 68.0 mIoU on CalCROP21 with 4.04M parameters — a 16.0-point gain over the standalone student and within 2.0 points of the 639M-parameter teacher (70.0), a 158× parameter reduction.
- Teacher representation quality dominates distillation outcome: under the identical pipeline, I-JEPA yields a 68.0 mIoU student versus 36.0 for DINO and 54.5 for a supervised ViT (Table IV).
Strengths
Real cross-architecture mismatch solved with a minimal, well-argued module
Teacher emits one flat 784x1280 token sequence while SegFormer yields a four-stage pyramid (Table I); the alignment module hϕ (flatten → single linear projection → bilinear upsample to 28x28) resolves dimension, resolution, and topology at once. Keeping the projector deliberately linear so the retained encoder carries the representational burden is explicitly justified (Sec. III-D), as is upsampling the student rather than pooling the teacher.
Above-average baseline fairness
SKD, CWD, and CIRKD feature-level objectives are routed through the same alignment module hϕ as JEDI (Sec. IV-B), under an identical frozen teacher, data, and training schedule — removing the usual confound of hand-tuned baseline adaptation. All results use five seeds with mean±std.
Clean component isolation
Honest efficiency accounting and limitations section
Weaknesses
Edge claims rest entirely on datacenter-GPU measurements
Unexplained negative transfer in the teacher-swap experiment
No statistical testing and suspiciously regular values
Internal inconsistency between Conclusion and Table III
Discussed-but-uncompared baselines and narrow scope
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
JEDI (arXiv 2609.07915, UC Riverside) is a method paper proposing a two-stage, cross-architecture knowledge-distillation framework that transfers representations from a 639M-parameter I-JEPA ViT-H/16 teacher to compact SegFormer (MiT-B0/B1/B2) students for binary cropland segmentation on CalCROP21. Central claims: (1) the two-stage design with persistent feature alignment beats standalone training and four prior distillation methods (KD, SKD, CWD, CIRKD) at every capacity (Table II); (2) JEDI-B0 reaches 68.0±0.4 mIoU with 4.04M parameters — 158× smaller and within 2.0 points of the 70.0 mIoU teacher; (3) persistent alignment matters most under aggressive compression. A secondary claim is that I-JEPA is a far better distillation source than DINO or a supervised ViT (Table IV).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is JEDI and how much smaller is the student than the teacher?
JEDI (JEPA-to-Edge Distillation) is a two-stage knowledge-distillation framework that transfers representations from a 639M-parameter I-JEPA Vision Transformer teacher to compact SegFormer networks (4.04M–28M parameters) for cropland segmentation. The smallest student is about 158 times smaller than the teacher and scores 68.0 mIoU versus the teacher's 70.0 on the CalCROP21 benchmark.
What does persistent feature alignment mean in this paper?
Most distillation pipelines stop aligning features once task training begins. JEDI keeps a feature-reconstruction loss — matching the student's projected features to the teacher's tokens — active during supervised segmentation training. Removing it (the γ=0 ablation) drops the B0 student from 68.0 to 65.0 mIoU, and removing the Stage-1 alignment initialization drops it to 63.0.
What data was used, and is the task binary or multi-class?
The paper uses CalCROP21: multi-temporal Sentinel-2 imagery over California's Central Valley with USDA-CDL-derived crop labels, cast as a binary cultivated-versus-non-cultivated segmentation task at 448x448 resolution. Only one image per tile is fed to the network, selected by maximum mean NDVI (a vegetation-greenness index); temporal sequences are otherwise discarded.
Is code available, and can I reproduce the results?
The paper links a GitHub repository (Kishor-Bhaumik/JEDI) and CalCROP21 is public, with complete hyperparameters listed (AdamW 1e-4, batch 8, 30+20 epochs, τ=2, loss weights 1.0/0.5/0.5, five seeds). Practical barriers are compute: fine-tuning the 639M ViT-H teacher for 50 epochs and rerunning six methods across five seeds on a 48GB-class GPU.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Two-column PDF extraction interleaves sentences in the Implementation paragraph of Sec. IV-A (the loss-weight sentence and teacher description overlap mid-sentence); all values remain recoverable and nothing is missing.
- Figures are raster: only captions and scattered axis labels were extracted, so Fig. 1/Fig. 3/Fig. 4 could be checked only against their captions and Tables III/V, not pixel-level.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.