Efficient Quantization-Aware Distillation with Cross-Modal Alignment for Edge Vision-Language Models
Jinwoo Jeon, GyuYeop Do, Yubin Lim, Nam-Joon Kim, Hyun Gon Ryu, Hyuk-Jae Lee, Byung-Jun Lee
cs.CV
Quick verdict
Unifies QAT with teacher-anchored distillation and a cross-modal adapter, lifting edge-VLM non-RGB accuracy; but evaluation lacks seeds and clean ablations.
Summary
The paper targets open-vocabulary classification (OVC) on edge devices with heterogeneous sensors (RGB plus depth/thermal). It argues that EdgeVL's two-stage pipeline - full-precision distillation followed by QAT with a student-space triplet loss - is self-referential and degrades under quantization noise. The authors unify distillation and QAT in one stage, anchoring the quantized student to frozen CLIP ViT-g-14 embeddings via relational KD (distance and angle terms), a symmetric InfoNCE loss against teacher embeddings, and a supervised contrastive term built on teacher pseudo-labels (confidence threshold 0.25). A single-layer cross-attention adapter (non-RGB queries, RGB keys/values, residual connection) enhances non-RGB features, adding about 9MB (roughly 16% of the Swin-T backbone). On EuroSAT and ScanNet with ViT-S/DAT-T/Swin-T at Int8, non-RGB top-1 accuracy improves in all six backbone-by-dataset cells (up to +4.9pp for ViT-S on ScanNet), multimodal accuracy rises up to +5.9pp (Tab. 2), and training time drops from 12h to 2.25h (5.3x), at 13.5-23.9% lower inference throughput. Cross-dataset gains on NYUv2/SUN-RGBD hold for ViT-S and DAT-T but not Swin-T. Main limitations: single-run results without variance or significance testing, configuration selection by best test non-RGB accuracy, a confounded and apparently mislabeled adapter ablation, and no code release.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 3
● low 3
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Anchoring QAT contrastive supervision to frozen CLIP embeddings (instead of EdgeVL's student-space triplet loss) suppresses hard negatives and enlarges decision margins under quantization (Sec. 3.2, Fig. 3).
- Unifying distillation and quantization into a single stage reduces training time by 81% (12h to 2.25h, 5.3x speedup) while improving accuracy (Tab. 3).
- A single-layer cross-attention adapter transferring RGB semantics to non-RGB features improves non-RGB and multimodal accuracy by 1.3-2.2pp on EuroSAT at roughly 9MB added size (Sec. 4.1, Tab. 8).
- Gains hold across ViT-S, DAT-T, and Swin-T on EuroSAT and ScanNet and transfer to NYUv2 and SUN-RGBD for ViT-S and DAT-T, but Swin-T cross-dataset results are inconsistent (Tabs. 4-5).
Strengths
Well-motivated unified objective with a decisive ablation
Sec. 3.2 replaces EdgeVL's student-space triplet loss with teacher-anchored RKD + InfoNCE + supervised contrastive terms; Tab. 6 shows RKD alone collapses (3.8% RGB / 5.1% non-RGB on ScanNet) while the full loss reaches 50.4/47.2, directly supporting the central 'semantic anchoring' argument.
Consistent in-distribution accuracy gains
Tab. 1: non-RGB top-1 improves in all six backbone-by-dataset cells, up to +4.9pp (ViT-S ScanNet 42.0 to 46.9) and +4.7pp (DAT-T EuroSAT 61.0 to 65.7); Tab. 2 multimodal accuracy rises up to +5.9pp (DAT-T EuroSAT 65.5 to 71.4).
Large, concretely reported training-efficiency gain
Cross-dataset generalization protocol with candid negative results
Weaknesses
No statistical rigor
Configuration selection appears test-set driven
Confounded and apparently mislabeled adapter ablation
Efficiency framing omits deployment cost of the adapter
Reproducibility gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What problem does this paper solve?
It addresses deploying CLIP-style open-vocabulary vision-language models on resource-limited edge devices that capture both RGB and non-RGB inputs (e.g., depth or thermal). The method distills CLIP semantics into a small Int8-quantized encoder in a single training stage, replacing the earlier two-stage distill-then-quantize pipeline of EdgeVL.
How much faster is training, and what is the trade-off?
Training drops from 12 hours (EdgeVL) to 2.25 hours, a 5.3x speedup, because distillation and quantization-aware training are unified in one stage. The trade-off is inference: the added cross-modal adapter costs about 9MB of model size and reduces TensorRT-measured throughput by 13.5% (Swin-T) to 23.9% (ViT-S).
Which datasets and tasks are evaluated?
Open-vocabulary classification (top-1 accuracy) on EuroSAT satellite imagery (10 land-use classes) and ScanNet indoor scenes (21 classes, subsampled), with Int8 quantization and ViT-S, DAT-T, and Swin-T student backbones. Cross-dataset generalization is tested by training on ScanNet and evaluating on NYUv2 and SUN-RGBD.
Does the method always beat EdgeVL?
Not everywhere. In-distribution, non-RGB accuracy improves on all six backbone-dataset combinations (up to +4.9 points) and multimodal accuracy up to +5.9 points. However, on cross-dataset transfer Swin-T underperforms EdgeVL on SUN-RGBD (non-RGB 24.8 vs 30.0), and one RGB result slightly drops (Swin-T EuroSAT 66.7 vs 67.1).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-3 are described only via captions in the extracted text; the decision-margin evidence supporting a central claim (Fig. 3) cannot be verified quantitatively.
- Equations are OCR-garbled but reconstructable; all result tables (1-8) are present and legible.
- This is an arXiv preprint (2609.16689, Sep 2026) and has apparently not undergone peer review.
- No appendix or supplementary material is included in the extraction, so implementation details beyond the main text could not be checked.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.