RGSQ: Riemannian Geometry-Sensitive Quantization for Large Vision-Language Models
Zhiping Wu, Dongdong Ren, Yangchengyu Zhou, Zhengjie Zhang, Wenbin Li, Hongbing Pan, Yang Gao
cs.CV · cs.AI
Quick verdict
Proposes modality-aware Fisher-metric PTQ for VLMs with strong low-bit gains, but table label inconsistencies and missing rotation baselines weaken evidence.
Summary
This paper proposes RGSQ, a post-training quantization framework for large vision-language models built on a modality-aware Fisher–Riemannian metric. Per linear layer, activation and gradient covariances are computed separately for vision and text tokens, weighted by token-gradient sensitivity, and fused into a Kronecker product; a whitening transformation then converts the Riemannian objective into an equivalent Frobenius form, so standard solvers such as GPTQ, AWQ, and SmoothQuant can be reused unchanged, and sparse Givens rotations steer residual quantization noise toward low-sensitivity directions. Evaluation spans LLaVA-OneVision (7B/72B), InternVL2 (8B/26B), and Qwen2-VL (7B/72B) on six image benchmarks plus video generalization. Reported highlights: under W3A16, LLaVA-OneVision-72B averages 74.1 versus 74.3 FP16 while AWQ collapses to 55.1; under W2A8, RGSQ beats MBQ by 4.0–4.2% on the two large models; and the component ablation on LLaVA-OneVision-7B W4A8 climbs from 56.1 to 68.4, above the 67.5 FP16. Main limitations: the abstract's 'up to 5.9%' gain is not traceable to any table; Table VI baseline rows duplicate numbers under different method labels; promised QuaRot/SpinQuant/Q-VLM baselines are absent from main tables; and no variance or significance testing supports results that exceed FP16.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Quantization error in VLMs is anisotropic and modality-dependent, so a unified modality-aware Kronecker–Fisher metric prioritizes calibration capacity better than Euclidean reconstruction objectives.
- Whitening converts the Riemannian objective into an exactly equivalent Frobenius form, letting unimodal PTQ solvers (GPTQ, AWQ, SmoothQuant) optimize multimodal error unchanged (Table VI: +1.6 to +1.9% on InternVL2-8B W4A8).
- Sparse Givens rotations steer residual quantization noise toward low-sensitivity directions, giving the largest gains at extreme low bits: +4.0–4.2% over MBQ under W2A8 on Qwen2-VL-72B and InternVL2-26B (Table VII), and up to +14.6% over RTN.
- The component ablation on LLaVA-OneVision-7B W4A8 attributes +8.0 points to the metric (56.1→64.1), +2.5 to whitening (→66.6), and +1.8 to steering (→68.4, above the 67.5 FP16) (Table IV).
- The image-calibrated metric generalizes to video benchmarks without video calibration, improving over MBQ by 1.6–3.1% on SEED-Video and Video-MME (Table VIII).
Strengths
Exact, reusable whitening reduction with verified plugin benefit
The identity ||ΔW||²_{S⊗A} = ||S^½ΔW A^½||²_F (Eqs. 13–15) is derived correctly and enables plug-and-play reuse of GPTQ/AWQ/SQ solvers; Table VI empirically shows +1.6–1.9% average uplift for InternVL2-8B W4A8 across three wrapped baselines, and whitening factors are absorbed offline so weights incur zero runtime overhead.
Unusually broad empirical coverage for a PTQ paper
Three model families at 7B–72B, six image benchmarks plus SEED-Video/Video-MME, four bit configurations (W3A16/W4A8/W2A8/W3A8), and a component ablation (Table IV) that cleanly attributes gains: 56.1 → 64.1 (RMAM) → 66.6 (ROW) → 68.4 (GGES) on LLaVA-OneVision-7B W4A8.
Mechanism-level diagnostics support the core thesis
Honest cost accounting
Consistent, fully specified experimental protocol
Weaknesses
Cross-table baseline bookkeeping is internally inconsistent
Most relevant rotation baselines absent from main tables
No uncertainty quantification despite FP16-surpassing claims
Headline abstract numbers are unverifiable from the provided tables
Prose overstates its own plugin result
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is RGSQ quantization?
RGSQ is a post-training quantization method for large vision-language models. It measures quantization error under a modality-aware Fisher–Riemannian metric that weights vision and text statistics by gradient sensitivity, converts that objective to a standard Euclidean form via whitening so solvers like GPTQ and AWQ work unchanged, and uses sparse Givens rotations to push residual error toward low-sensitivity directions. Calibration costs about 2.1 GPU-hours for a 7B model and adds essentially no inference-time operators for weights.
How much better is RGSQ than MBQ or MQuant?
In the paper's tables, RGSQ beats MBQ by about 4.0–4.2 points of six-benchmark average under W2A8 on Qwen2-VL-72B (78.7 vs 74.5) and InternVL2-26B (80.4 vs 76.4), and by roughly 0.3–2.5 points at the milder W3A16/W4A8 settings. The abstract's 'up to 5.9%' claim could not be matched to any table in the text and likely refers to figure-only low-bit results.
Does RGSQ slow down inference?
According to the authors, weight rotations are absorbed offline into the quantized weights, so there is zero runtime overhead on the weight side; the activation rotation uses K=4 sparse Givens layers costing about 4 multiply-adds per channel, much cheaper than the dense rotations used by QuaRot- or SpinQuant-style methods.
Is the RGSQ code available?
The paper states code is publicly available at github.com/RL-MIND/RGSQ and uses the public LMMs-Eval harness with ShareGPT4V COCO-caption calibration data. This review could not verify the repository's existence or completeness.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 2–5 are not analyzable from the extracted text; low-bit W2A8/W3A8 detail beyond Table VII averages is unavailable.
- Abstract headline numbers (up to 5.9%, 8.6%) could not be located in any extractable table; they presumably live in Figs 3–4.
- The IEEE-style future-dated header (Vol. 14, No. 8, July 2027) suggests a preprint/template; peer-review status unknown.
- Code repository (github.com/RL-MIND/RGSQ) existence and completeness not verified in this review.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.