REVE: Efficient Hallucination Correction for Large Audio-Language Models via Reused Encoder States
Hongjin Song, Jiasheng Kuang, Xinyu Yang, Qiuyu Fang, Ziyu Wu, Guowu Tan, Xiang Xie
cs.SD · cs.AI
Quick verdict
Proposes reusing LALM encoder states to verify audio event mentions; removes 92.9% of unsupported ones at 0.756 recall, but test sets (n=500) are small.
Summary
The paper addresses event-level hallucinations in large audio-language models (LALMs), whose captions may mention acoustic events absent from the input. Prior post-hoc correction queries an external audio tagger such as CED, which needs a second encoder and an extra forward pass. REVE instead reuses the frame-level encoder states the target model has already computed before its modality projector. Two lightweight readouts are trained on AudioSet-balanced-train: one summarizes frame-level class-score statistics, the other reads out mean-pooled states of four contiguous temporal segments; a class-aware calibrator with 1,057 coefficients fuses the two scores. Caption spans are mapped to AudioSet ontology classes and pruned below a threshold. On 500 AudioSet-eval test clips with Qwen2-Audio-7B, REVE removes 92.9% of label-unsupported mentions at 0.756 faithful-mention recall (residual density 0.844 to 0.060), comparable to CED-Tiny (92.7%) and close to CED-Base (93.8%), while adding 3.38M parameters and 0.57 ms complete latency versus 10.32 ms for CED-Base. Ablations confirm both readouts and the calibrator help; controlled DESED mixtures reach 0.959 AUROC; Qwen2.5-Omni-7B and SALMONN-13B show 80.1% and 74.3% reduction. Main limitations: small test sets without variance estimates, unreported ontology-mapping coverage, and an ambiguous DESED training setup.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 18 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Frame-level encoder states already computed by the target LALM contain sufficient event-presence evidence to verify caption mentions without a second audio encoding pass.
- On 500 AudioSet-eval test clips with Qwen2-Audio-7B, REVE removes 92.9% of label-unsupported event mentions at 0.756 faithful-mention recall (residual density 0.844 to 0.060), comparable to CED-Tiny (92.7%) and close to CED-Base (93.8%).
- REVE's complete verification latency (0.57 ms) is about 1/18 of the CED-Base path (10.32 ms) with 3.38M added parameters versus 85.71M.
- The statistics and segment readouts provide complementary evidence, and class-aware calibration (1,057 coefficients) converts a common AUROC level into substantially lower residual density (0.082 to 0.060, Table 3).
- REVE generalizes to controlled DESED mixtures (0.959 AUROC vs CED-Base 0.901) and to other LALM architectures (80.1% and 74.3% reduction on Qwen2.5-Omni-7B and SALMONN-13B).
Strengths
Deployment-relevant efficiency, carefully costed
Table 2 reports a concrete timing protocol (CUDA events, FP32, NVIDIA L20, 10-s clip): REVE adds 3.38M parameters with 0.57 ms complete latency versus CED-Base's 85.71M / 10.32 ms (~1/18), and unlike CED-Tiny/CED-Base it needs no second waveform encoding pass.
Honest, recall-constrained evaluation protocol
Eqs. 12-16 define residual false-mention density together with faithful-mention recall; thresholds and the calibrator are selected only on a 500-clip dev split under an explicit recall >= 0.75 constraint applied uniformly to every scored method, pre-empting the delete-everything degeneracy the authors themselves flag.
Label-ambiguity control via synthetic mixtures
Component-level ablation
Cross-architecture evidence
Weaknesses
No statistical uncertainty on hair-thin margins
Ontology-mapping coverage unreported
Ambiguous DESED configuration weakens the generalization claim
Deletion editing unevaluated for fluency
Missing training details and unreleased code
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
REVE is a four-page, ICASSP-style method paper on event-level hallucination correction for large audio-language models (LALMs). The problem: captions generated by systems such as Qwen2-Audio sometimes name acoustic events absent from the input (thunder in a rain-only clip), and caption-level confidence cannot localize the offending text span. The core claim (Sec. 1-2): event-presence evidence is already contained in the frame-level encoder states the target model computes before its modality projector, so verification needs neither a second encoder nor an extra waveform pass. Stated contributions: (1) formulating post-hoc correction as ontology-aligned event verification with a two-scale reused-state verifier; (2) evaluation under a faithful-mention recall constraint with explicit cost accounting; (3) source-disjoint DESED and cross-architecture experiments. Headline numbers: 92.9% removal of label-unsupported mentions at 0.756 recall, residual density 0.844 to 0.060 (Table 1); 0.57 ms complete verification latency versus 10.32 ms for CED-Base and 3.38M added parameters versus 85.71M (Table 2); DESED AUROC 0.959 versus CED-Base 0.901 (Table 4); cross-model reductions of 80.1% (Qwen2.5-Omni-7B) and 74.3% (SALMONN-13B, Table 5).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is REVE in audio-language model research?
REVE (Reused Encoder States for Verifying Events) is a lightweight post-hoc method that checks whether the events mentioned in a large audio-language model's caption are actually present in the input audio. Instead of running a separate audio tagger such as CED on the waveform again, it reuses the frame-level encoder states the target model already computed and applies two small trained readouts whose fused, class-aware scores decide whether each event mention is kept or removed from the caption.
How much does REVE reduce hallucinations compared with CED?
On 500 AudioSet evaluation clips with Qwen2-Audio-7B, REVE removed 92.9% of event mentions unsupported by the reference labels while keeping 75.6% of faithful mentions. Under the same recall constraint, CED-Tiny achieved 92.7% and CED-Base 93.8%. However, REVE adds only 3.38M parameters and 0.57 ms of complete verification latency versus 10.32 ms for CED-Base - roughly 1/18 - because it never re-encodes the audio.
What are the limitations of REVE?
Evaluation uses small test sets (500 AudioSet clips, 500 DESED mixtures) without confidence intervals; the paper does not report what fraction of hallucinated mentions can actually be mapped to AudioSet ontology classes (unmapped spans are left unedited); the DESED training configuration is ambiguous; and deleting spans may harm caption fluency, which is not evaluated. The readouts also require ontology-labeled training audio (AudioSet-balanced-train).
Does REVE work on models other than Qwen2-Audio?
The paper reports cross-model results on Qwen2.5-Omni-7B (80.1% reduction of false mentions at the 0.75 recall target, 0.865 mention AUROC) and SALMONN-13B (74.3% reduction, 0.841 AUROC). Both are lower than Qwen2-Audio-7B's 0.930 AUROC and 92.9% reduction, so effectiveness varies noticeably with the target architecture.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-2 are available only as text fragments in the extraction; visual details of the efficiency landscape (Fig. 2a) and architecture diagram could not be verified.
- Training hyperparameters (optimizer, learning rate, epochs, readout width, ontology class count K) are absent from the extracted text; this may reflect extraction truncation rather than omission.
- No appendix was included in the extracted text, so any additional robustness analyses the authors may have cannot be assessed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.