Mizar: A 159M-Parameter Audio-Language Model for Audio Understanding
Kaiyang Li, Shaobo Han, Yue Tian, Shihao Ji
cs.SD · cs.CL
Quick verdict
Claims sub-200M SOTA for a 159M ALM via data+3-stage training; well-ablated and honest, but headline margin rests on its own reevaluation of Mellow.
Summary
The paper introduces Mizar, a 159.3M-parameter audio-language model built from a frozen CED-Small audio encoder (21.4M), a 3.37M frequency-merging mapper, and SmolLM2-135M, targeting sub-200M ALMs for CPU-local inference. Training has three stages: audio-language alignment on 1,477,976 rows from ReasonAQA plus AudioMCQ (including 150K chain-of-thought examples); audio-dependent fine-tuning on 57,984 strongAC and AVQA examples; and post-training on 12,800 examples combining answer-position balancing with teacher-assisted four-quadrant sampling (teacher: fine-tuned Qwen2.5-Omni). Across five seeds Mizar reports 52.92% on MMAU (9,000-question test), 42.42% on MMAR, and 36.02% on ADQA-clean, exceeding a locally reevaluated Mellow (41.47/34.40/30.25) under an identical protocol; the authors note Mellow's published 52.11% on MMAU stems from an earlier benchmark revision. Ablations attribute most gains to data (Mellow-native retrains to 39.04 mean vs 35.37 released), with smaller contributions from the mapper and CoT examples. Diagnostics show larger accuracy drops for Mizar than Mellow when audio is silenced or replaced, indicating greater audio reliance, and CPU inference completes MMAU answers in 1.09s mean with 1.66 GiB peak memory. Limitations: Stage 1 ablations are single-seed without variance estimates; no significance tests accompany the 0.5-0.9-point post-training gains; evaluation is MCQ-only with a 20-second audio cap; and the margin over Mellow depends on the authors' own reevaluation.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A 159.3M-parameter ALM (frozen CED-Small + frequency-merging mapper + SmolLM2-135M) surpasses the previous best sub-200M model, Mellow, on MMAU, MMAR, and ADQA-clean under an identical reevaluation protocol.
- Across five random seeds, Mizar achieves 52.92% on MMAU-9k, 42.42% on MMAR, and 36.02% on ADQA-clean.
- Most of the improvement over released Mellow comes from the expanded data and three-stage recipe rather than the architecture (Mellow-native retrains to 39.04 mean vs 35.37 released; Mizar S1 is 40.01).
- Stage 3's answer-position balancing and teacher-assisted four-quadrant sampling raise the three-benchmark mean from 42.38 (Stage 2) to 43.78, with Q2 beating R2 by 0.51–0.88 points per benchmark.
- Mizar runs single-CPU local inference: 1.09s mean latency to a complete MMAU answer, 1.66 GiB peak memory, four threads, FP32, no quantization.
Strengths
Protocol-matched baseline reevaluation
Mellow is reevaluated locally under the same parser and official scoring service as Mizar (Table 1 footnote, Sec. 3.1), rather than comparing against numbers from different benchmark revisions — an unusual fairness measure in this literature.
Multi-seed headline reporting with shared initializers
Post-training results are mean ± sample SD across five seeds sharing one Stage 1/2 initializer (Table 3: 52.92±0.11, 42.42±0.24, 36.02±0.25), which quantifies the stability of the final recipe.
Factorial Stage 3 dissection
Honest attribution of gains to data over architecture
Audio-reliance and cost diagnostics
Weaknesses
SOTA margin depends entirely on the authors' own Mellow reevaluation
Single-seed ablations without variance
No statistical tests despite having seeds
MCQ-only evaluation with 20-second audio cap
Unquantified teacher dependency in Stage 3
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Mizar is a method paper targeting sub-200M-parameter audio-language models (ALMs) for CPU-local audio understanding. Its central claims: (1) a 159.3M architecture — frozen CED-Small encoder (21.4M) + 3.37M frequency-merging mapper + SmolLM2-135M — over a 20-second audio window (Sec. 2.1); (2) a three-stage recipe: 1,477,976-row audio-language alignment (ReasonAQA + AudioMCQ + 150K CoT), 57,984-example audio-dependent fine-tuning (strongAC + AVQA), and 12,800-example post-training combining answer-position balancing with teacher-assisted four-quadrant sampling (teacher: fine-tuned Qwen2.5-Omni, Sec. 2.2); (3) state of the art below 200M — 52.92% MMAU-9k, 42.42% MMAR, 36.02% ADQA-clean across five seeds (Table 3), beating a locally reevaluated Mellow (41.47/34.40/30.25) by 11.45/8.02/5.77 points; (4) single-CPU inference at 1.09s mean per MMAU answer (Sec. 3.6). Table 1 situates Mizar above Gemma 3n E2B (2B, 52.06) and GPT-4o mini Audio (51.03), but below Qwen2-Audio-Instruct (7B, 57.40) and Audio Flamingo 3 (8.2B, 72.42).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Mizar and how large is it?
Mizar is a 159.3M-parameter audio-language model that answers multiple-choice questions about sound recordings. It combines a frozen CED-Small audio encoder (21.4M parameters), a small frequency-merging mapper (3.37M), and the SmolLM2-135M language model, trained in three stages on public datasets (ReasonAQA, AudioMCQ, AVQA). It processes 20-second audio windows and reports 52.92% accuracy on the MMAU benchmark test set.
How does Mizar compare to Mellow?
Mellow is the previous best audio-language model under 200M parameters (167M). Under the authors' identical reevaluation protocol on MMAU-v05.15.25 (9,000 questions), Mizar scores 52.92% vs Mellow's 41.47%. Note that Mellow's originally published 52.11% came from an earlier MMAU revision with a different scoring protocol, so per the authors those published numbers are not directly comparable to Mizar's.
Can Mizar really run on a CPU?
Yes. On an AMD EPYC 7542 server CPU with four threads, FP32 precision and no quantization, Mizar produced a complete MMAU answer in 1.09 seconds on average (0.64 seconds to the first token, 95th percentile 1.37 seconds), with peak process memory of 1.66 GiB including framework initialization.
Where can I get the code and checkpoints?
The paper states that code and checkpoints are available at github.com/KaiyangLi1992/Mizar_159M. Training data (ReasonAQA, AudioMCQ, AVQA) and evaluation benchmarks (MMAU, MMAR, ADQA-clean) are public, and MMAU test predictions are scored by the official evaluation service.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Review is based on extracted text; Figure 1 is available only as caption text, so the architecture diagram itself was not visually verified.
- Some cited works (refs. 9, 21, dated 2026) and the MMAU leaderboard reference could not be independently checked from the text.
- Hyperparameters are deferred to the open-source implementation ('documented in our open-source implementation'), so training details are not fully verifiable from the paper.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.