Investigating catastrophic forgetting in sound event classification
Riccardo Casciotti, Annamaria Mesaros
eess.AS · cs.AI · cs.SD
Quick verdict
Claims frozen features plus a dynamic head best curb forgetting in incremental sound classification; convincing, but baselines and variance reporting are thin.
Summary
This empirical study compares strategies against catastrophic forgetting in class-incremental, multi-label sound event classification on FSD50K and AudioSet (the 50 largest classes of each). Using a PANNs-style six-block CNN, the authors split 50 classes into five disjoint tasks (30, then 4×5 classes) under 'ordered' and 'not ordered' configurations, averaging five random class orderings. Compared methods — all sharing a dynamically expanding classifier head — are knowledge distillation (KLD), pruning-rank-based kernel freezing at 12.5–100%, a Hebbian-style learning-modulation method, and full freezing of the feature extractor with head-only training (called 'fine-tuning'). Evaluation uses cumulative mAP plus BWT, FM, and IM against a joint reference model. The frozen extractor dominates: on AudioSet it reaches BWT −0.5/−0.2, FM 0.1–0.2, IM about 5.2–5.5, versus BWT −11.5 to −21.8 for rank and modulation methods; FSD50K shows the same pattern and KLD ranks second with FM 1.1–4.0. The authors conclude that forgetting concentrates in the classifier head and that feature-level modulation is unnecessary when tasks share a distribution. Limitations: absolute mAP values appear only in figures, no naive full-network or classic regularization baselines (EWC, SI, replay) are run, the five-run averages carry no variance or significance reporting, and the 50-class, single-architecture scope limits generality.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Catastrophic forgetting in class-incremental sound event classification concentrates in the deeper layers, especially the classifier head, not the feature extractor.
- Full freezing of the feature extractor combined with a dynamically expanding classifier head yields near-zero forgetting (BWT −0.5/−0.2, FM 0.1–0.2 on AudioSet) and the best stability-plasticity balance among the tested methods.
- Pruning-rank-based kernel freezing (12.5–100%) and Hebbian-style learning-modulation introduce more instability than they prevent, with BWT between −11.5 and −21.8 across datasets.
- Knowledge distillation alone provides intermediate protection (FM 1.1–4.0) but converges worse than head-only training.
Strengths
Practical headline result with large margins
Full feature-extractor freezing plus the dynamic head attains BWT −0.5/−0.2 and FM 0.1–0.2 on AudioSet (Table I) versus BWT −13.3 to −21.8 for rank-based freezing and −11.5 to −19.9 for learning-modulation, echoing the strong-baselines critique (Hsu et al. [5]).
Controlled multi-condition protocol
Two datasets (FSD50K, AudioSet top-50), two orderings ('ordered' by class size, 'not ordered'), five random class orderings averaged, and a fixed 30+4×5 disjoint-task split with no access to old data (Section IV).
Standard, correctly defined CL metrics
Code release
Weaknesses
Key baselines missing
Layer-level claim is indirect
No dispersion across the five runs
Plasticity cost under-discussed
Narrow scope and confusing naming
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This arXiv preprint (2609.11447) from Casciotti and Mesaros (Tampere University) is an experimental comparison of continual-learning strategies for class-incremental, multi-label sound event classification. Three central claims: (1) catastrophic forgetting concentrates in the deeper layers, 'in particular in the classifier head'; (2) fully freezing the feature extractor after the first task and training only a dynamically expanding head — confusingly named 'fine-tuning' — is the most efficient remedy, with near-zero forgetting and the best stability–plasticity balance; (3) regularization-style interventions (knowledge distillation, pruning-rank kernel freezing at 12.5–100%, Hebbian-style learning-modulation) add instability rather than protection. The scope is explicitly in-domain: all tasks come from the same dataset's top-50 classes, and the authors confine their conclusion to tasks following the same probability distribution.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the best method to prevent catastrophic forgetting in sound event classification according to this paper?
The authors find that fully freezing the convolutional feature extractor after the first task and training only a dynamically expanding classifier head gives the least forgetting — backward transfer around −0.5 and forgetting measure about 0.1–0.2 on AudioSet — beating knowledge distillation and kernel-level freezing or modulation methods on both FSD50K and AudioSet.
Which datasets and evaluation metrics does the paper use?
It uses the 50 largest classes of FSD50K and AudioSet, split into five disjoint tasks (30 classes first, then four tasks of 5 classes each). Performance is measured with cumulative mAP plus continual-learning metrics: Backward Transfer (BWT), Forgetting Measure (FM), and Intransigence Measure (IM) computed against a jointly trained reference model, averaged over five random class orderings.
Does the paper show whether forgetting happens in the feature extractor or the classifier?
The authors conclude that forgetting mainly occurs in deeper layers, especially the classifier head, based on the strong results of freezing the whole feature extractor; however, this is inferred indirectly from the freezing experiments rather than measured layer by layer.
Is code available for this paper?
Yes. The paper links a public GitHub repository (RiccardoCasciotti/Class-Incremental-SEC) for reproducing the experiments, and both datasets used (FSD50K and AudioSet) are publicly documented.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 2 (cumulative mAP curves) is referenced but its data points are not present in the extracted text; absolute mAP values could not be verified, and quantitative judgments rest on the CL-metric tables.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.