Not All Speech Is Intent: Adaptive Self-Correcting Inference Layer for Post-ASR False Wake-Up
Preeti Saraswat, Divya Neelagiri, Anil Yadav
cs.CL · cs.AI
Quick verdict
Samsung ASCIL: post-ASR feedback loop cuts false wake-ups (54.27% on failure-only subset, 24.39% issue-tagged); feedback heuristics unvalidated.
Summary
The paper introduces ASCIL, a post-ASR self-correcting inference layer that reduces false wake-up activations in voice assistants. After a wake-up is handled, the system infers noisy behavioral feedback from next-turn user reactions (cancellations, repetitions, hesitation, silence, disengagement), extracts generalized acoustic/linguistic/contextual error patterns, stores them on-device, and injects them as LLM context at future inferences — without retraining the Qwen3-14B-Instruct base classifier (enriched with Voxtral Mini 3B annotations). On a proprietary dataset of 3,667 human-annotated interactions (2,358 intentional, 1,309 unintentional), a session-disjoint held-out set of 180 baseline failures yields a 54.27% relative error reduction at threshold 0.90 (CAR 70.32%, UICR 39.88%); the builder-contaminated issue-tagged slice improves up to 24.39% at 0.90; clean audio worsens 26.25% relatively at 0.85. Median added latency is below 60 ms. Main limitations: the feedback heuristics' precision/recall are never validated; the 54.27% headline is conditional recovery on a failure-only subset, not full-dataset improvement; the split is session-disjoint, not user-disjoint, so personalization rests on indirect evidence; no per-condition results for the 14 tags; proprietary data and no code preclude independent verification.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- ASCIL is, to the authors' knowledge, the first system that converts automatically inferred post-interaction user feedback into persistent, generalized, on-device corrective patterns applied at inference without retraining base ASR or NLU models (§2, §3).
- On a session-disjoint held-out subset of 180 baseline failures, ASCIL reduces Perror from 100% (by construction) to 45.73% at threshold 0.90 — a 54.27% relative error reduction, with CAR 70.32% and UICR 39.88% (§5.1).
- On the issue-tagged evaluation slice (2,258 intentional + 1,309 unintentional), ASCIL achieves up to 24.39% relative error reduction at threshold 0.90 while simultaneously improving CAR (89.15→93.49%) and UICR (25.44→40.72%) (§5.2, Table 2).
- The correction layer runs in parallel with NLU execution, adds a median latency below 60 ms on the target benchmark hardware, and retains no raw audio or verbatim transcripts on-device (§3.3–3.4).
Strengths
Unusually honest result qualification
§5.1 explicitly states the 54.27% figure is 'conditional recovery from those failures rather than an absolute error reduction over the full dataset'; §5.2 concedes the issue-tagged slice 'includes builder-derived cases and is therefore a system-level diagnostic rather than a contamination-free held-out test'; §5.3 openly reports the 0.85-threshold regression (−26.25% on clean audio); the Conclusion frames the work as 'a preliminary systems study'.
Clean internal ablation design
§4.2 builds the baseline as a prompt-only Qwen3-14B-Instruct classifier with 'same multimodal features and LLM classifier as ASCIL, but without pattern storage', so the reported deltas isolate the pattern-memory layer rather than confounding backbone or feature differences.
Rigorous label provenance discipline
Transparent multi-label slice accounting
Deployment-realistic constraints
Weaknesses
Core feedback heuristics are never validated
Ambiguity about what actually seeded pattern induction
Headline metric has a constructed denominator
No statistical uncertainty quantification
Missing external comparisons and per-condition results
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This Samsung Research America systems paper (arXiv:2609.12469, cs.CL/cs.AI) targets false wake-up activations: utterances phonetically near a wake word that ASR transcribes cleanly and the assistant wrongly executes. The contribution is ASCIL, a post-ASR correction layer that (i) infers noisy behavioral feedback from next-turn user reactions — cancellations and repetitions (explicit), hesitation, disengagement, silence (implicit); (ii) converts these into generalized on-device error patterns keyed by false-positive/false-negative type; and (iii) injects those patterns as natural-language context into a prompt-only Qwen3-14B-Instruct classifier (enriched with Voxtral Mini 3B annotations) at future inferences, without modifying base ASR/NLU parameters (§3, Figure 1). The authors position this as the first closed feedback loop after wake-up handling, complementary to same-turn device-directedness classifiers (§2; closest prior art: Mallidi et al. 2018; Kanungo et al. 2026 patent). Headline claims: 54.27% relative error reduction on a 180-case session-disjoint held-out subset of baseline failures at threshold 0.90; up to 24.39% on the issue-tagged slice; median added latency below 60 ms.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ASCIL and what problem does it solve?
ASCIL (Adaptive Self-Correcting Inference Layer) is a post-ASR correction layer for voice assistants that targets false wake-ups: speech phonetically similar to the wake word that gets transcribed cleanly and wrongly executed. After a wake-up is handled, it learns from the user's next-turn behavior (canceling, repeating, staying silent), stores generalized error patterns on-device, and uses them at future inferences to suppress likely misactivations — without retraining the underlying ASR, NLU, or classifier models.
How much does ASCIL reduce false wake-up errors?
The paper reports a 54.27% relative error reduction on a held-out set of 180 cases that the baseline had misclassified on all of them (at confidence threshold 0.90), and up to a 24.39% relative error reduction on a 3,567-case issue-tagged evaluation slice. Gains are threshold-dependent: on clean audio the method worsens errors at threshold 0.85 but improves them at 0.90 and 0.95. All figures come from a proprietary Samsung dataset of 3,667 annotated interactions.
Is the ASCIL dataset or code publicly available?
No. The evaluation uses a proprietary dataset of 3,667 commercial voice-assistant interactions with human-annotated intentional/unintentional labels, and no code, prompt templates, or pattern schemas are released, so independent replication is not currently possible.
Does ASCIL work for new users (cold start)?
The authors list cold-start as an open challenge: the layer depends on accumulated per-user interaction history, so new users get no benefit until patterns build up. Additionally, the evaluation split is session-disjoint rather than user-disjoint, so generalization to users never seen during pattern building is untested.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.