GUARD: Natural Forgetting in Large Reasoning Models via Guided Answer-Reasoning Distillation
Zeyu Yan, Guanghao Zhou, Minghui Qiu, Ming Gao, Cen Chen
cs.AI
Quick verdict
Trajectory-level LRM unlearning via rewrite+guided distillation+NFRS metric; strong R-TOFU/STAR-1 gains, but single-seed results and Pareto overclaim.
Summary
This method paper tackles machine unlearning in large reasoning models (LRMs), where sensitive facts or unsafe rationales can leak through chain-of-thought traces even when the final answer is suppressed. The authors first diagnose three failure modes of suppression-based unlearning — hallucinated fabrication, structural break, and language collapse — supported by mechanism-level analyses of probability-mass redistribution, refusal-trajectory mismatch, and representation drift. They propose GUARD, a three-stage pipeline: an offline frontier LLM rewrites model-generated unsafe disclosures into safe-exit trajectories; Guided Trajectory Alignment (GTA) trains soft guidance tokens on a frozen model with block-normalized cross-entropy and a Top-K KL retain penalty; Answer-Reasoning Distillation (ARD) then internalizes the guided teacher into LoRA parameters, removing any runtime dependency. A new metric, NFRS, combines deterministic structural gating with LLM-judged naturalness and hallucination checks. On R-TOFU (1/5/10% splits) across DeepSeek-R1-Distill 8B/14B, GUARD attains average scores of 0.76–0.80 and NFRS 0.82–0.92; R2MU forgets more aggressively (AFE up to 0.96) but its NFRS collapses to 0.02–0.30. On STAR-1, GUARD reaches 0.96 average safety with NFRS 0.89–0.90 while matching pre-unlearning reasoning benchmarks. Main limitations: single-seed runs without variance estimates, evaluator–rewriter circularity (GPT-5.4-mini serves as rewriter, checkpoint scorer, and judge), and dependence on a frontier rewriter.
Review complete
Assessment at a glance
4
risk flags identified
● high 1
● medium 2
● low 1
4 strengths · 4 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Existing LRM unlearning objectives under-specify post-forgetting behavior, systematically producing hallucinated substitutes, structural break, or language collapse (Sec. 3, Figs. 1–2).
- GUARD's rewrite→GTA→ARD pipeline converts model-generated disclosures into natural non-disclosing trajectories and internalizes them so the deployed model needs no inference-time guidance tokens (Sec. 4).
- NFRS, a structural-gate plus LLM-judge 0–1 score, reliably measures replacement quality, with cross-judge NFRS 0.85–0.90 and 97–100% human agreement (App. C).
- On R-TOFU (1/5/10%, 8B/14B) GUARD attains the highest average score (0.76–0.80) and NFRS (0.82–0.92); on STAR-1 it reaches 0.96 average safety with NFRS 0.89–0.90 while matching pre-unlearning reasoning (Tab. 3: 0.47–0.48 avg).
Strengths
Mechanism-level diagnosis of baseline failures
Sec. 3 backs the three failure families (Fig. 1) with targeted diagnostics (Fig. 2): GA's protected mass leaks into alternative tokens, PO shows weak CoT–answer refusal coupling, R2MU's drift propagates into boundary/answer positions — objective-level evidence, not just benchmark deltas.
Deployable design validated by ablations
GTA→ARD removes runtime guidance tokens; Tab. 6 shows answer-only supervision collapses CFE to 0.14 and CoT-only drops AFE to 0.36, while full-trajectory ARD reaches avg 0.76 — directly supporting the coupled answer-reasoning objective.
Metric validation unusually thorough for an LLM-judge score
Broad experimental coverage with honest accounting
Weaknesses
Single-seed results with no variance reporting
Pareto dominance claim contradicted by own Table 1
Evaluator–rewriter circularity in the NFRS pipeline
CoT-level leakage only partially removed, with judge overlap in STAR-1
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper reframes machine unlearning for large reasoning models (LRMs) as a trajectory-level problem: private facts or unsafe rationales can surface in chain-of-thought (CoT) traces even when the final answer is suppressed. Three claims are advanced: (1) existing objectives — likelihood suppression (GA/GD/KL), preference optimization (PO), and representation redirection (R2MU) — under-specify the post-forgetting trajectory, yielding hallucinated fabrication, structural break, or language collapse (Sec. 3, Fig. 1); (2) GUARD, a three-stage pipeline of offline trajectory rewriting, Guided Trajectory Alignment (GTA) with soft guidance tokens, and Answer-Reasoning Distillation (ARD) into LoRA, learns a natural forgetting behavior deployable without runtime tokens; (3) the proposed NFRS metric measures replacement quality beyond leakage. Evaluation spans R-TOFU (1/5/10% forget rates) and a STAR-1-derived safety setting on DeepSeek-R1-Distill-LLaMA-8B and Qwen-14B.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is GUARD in LLM unlearning?
GUARD (Guided Answer-Reasoning Distillation) is a three-stage framework for making large reasoning models forget sensitive content: an offline LLM rewrites unsafe disclosures into safe CoT-and-refusal trajectories, soft guidance tokens align a frozen model to those trajectories, and the behavior is distilled into LoRA parameters so the deployed model needs no special tokens at inference.
What is the NFRS metric?
The Natural Forgetting Response Score rates response quality after unlearning on a 0–1 scale. Structurally broken outputs (missing </think> boundary, heavy repetition, gibberish) automatically score 0; surviving outputs get 1–5 scores from an LLM judge for reasoning naturalness and absence of hallucinated substitutes, then normalized by 5.
Does GUARD leak less than other unlearning methods?
Not uniformly. On R-TOFU, GUARD forgets less aggressively at the CoT level (CFE 0.60–0.70) than R2MU (0.68–0.94), but its outputs stay natural (NFRS 0.82–0.92 vs R2MU's 0.02–0.30) and its average score is highest (0.76–0.80). On STAR-1 safety prompts it reaches 0.96 average safety.
Is code available for GUARD?
The paper states code is at github.com/zeyu-Yan/GUARD and publishes hyperparameters, rewrite prompts, and judge prompts; however, the pipeline depends on a proprietary frontier LLM (GPT-5.4-mini) for rewriting and judging, and headline results are single-seed.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–5 are plots; only captions and axis labels were extracted, so sweep shapes (e.g., Fig. 4) are taken from captions, not data points.
- MU/AFE/CFE definitions are inherited from R-TOFU and not fully restated in this text; the exact composition of MU in Table 1 is inferred, not verified.
- Fig. 8 case highlights are color-coded (orange/yellow/red) and could not be visually verified from extracted text.
- Minor extraction artifacts exist (e.g., 'Guide' vs 'Guided' in the abstract); judgments use the best reading.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.