Safin-1: Safety from Within through Memory-Native State Evolution
Ming Zhang, Kaisen Yang, Shu Yu, Ermo Hua, Zhekai Chen, Cheng Jin, Jingnan Zheng, Yi Zhang, et al.
cs.LG
Quick verdict
Safin-1 adds routable recurrent-state memory (MARCH) plus a detachable Safety State; broad, well-ablated gains, but single-run and statistically untested.
Summary
This paper introduces Safin-1, a family of foundation models built on MARCH (Memory-Anchor Routing across Context History), an architecture in which a recurrent model (Gated DeltaNet family) periodically checkpoints its evolving state into addressable 'anchors' that later tokens retrieve through content-conditioned routing with a learned null option. The same routing bank hosts learnable persistent capability states, instantiated as a detachable Safety State trained while the backbone stays frozen. Controlled 0.8B pretraining (50B tokens) shows MARCH improving GDN, Kimi Delta Attention, and GDN2 on language modeling, LongBench (11.9→14.9 average), in-context retrieval, and RULER NIAH (31.58→46.43 six-task average for GDN). Scaling from Qwen3.5-4B and 35B-A3B through matched continual pretraining and SFT raises a ten-benchmark macro-average (66.79→69.20; 76.25→78.35) and AIME 2025 Avg@64 (55.57→63.59 and 71.88→80.10). The Safety State cuts average jailbreak attack success rate by 42.3% (4B) and 52.3% (35B-A3B) relative, with less over-refusal than a training-matched rank-8 LoRA. Main limitations: single-run results without significance testing, a small English-only safety corpus, no adaptive or state-tampering attacks, and task-dependent capability retention at 35B.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Checkpointing recurrent states into content-routed anchors (MARCH) improves language modeling, long-context understanding, and retrieval across GDN, KDA, and GDN2 at 0.8B, with gains persisting after scaling to 4B/35B-A3B via matched continual pretraining and SFT.
- A persistent Safety State learned with a frozen backbone reduces average jailbreak attack success rate by 42.3% (4B) and 52.3% (35B-A3B) relative, with less over-refusal than a training-matched rank-8 LoRA control.
- Top-4 sparse state routing makes historical retrieval practical at 128K tokens, more than doubling throughput versus dense routing.
- Safety can be represented as a detachable, router-invoked model state ('Safety from Within'), though its supervision remains external and its integrity must be protected against removal or tampering.
Strengths
Controlled multi-backbone validation
Section 3 pretrains matched 0.8B models on 50B tokens and applies MARCH to GDN, KDA, and GDN2 (Table 4): NIAH six-task average improves 31.58→46.43 for GDN, with gains in 11 of 12 backbone–length settings, isolating the architecture from scale effects.
Unusually deep ablations
Tables 5–7 cover RoPE removal (which smooths 32K/64K extrapolation: GDN+MARCH 64K NIAH 4.13→30.25), anchor interval including inference-time bank re-configuration and Fenwick-tree transfer (Table 6b), routing dimension, Top-4 sparsity, and null-option necessity (removing it degrades all four aggregates in Table 7).
Training-matched control for the safety claim
Candid limitations section
Serious systems work
Weaknesses
No statistical uncertainty anywhere
Safety base rates are tiny and judges unreported
Substantive 35B capability cost of the Safety State
Narrow control set for the central claim
Reproducibility friction
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Safin-1: Safety from Within through Memory-Native State Evolution — Preliminary Review
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is MARCH in Safin-1?
MARCH (Memory-Anchor Routing across Context History) is an architecture add-on for recurrent LLMs: every 512 tokens the model's evolving recurrent state is saved as an addressable 'anchor' with a compact routing key; later tokens score these anchors (plus a learned 'null' option) against a routing query and fuse the retrieved historical states back into the current computation without changing the recurrence itself.
How much does Safin-1's Safety State improve safety?
On five jailbreak benchmarks, the unweighted average attack success rate fell from 6.65% to 3.84% at 4B (a 42.3% relative reduction) and from 5.89% to 2.81% at 35B-A3B (52.3% relative), while over-refusal on XSTest rose by only 0.4 and 2.0 points — a smaller over-refusal increase than a training-matched rank-8 LoRA control.
Does the Safety State require retraining the whole model?
No. The language-model backbone stays frozen; only the persistent Safety State inserted into the routing bank is trained — on about 5,745 examples (1,000 harmful plus 915 benign targets, each with three context variants), one epoch, 37 optimizer steps — and it can be attached or removed without touching the backbone weights.
What are the main limitations of the paper?
Results are single-run with no significance tests; the safety corpus is small and English-only; adaptive, multilingual, and state-tampering attacks were not evaluated; and at 35B the Safety State costs capability on some benchmarks (e.g., MMLU-Pro drops 3.4 points, AIME 2025 5.9 points).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–4 are graphics: only axis labels, legends, and captions were extracted, so exact per-point NIAH and throughput values come from tables and running text, not figure interiors.
- GitHub and Hugging Face release badges are present in the page header, but their URLs were not extractable; codeAvailable=true is inferred from the badges.
- The abstract in the supplied metadata is cut mid-word, but the complete abstract is present in the extracted body; the text runs through Appendix A.3 (Table 13) and appears complete.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.