SEAL: Reinforcing Global Safety in Mixture-of-Experts through Shared Expert ALignment
Qingyu Meng, Yiwei Zha, Jiahuan Pei, Koen Hindriks, Herbert Bos, Min Chen
cs.LG · cs.AI · cs.CR
Quick verdict
SEAL makes shared experts a router-independent MoE safety anchor (DPO-LoRA), cutting ASR up to 53pts at ≤1.4% capability cost; promising, evidence gaps remain.
Summary
The paper targets safety vulnerabilities of hybrid Mixture-of-Experts LLMs, where adversarial prompts, malicious fine-tuning, and neuron pruning exploit the fact that safety depends on which routed experts the router selects. The authors observe that shared experts execute unconditionally on every token and carry safety-critical neurons at 0.63-0.89 per-parameter density relative to routed experts across four open-weight models (Qwen1.5-MoE-A2.7B, DeepSeekMoE-16B, GLM-4.7-Flash, Qwen3.5-35B-A3B). They propose SEAL, training DPO-based LoRA adapters (rank 64) restricted to shared-expert projections (~0.06-0.25% of parameters), and SEAL++, which adds an orthogonal constraint protecting profiled safety subspaces (z-score > 2.0 activation differentials). Across harmful prompts, jailbreaks, and malicious fine-tuning, with and without neuron pruning at three scopes, SEAL/SEAL++ cut attack success rates - for example DeepSeek harmful ASR from 61.3% to 8.5% and Qwen1.5 MFT+pruning both-scope ASR from 94.1% to 44.7% - at a five-benchmark capability cost of at most 1.4 points, with zero inference overhead. Limitations: single-run results without significance testing; on Qwen3.5, SEAL worsens routed-scope pruning ASR (5.9% to 13.2%); post-MFT ASR remains high (up to 87.5%); adaptive attackers exploiting the published protection mask are not evaluated; and the abstract's composed-defense figure (ASR 4.8%) conflicts with the Discussion deferring composition validation to future work.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Shared experts execute on every token and carry safety-critical neurons at 0.63-0.89 per-parameter density relative to routed experts across four hybrid-MoE models, making them a router-independent safety anchor.
- SEAL, DPO-based LoRA restricted to shared-expert projections (0.06-0.25% of parameters), cuts ASR across harmful prompts, jailbreaks, and malicious fine-tuning at a five-benchmark capability cost of at most 1.4 points.
- SEAL++'s orthogonal constraint preserves pre-existing safety subspaces and raises pruning attack cost, with a claimed cost-amplification factor gamma >= 4.3 on DeepSeek (both-scope).
- SEAL composes with router-level defenses, reducing ASR to 4.8% - more than five times lower than router-level defense alone.
Strengths
Novel, architecturally grounded defense surface
First work to target shared experts for safety: Eq. (1) guarantees unconditional execution, and Fig. 3/Table 1 quantify the basis - shared-to-routed safety-neuron density ratios of 0.63-0.89 and a trainable budget of 0.06-0.25% across four architectures.
Strong headline efficacy at near-zero cost
DeepSeek harmful ASR 61.3% to 8.5% (Table 2), Qwen1.5 MFT+pruning both-scope 94.1% to 44.7% under SEAL++ (Table 5), pruning cost amplification gamma >= 4.3 on DeepSeek (App. D), five-benchmark capability cost <= 1.4 points, and zero inference overhead after merging the LoRA.
Threat-model-aware evaluation design
Data hygiene and mechanism validation
Weaknesses
No uncertainty quantification for headline results
Internal contradiction on defense composition
Copyediting and numbering defects
Residual weakness under the strongest attacks
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
SEAL (to appear at ACM CCS 2026; arXiv 2609.02293) is a defense paper for hybrid Mixture-of-Experts LLMs. The threat model (section 3) defines three adversaries with escalating privileges: Ainput (prompt injection and jailbreaks), Atrain (malicious fine-tuning on 300 harmful examples), and Aweight (activation-differential neuron pruning). The central claim is that shared experts - always active by Eq. (1) - form a router-independent safety anchor: training DPO-LoRA adapters on them alone (SEAL), optionally with an orthogonal constraint protecting profiled safety subspaces (SEAL++), reduces ASR across six attack scenarios at at most 1.4 points of five-benchmark capability cost, and composes with router-level defenses (claimed ASR 4.8%).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is SEAL in MoE safety?
SEAL (Shared Expert ALignment) is a training-time defense for hybrid Mixture-of-Experts LLMs. It trains DPO-based LoRA adapters only on the shared experts - components that run on every token regardless of routing - so safety behavior no longer depends on which routed experts the router happens to select. It modifies about 0.06-0.25% of model parameters and adds no inference-time cost once the adapters are merged.
How much does SEAL improve safety?
On four open-weight MoE models (14B-35B), SEAL and SEAL++ lowered attack success rates across harmful prompts, jailbreaks, and malicious fine-tuning, with and without neuron pruning. Examples: DeepSeekMoE-16B harmful-prompt ASR fell from 61.3% to 8.5%, and Qwen1.5 MFT+pruning both-scope ASR fell from 94.1% to 44.7%, with at most 1.4 points of capability loss averaged over five benchmarks and zero inference overhead.
What are SEAL's limitations?
Main-table results are single runs without significance tests; on Qwen3.5 the defense actually increases routed-scope pruning ASR (5.9% to 13.2%); post-malicious-fine-tuning ASR remains high (up to 87.5%); adaptive attackers who exploit the published protection mask were not evaluated; and the abstract's composed-defense figure (ASR 4.8%) conflicts with the discussion deferring composition experiments to future work.
Is SEAL's code available?
The paper states that code, training scripts for all four architectures, and profiling/training datasets are released via an anonymous repository (anonymous.4open.science/r/SEAL-38AC), using only open-weight models and public datasets. Reproduction is GPU-heavy: the paper reports using 2x H100 for the two smaller models and 4x H100 for GLM and Qwen3.5.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt with marked [...sampling gap...] sections; a composition experiment, router-baseline comparison table, or full classifier-agreement analysis could exist in skipped sections - the abstract-vs-discussion tension is verified only as a wording inconsistency.
- Figures are available only as text-rendered axis labels and annotations in the excerpt; numeric values were cited only where printed in the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.