The Router Within: Eliciting Native Skill Routing from a Frozen LLM
Ruishuo Chen, Xun Wang, Yu Chen, Zhuoran Li, Longbo Huang
cs.LG · cs.AI · cs.CL
Quick verdict
Gavel reads skill routing out of a frozen LLM's mid-layer states with two 7.9M-parameter linear maps and beats retriever pipelines by up to 13.4 points, but its headline metric is an LLM judge with only 86% human agreement and its two smallest benchmarks have 75 and 45 items.
Summary
Gavel (Glance And Verdict from a frozen LLM) addresses skill routing for LLM agents. Progressive disclosure — preloading every skill's name and description into the system prompt, as Claude Code and Codex do — crowds the context and decays with library size, while retrieve-and-rerank pipelines keep the context clean but hand selection to external models that never see the rollout and do not improve as the agent does. Gavel instead elicits the routing signal from the agent's own forward passes. A glance grafts a trained query map Wq and key map Ws, 768 x 5120 and 7.9M parameters in total, onto one mid layer l* = 44 of Qwen3-32B's 64 blocks, selected by the floor of the matrix-entropy compression valley; at installation each skill costs one forward pass that stores a bank of unit keys, thinned to an epsilon-cover at radius 0.83 (about 8.5x smaller, at most 1.6 points cost, with a proved distortion bound). Each task token votes by max-similarity over every bank, contributing only to its top-k skills, which ranks the whole library. A verdict then resumes the installation pass with the task appended and reads two native signals: the mean log-likelihood L of the task given the skill, and the yes/no log-odds V at a fixed question. The three are fused as a product of experts, S = g + alpha*L + gamma*V with (alpha, gamma) = (1.0, 0.025), over the roughly nine candidates scoring within Delta = 0.133 of the glance's best. Trained once on 51,104 SkillRet queries over 9,084 skills, Gavel transfers zero-shot. Adjudicated Hit@1: SkillRet test .9460 (.8459 raw), SRA-Bench .8734 (.8223), Eval-Core easy .9467 and hard .9333 (.6933 raw), beating the strongest pipeline by 3.8, 13.4 and 1.3 to 2.7 points. On SkillTraj, a new 372-trajectory benchmark of mid-rollout routing, it leads by 8.6 to 21.9 points. Integrated into mini-swe-agent, Qwen3-32B triggers the correct skill on .909 of Skill-Use tasks versus .011 under progressive disclosure. Main limitations: the primary metric is an LLM judge whose agreement with the authors is 86% overall and much lower on the classes that decide credit; Eval-Core has 75 queries and the wrong-skill-recovery scenario 45; SkillTraj is synthetic; and the cost model prices a self-hosted 32B model at frontier API rates.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 3
● low 2
6 strengths · 6 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The routing signal for skill selection is already present in the frozen agent LLM's forward passes, and two linear maps reading one mid layer suffice to extract it, with no skill text in the context and no model beside the backbone.
- A token-level glance that lets each task token vote only for its top-k skills, combined with epsilon-cover compression of skill key banks, ranks libraries of tens of thousands of skills at installation-time cost of one forward pass per skill.
- Generative and discriminative read-outs from a resumed forward pass — task likelihood L and yes/no log-odds V — are commensurate with the glance's contrastive score and can be fused as a product of experts.
- Trained once on SkillRet, Gavel transfers zero-shot and beats progressive disclosure and retrieve-and-rerank pipelines adding 1.2B to 16B parameters by 3.8 points on SkillRet, 13.4 on SRA-Bench and 1.3 to 2.7 on Eval-Core, and by 8.6 to 21.9 across SkillTraj's four mid-rollout scenarios.
- Because the read-out is drawn from the agent itself, routing accuracy climbs with backbone capability: Gavel on a 0.6B backbone already beats a 32B backbone picking from a metadata menu.
Strengths
Genuinely new interface: routing from the agent's own forward pass rather than from added models
Sections 3.2-3.4 make the case concretely. Only Wq and Ws (768 x 5120, 7.9M parameters) are trained; gradients stop at the hidden states; a new skill costs one forward pass and no training; no skill text enters the context until one is chosen. That satisfies all three requirements the paper sets out in Section 3.1, and it is a real alternative to both deployed designs in Figure 1.
Compression comes with proofs, not just an engineering trick
Proposition 1 bounds score distortion by epsilon for every unit query, Corollary 1 extends it through the top-k vote with an explicit boundary term, and Proposition 2 pins the retained bank size between covering and packing numbers. The paper also checks the empirical cost: epsilon = 0.83 shrinks banks about 8.5x for at most 1.6 points.
Negative results are reported and they motivate the design
Thorough ablations isolate each component
Cross-family replication and cost accounting
Honest disclosure of metric weaknesses
Weaknesses
The primary metric is an LLM judge with modest validated agreement
Two of the four evaluations are too small to support their margins
SkillTraj is synthetic and generated by the same family of models being evaluated
The deployment comparison is confounded by harness
Cost analysis prices a self-hosted open model at proprietary API rates
No code release and no report of bank storage at scale
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Gavel targets a problem that has become concrete only recently: an agent with thousands of installed SKILL.md files must decide which one to load, and both deployed solutions have structural flaws. Progressive disclosure — Claude Code and Codex preload every skill's name and description — crowds the context in proportion to the library; retrieval pipelines relieve the context but hand selection to external models that never see the rollout and do not improve with the agent. Section 3.1 converts this into three requirements: task-only context, no standalone model, installation-time indexing with no training per skill.
The paper's claim is that a frozen backbone already carries the routing signal and two linear maps suffice to read it. This is a well-scoped claim and the paper delivers on all three requirements. What it does not claim is worth noting too: there is no measurement of end-to-end task success attributable to routing — the deployment result is a trigger rate, not a task completion rate.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 6 strengths & 6 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Gavel and what problem does it solve?
Gavel is a skill router for LLM agents. When an agent has thousands of installed skills, current practice either preloads every skill's name and description into the prompt (which crowds the context and decays with library size) or uses a separate embedding model and reranker (which adds external models that never see the agent's live conversation). Gavel instead reads the routing decision out of the agent model's own internal states, using just two trained linear projections of 7.9 million parameters, so nothing is added to the context and no second model is needed.
How does Gavel route without putting skill text in the context?
In two steps. A glance projects the task's hidden states at one middle layer through a learned query map, and each skill's hidden states through a learned key map; each task token then votes by maximum similarity for the few skills it matches best, ranking the entire library. A verdict then takes the top candidates and re-reads the frozen model's own predictions with the skill prepended: how likely the model finds the task given that skill, and its own yes/no answer to whether the skill provides what the task needs. The three scores are combined multiplicatively.
How much better is Gavel than existing skill-selection methods?
On three public benchmarks it scores highest by adjudicated Hit@1: .9460 on the SkillRet test set, .8734 on SRA-Bench and .9333 to .9467 on Eval-Core, ahead of the strongest retrieve-and-rerank pipeline by 3.8, 13.4 and 1.3 to 2.7 points. On SkillTraj, the authors' new benchmark of 372 simulated agent trajectories where a skill becomes necessary mid-conversation, it leads by 8.6 to 21.9 points. Integrated into a bash-agent harness, a 32B model loads the correct skill on 90.9% of Skill-Use tasks versus 1.1% under the standard prompt-based approach.
What are the main caveats when reading this paper?
Three. The headline metric is adjudicated by another LLM, and the authors' own validation finds only 86% agreement with human labels on 200 pairs, lower in exactly the cases that decide whether a pick is credited. Two evaluations are small: Eval-Core has 75 queries and one SkillTraj scenario has 45 trajectories, with no confidence intervals reported. And the deployment comparison puts a 32B model in one harness against frontier models in Codex, a difference the paper shows can move the trigger rate by up to 38 points on its own.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 3-6 and 8 are bar charts whose numeric values are not in the text; all scores quoted here come from Tables 1-9 in the appendices, which pair each adjudicated score with its raw Hit@1.
- The paper is an arXiv preprint citing many 2026 preprints (SkillRet, SRA-Bench, SkillRouter, Skill-Use, SkillsBench) that I could not consult; characterisations of those benchmarks follow the authors' descriptions.
- Appendix J's cost model is analytic and uses GPT-5.6 Sol API prices for a locally served backbone; it is reported here as the authors state it, not as a measured cost.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.