TRIAGE: Three-level Routing and Intelligent Agent Guidance for Efficient Execution
cs.LG
Quick verdict
Claims 62.3% token savings via trajectory reuse routing for ReAct agents; promising, but synthetic data and inconsistent ablation numbers weaken the evidence.
Summary
The paper proposes TRIAGE, a three-level routing framework that reduces token consumption of ReAct-style LLM agents by reusing historical execution trajectories. Core to the design is TaaS (Trajectory-as-a-Skill): successful trajectories are stored, embedded with all-MiniLM-L6-v2, and retrieved by cosine similarity. Queries route to Level 1 (exact match, direct reuse), Level 2 (deterministic parameter substitution against automatically extracted parameterized Skills), or Level 3 (full ReAct), with thresholds 0.98/0.90. On a self-built set of 1,007 security-monitoring queries (15 parameterized SQL templates plus recurring daily reports) using deepseek-v3.2, TRIAGE reports 62.3% token savings (199,782→75,238), 56.0% of queries at Level 2 and 5.5% at Level 1, and a 100% execution success rate. An online-learning run shows the Level-2 hit rate rising from 0% to 57% within the first 100 queries. Cross-domain testing on 345 ToolBench queries across 15 domains reports 76.3% token reduction, though there Level 2 uses LLM-based reference generation rather than zero-token Skill substitution. An ablation finds an LLM-based router costs more than it saves (net −37.3%). Main limitations: the headline gains follow directly from the authors' synthetic query mixture (admittedly 'not statistically rigorous'), no answer-correctness evaluation beyond an undefined success rate, inconsistent internal numbers (Table 4's 55.0% ablation savings; baseline 199,782 vs 198,508; L1 peak 87% vs 67%; conclusion's 97.6%), and no comparison with existing semantic-caching baselines such as GPTCache.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 4
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Three-level routing over stored trajectories (direct reuse / skill substitution / full ReAct) cuts total token consumption by 62.3% (199,782→75,238) on 1,007 security-monitoring queries with deepseek-v3.2.
- TaaS — abstracting historical trajectories into automatically extracted parameterized Skills — serves 56.0% of queries at zero token cost via deterministic parameter substitution.
- Threshold-based routing (cosine ≥0.98/0.90 with all-MiniLM-L6-v2) beats LLM-based routing, which incurs 225,628 routing tokens and a net −37.3% loss; the right choice depends on the routing-overhead-to-execution-cost ratio.
- The framework generalizes across domains, achieving 76.3% average token reduction on 345 ToolBench queries across 15 domains.
- Online learning yields cold-start-to-mature evolution, with the L2 hit rate rising from 0% to 57% within the first 100 queries and average cost dropping from 198 to 74.7 tokens.
Strengths
Informative routing-strategy ablation
Ablation 1 (§4.4.1, Table 3) quantifies that an LLM router's 225,628-token overhead turns a 62.3% saving into a net −37.3% loss, and the paper correctly generalizes this into a routing-overhead-vs-execution-cost tradeoff rather than a universal claim.
Honest scoping of cross-domain results
§4.5 explicitly discloses that ToolBench Level 2 queries used LLM-based reference generation (partial cost) rather than the 0-token Skill substitution of the main SQL experiment, tying the lower L2 rate (11.6% vs 56.0%) to this gap.
Transparent system specification
Coherent self-improvement account
Useful engineering framing
Weaknesses
Headline number is a function of a self-built distribution
Component-ablation numbers do not add up
Internal numeric inconsistencies
No correctness evaluation and no caching baselines
Correctness and staleness risks unmeasured
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
TRIAGE (arXiv 2609.01428, cs.LG) is a single-author method paper targeting token waste in ReAct-style LLM agents: identical or structurally similar queries re-run full reasoning loops. The framework stores executed trajectories, embeds them with all-MiniLM-L6-v2 (384-d), and routes by cosine similarity into Level 1 (exact match, direct reuse), Level 2 (deterministic parameter substitution against automatically distilled 'Skills' — the TaaS, Trajectory-as-a-Skill, contribution), or Level 3 (full ReAct), at thresholds ≥0.98 and ≥0.90. Central claims: 62.3% token savings on 1,007 security-monitoring queries (199,782→75,238) with deepseek-v3.2; 56.0% L2 / 5.5% L1 routing (Table 2); a ToolBench cross-domain run (15 domains, 345 queries) at 76.3% savings (424,792→100,875, Table 5); online learning lifting the L2 hit rate from 0% to 57% within the first 100 queries; and an ablation (Table 3) showing an LLM router is net-negative (−37.3%, with 225,628 routing tokens).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the TRIAGE framework for LLM agents?
TRIAGE (Three-level Routing and Intelligent Agent Guidance for Efficient Execution) is a framework that reduces the token cost of ReAct-style LLM agents by storing past execution trajectories and routing each new query by semantic similarity: exact matches are answered by direct reuse (Level 1), similar queries run a deterministic parameter substitution against automatically extracted execution 'Skills' (Level 2, zero tokens), and only novel queries trigger a full ReAct loop (Level 3). Routing uses cosine similarity thresholds of 0.98 and 0.90 with the all-MiniLM-L6-v2 encoder.
How much does TRIAGE reduce token consumption?
In the paper's main experiment on 1,007 self-built security-monitoring queries (deepseek-v3.2 as the ReAct model), TRIAGE reports 62.3% total token savings (199,782 → 75,238 tokens), with 61.5% of queries served at zero token cost (56.0% Level 2, 5.5% Level 1). A separate ToolBench run across 15 domains (345 queries) reports 76.3% average reduction. Both figures come from the authors' own query mixtures, so they reflect those distributions rather than guaranteed savings on other workloads.
Is TRIAGE's code and data available?
The paper states that an open-source implementation is available at github.com/weiruocan/triage. The 1,007-query dataset was generated by the authors from 15 parameterized SQL templates plus 60 recurring daily-report queries over a synthetic security-monitoring database; the templates themselves are not reproduced in the paper text, so exact regeneration depends on the repository contents.
What are the main criticisms of the TRIAGE paper?
The evaluation set is synthetic and, by the authors' own description, 'not statistically rigorous', so headline savings track the constructed query mixture; the 100% success rate is never defined and answer correctness is not evaluated; several internal figures conflict (Table 4's 55.0% ablation savings versus an arithmetic estimate near 5.5%; baseline totals 199,782 vs 198,508; an 87% vs 67% L1 peak; an unsupported 97.6% in the conclusion); and there is no comparison against existing semantic-caching systems such as GPTCache.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure images (Figs. 1–5) are not part of the extraction; plot-level behavior (e.g., the L1 hit-rate curve in Fig. 5) is assessed only via captions and in-text numbers.
- Minor PDF-extraction glyph artifacts exist (e.g., '¡1.0' for '<1.0', 'ightarrow' for '→'); none affect any number cited in this review, and no truncation was detected — the text runs to the end of the reference list.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.