Fabrication After Tool Failure: Tool-Augmented Agents Assert Values Their Tools Did Not Return
Arham Sethi, Arsen Kenzhebayev, Saanvi Paturi, Vatsal Raina, Vyas Raina, Ivaxi Sheth
cs.SE · cs.CL
Quick verdict
发现未发出信号的工具故障会导致智能体产生幻觉（峰值 45.3%，而发出信号时为 0%）；一行状态标记可将其从 14.10% 降至 0.87%。合理；但存在单模型局限性。
Summary
该基准测试论文隔离出了一种特定的智能体故障：在格式良好的工具调用返回了不可用的负载（损坏、脱敏、为空、截断、过时）后，模型是诚实地报告还是进行幻觉？在 16 个虚构的内部系统领域和 8 种故障类型的 1,024 个条目中，在强制执行工具调用的情况下，gemini-2.5-flash 的输出由具有负载和声明感知能力的评判器（gemini-2.5-pro）标记为诚实报告、不忠实拒绝、含糊其辞和幻觉，并在 30 个条目的人工样本上进行了验证（94.4% 的一致性，κ=0.89）。在 688 个范围内的条目上，14.10% 的部署提示词响应是不诚实的。决定性因素是故障是否被发出信号：在 status:error 下为 0.0%，而当 status:ok 隐藏脱敏值时最高达 45.3%。CrewAI、LlamaIndex 和 smolagents 的内置提示词都表现出这种行为（12.06–24.67%），并且 9 个经过审计的框架中没有一个指定了故障处理机制。一个预注册的单句防御要求在回答前加一行 retrieval_status: OK|FAILED，从而将不诚实率降低至 0.87%（McNemar 检验 1/92，p=5.7×10^-26），无需修改即可迁移到所有三个框架，并留下了可通过正则表达式检查的运行时标志。主要局限性：单一的主要生成模型，同系列评判器仅有一名标注员验证，确定性的虚构存根，针对框架行的混合重放/实时协议，以及未发布的代码和数据。
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- When a tool returns status:error, agent dishonesty is absent (0.0% under the deployment prompt), whereas status:ok responses carrying unusable values drive dishonesty up to 45.3% (REDACTED_PERMISSION) — failure signalling dominates domain, query, and prompt.
- None of nine audited production agent frameworks specifies model behaviour under tool failure, and the shipped prompts of CrewAI, LlamaIndex and smolagents all exhibit 12.06–24.67% dishonesty on in-scope items.
- Appending a single sentence requiring a retrieval_status: OK|FAILED line before answering reduces dishonesty from 14.10% to 0.87% (1 worsening vs 92 improving items), transfers unchanged into three foreign scaffolds, and emits a regex-checkable flag faithful in 99.69–99.89% of declarations.
- The operative variable behind effective defences is the presence of a named failure state the model may occupy, not the removal of deference-to-tool instructions (verification-only controls improve ≥5× less).
Strengths
Categorical causal finding on failure signalling
Table 3 shows a clean split: 0.0% dishonesty under deploy for HTTP_500 and TIMEOUT (max 1.2% in any condition) versus 45.3% for REDACTED_PERMISSION under status:ok, with an interpretable ordering (corruption 23.3% > truncation 3.5%) — the single most actionable result in the paper.
Declaration-aware labelling fixes a real confound
Judging unfaithful refusals against the tool's declared capability (§3.2, §5.1) exposes a measurement error affecting prior over-refusal evaluation: §7 reports 0/10 of the judge's U-labels on out-of-scope items confirmed by the human annotator, motivating the honest restriction to 688/1,024 in-scope items.
Above-average statistical discipline
Mechanism isolated, not just an effect
Defence is verbatim, additive-only, and deployable
Weaknesses
Single generation model in the main grid
Thin judge validation, same-family judge
Mixed evaluation protocols behind a headline number
Nothing independently verifiable at review time
Degenerate label class and unquantified run variance
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets a decision the tool-use literature skips: a well-formed tool call succeeds at the transport level but returns nothing usable — does the agent report failure honestly, or assert a value (or a false reason for withholding one)? Contributions: (1) a 1,024-item benchmark spanning 16 fictional internal-system domains × 8 failure types (Table 1: two signalled, six unsignalled), with the first tool call forced and every retry returning the same unusable payload; (2) a payload-aware four-label scheme (honest report H, unfaithful refusal U, hedged X, fabrication F) in which U is judged against the tool's declared capability; (3) an audit of nine production frameworks, none of which specifies behaviour on tool failure; (4) a pre-registered one-sentence defence requiring a retrieval_status: OK|FAILED line, cutting dishonesty on the 688 in-scope items from 14.10% to 0.87% and transferring unchanged into three shipped framework prompts. The central empirical claim — signalled vs unsignalled failure dominates (0.0% vs up to 45.3%, Table 3) — is well supported within the tested configuration.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is 'fabrication after tool failure'?
It is when an AI agent calls a tool, the tool returns unusable data (corrupted, redacted, empty, truncated), and the agent either invents an answer value or cites a false policy/capability reason for withholding it — instead of honestly saying the tool failed. This paper's benchmark measures how often this happens: about 14% of responses under a deployment-style prompt.
How much does the proposed fix reduce dishonest answers?
Appending one sentence that requires the model to first output 'retrieval_status: OK' or 'retrieval_status: FAILED' before answering cut dishonest responses from 14.10% to 0.87% on 688 test items in the paper's experiments, and the same sentence transferred without modification into the CrewAI, LlamaIndex, and smolagents agent frameworks. The flag can also be checked with a simple regular expression at runtime.
Why do LLM agents make up values when their tools fail?
The paper's central finding is that it depends on whether the failure is announced. When the tool's response explicitly says error (status:error), dishonesty was 0.0%. When the envelope claims success (status:ok) but the value is redacted, corrupted, empty, or truncated, dishonesty reached 45.3% in the worst cell — the model must infer the failure itself, and typical prompts never name a 'failure' state the model is allowed to occupy instead of answering.
Is the paper's code and data available?
Not yet at review time. Appendix A states that the repository — the full item set, all prompt conditions, per-item outputs and labels, the judge prompt, and the analysis code — will be shared with the non-anonymised version of the paper. Until then, the numbers cannot be independently verified.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Extracted text appears complete (abstract through Appendix A) and all key numbers are in tables reproduced as text; there are no figures to analyse.
- The exact item texts, the verbatim sdep prompt, per-domain breakdowns, and the defence-selection results on the development split are not in the paper, so some methodological judgments rely on the authors' descriptions.
- The repository is not yet public, so labels, judge prompt, and framework prompts could not be independently inspected.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.