MeshHeal: Two-Timescale Self-Healing for Gray Failures in Decentralized LLM Agent Networks
Keru Chen, Sen Lin, Yingbin Liang, Nathaniel D. Bastian, Shaofeng Zou
cs.AI · cs.CL · cs.DC
Quick verdict
MeshHeal's two-timescale peer review heals gray LLM-agent failures; thorough evaluation, but injected degradation (120B→1B swap) is extreme, easing detection.
Summary
MeshHeal targets gray failures in decentralized LLM multi-agent systems, where an agent stays responsive but its task-solving quality persistently degrades. The framework couples two timescales. Fast: ability-matched neighbors review every output (median of three single-reviewer calls); uncertain or low-scoring outputs escalate to a two-reviewer committee and, if needed, Mixture-of-Agents-style Takeover correction. Slow: a task- and ability-conditioned peer-relative detector accumulates review scores, normalized by healthy-neighbor variation, moving agents through normal, watched, and isolated states, with recovery probes enabling reintegration. The paper also introduces Model-Backed MAS Evaluation, showing that system-prompt ability assignments fail to constrain agents (added accuracy gaps of only 0.015-0.058 across five models) and using backend swaps (gpt-oss-120b versus llama-3.2-1b) to inject failures with known ground truth. Across BBH, MATH, and MMLU-Pro with six agents and five seeds, MeshHeal reports 0.839 degraded-phase accuracy at 51k tokens per task versus Symphony's 0.807 at 115k, and isolates all degraded agents in networks up to 576 agents. Limitations found in review: injected degradation is extreme (review scores 0.08 versus 0.89), degraded-phase accuracy exceeding healthy-phase accuracy (0.839 vs 0.831) is unanalyzed, no significance tests are reported, and no code release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- MeshHeal achieves 0.839 degraded-phase accuracy at 51k total tokens per task across BBH, MATH, and MMLU-Pro (six agents, five seeds), versus the strongest baseline Symphony at 0.807 and 115k tokens.
- System-prompt ability assignments do not reliably constrain agent competence (added assigned-vs-unassigned accuracy gaps of only 0.015-0.058 across five models), motivating Model-Backed MAS Evaluation with backend-selected abilities and controlled gray-failure injection.
- The peer-relative detector with fixed thresholds yields zero missed detections and zero false isolations across two model pairs, where an absolute-threshold rule falsely isolates 4/4 healthy agents under pair A (Tab. C7).
- MeshHeal scales to complete and sparse networks of up to 576 agents, isolating every degraded agent (32/96/192) with sparse-graph accuracy within 0.010 of complete graphs (Tab. 2b).
- Recovery probes reintegrate recovered agents in the staggered schedule, with detection delays of 5 and 15 tasks and reintegration delays of 51 and 45 tasks (Tab. C5).
Strengths
Genuine evaluation-methodology contribution
Model-Backed MAS Evaluation (Sec. 5, App. A) quantifies a real confound: Table 1 shows declaring unassigned abilities 'unreliable' adds only 0.015-0.058 to the assigned-vs-unassigned gap across five models, and assigned-ability accuracy drops for four of five — so prompt-only MAS evaluations can hide routing errors. Backend swaps give clean failure ground truth while preserving responsiveness.
Well-motivated two-timescale architecture
Secs. 1-2 articulate why within-task aggregation (SAC, Symphony) cannot retain producer-specific evidence and why routing-only adaptation (AgentNet, RAPS) cannot protect the current task or reintegrate recovered agents; MeshHeal's fast review loop plus slow peer-relative detector with recovery probes (Sec. 4.2) addresses all three gaps in one design.
Unusually complete and honest empirical package
Fixed hyperparameters across settings
Weaknesses
Injected 'gray' failure is extreme, not gray
Unexplained accuracy inversion and cross-table inconsistency
Thin statistical support
Baseline asymmetry and possibly favorable configuration choices
Reproducibility and untested structural assumptions
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
MeshHeal (arXiv 2609.29015, cs.AI/cs.CL/cs.DC) proposes a fully decentralized self-healing framework for LLM multi-agent systems under *gray failures* — agents that stay online and protocol-compliant while task-solving quality persistently degrades (Sec. 3.2). Three claims carry the paper: (1) coupling fast per-task review/correction with slow cross-task detection, rerouting, and probe-based reintegration protects accuracy at lower cost than aggregation baselines (0.839 at 51k tokens vs Symphony's 0.807 at 115k, Tab. 2a); (2) system-prompt ability assignments do not bind agents, so faithful routing evaluation requires backend-backed abilities (Model-Backed MAS Evaluation, Sec. 5); (3) fixed detector thresholds transfer across datasets and model pairs.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is a gray failure in this paper?
A gray failure is when an agent in a multi-agent system stays online and follows the communication protocol, but its task-solving quality persistently drops relative to its nominal ability (for example, due to a degraded model backend or corrupted context). Heartbeat or connectivity checks see nothing wrong, so the failure must be inferred from the semantic quality of the agent's outputs — which is exactly what MeshHeal's peer review does.
How does MeshHeal detect a degraded agent?
Neighbors that hold the matching ability score every output the agent produces (median of three LLM-judge calls, committee review when scores are low or disagree). A per-(task-type, ability) detector then compares the agent's average shortfall against healthy peers' scores on the same task type, normalized by how much healthy peers naturally vary. A z-score of at least 1.2 puts the agent under mandatory committee review (watched); two consecutive updates at 2.0 or above isolate it from ordinary routing. Recovery probes let isolated agents earn their way back.
What results does MeshHeal report?
On BBH, MATH, and MMLU-Pro with six agents and five seeds, MeshHeal reports 0.839 accuracy during the degraded phase using about 51k model tokens per task, versus the strongest baseline Symphony at 0.807 using 115k tokens. In networks of 96 to 576 agents, including sparse graphs with under 1% of complete-graph edges, every injected degraded agent is isolated.
Is code available for MeshHeal?
No code or artifact release statement appears in the extracted paper text. The method is specified in unusual detail across the appendices (judge prompts, all parameters, ability mappings, worked traces), but whether an implementation will be released must be confirmed with the authors, and reruns require specific hosted model backends (gpt-oss-120b and llama-3.2-1b).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Extracted text omits portions of Sec. 4.1, 6.2, C.1, B.12, and D.2 (marked as omitted); figures (Fig. 1, Fig. C1) are not analyzable from text alone.
- The Tab. C6 ablation configuration (model pair, stream) is not stated in the extracted text; the 0.839-vs-0.900 discrepancy may be explained in the missing portion.
- No code-availability statement was found in the provided text; one may exist in an omitted section or the paper's landing page.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.