TriFleetRCA: On-Premise LLM Root Cause Analysis for Kubernetes
Rohit Patel, Susil Kumar Mohanty, Jeenal Chaudhary
cs.CR · cs.AI · cs.DC · cs.LG
Quick verdict
Single-GPU on-prem RCA for K8s hits 0.85–0.95 over 4 injected faults (n=100); honest, small-scale evaluation with wide CIs and one toy attack; reproducible.
Summary
TriFleetRCA is a root cause analysis pipeline that runs entirely on one on-premise GPU, targeting operators who cannot ship production logs to hosted models. It collects evidence from pod logs (Loki), Kubernetes events and API objects at one of three scopes (pod, namespace, cluster), reduces lines to templates, de-duplicates them, ranks survivors with BM25 into a fixed 25-line prompt, filters runbooks through a pattern-based ingest guard, and has Qwen2.5-14B-Instruct return a root cause with cited evidence lines. Evaluation uses a live three-node k3d cluster with four injected faults (oom, badimage, backend-down, dns-blocked), fresh namespaces and injection-bounded log windows; 100 analyses (4 faults × 5 repetitions × 5 configurations) at temperature 0. Hit rates were 0.85/0.90/0.95 at pod/namespace/cluster scope with overlapping Wilson intervals; the entire scope effect comes from dns-blocked, whose NetworkPolicy cause is only collected at wider scopes, and cluster scope costs 55% more tokens. Template de-duplication raised hit rate from 0.75 to 0.90 at namespace scope at equal token cost. A poisoned runbook was blocked by the guard deterministically, but with the guard disabled the model refused it in all 20 analyses — the guard protected nothing here. Grounding was 0.75 overall, with all failures in dns-blocked: the model reasons from the cause yet cites symptom lines. Limitations: few fault types, one model family, one cluster, regex-based grading.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A single on-premise GPU running Qwen2.5-14B-Instruct achieves 0.85–0.95 hit rates on four injected Kubernetes faults, with the retrieval-scope effect confined to the one fault whose cause is a cluster-level object (dns-blocked).
- Template de-duplication before BM25 ranking raises hit rate from 0.75 to 0.90 at namespace scope at equal token cost (2176 vs 2149 median tokens).
- The ingest guard deterministically blocks the poisoned runbook in all 80 guard-enabled analyses, but with the guard disabled the model declined it in all 20 — so the guard is defence in depth, not a demonstrated barrier.
- Grounding (cited evidence actually containing the fault signal) is a property distinct from accuracy: overall 0.75, with all failures in dns-blocked where the model reasons from the cause but cites symptoms.
Strengths
Honest statistical reporting
Wilson 95% CIs on every hit rate (Tables I, III), and the authors explicitly refuse to claim scope differences are significant despite a monotonic 0.85→0.90→0.95 trend (VI-A) — rare discipline at n=20/cell.
Negative result reported against interest
Section VI-C shows poison-followed 0/20 with the guard disabled, and states plainly that 'a claim that it protects against runbook poisoning is not supported by these data' — resisting the incentive to oversell their own TriShieldRAG guard line.
Grounded-citation metric separates accuracy from support
Strong reproducibility design
Practitioner-useful failure analyses
Weaknesses
Very small empirical base
Adversarial evaluation is a single data point
Regex grading revised post hoc
No external baselines or alternative retrievers
Guard false-positive cost unmeasured
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
TriFleetRCA (arXiv 2609.23766, IIT Jodhpur) presents a root cause analysis pipeline that runs entirely on one on-premise GPU, motivated by two constraints: many operators cannot ship production logs to hosted models, and runbooks are editable by many people. The system collects evidence at one of three scopes (pod, namespace, cluster) from Loki logs, Kubernetes events and API objects, de-duplicates log templates, ranks with BM25 into a fixed 25-line prompt, filters runbooks through a pattern-based ingest guard, and has Qwen2.5-14B-Instruct (temperature 0) return a root cause plus cited evidence lines. Five central claims: (1) one workstation GPU suffices, at 0.85–0.95 hit rate over four injected faults; (2) the retrieval-scope effect is confined to the single fault whose cause lives above the namespace (dns-blocked); (3) template de-duplication is worth 15 points at equal token cost; (4) the ingest guard is defence in depth — the model refused the poisoned runbook even with the guard off (0/20); (5) grounding (does cited evidence contain the fault signal?) is a property separate from accuracy. A fleet/multi-site component is named but explicitly not evaluated (VIII-G).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is TriFleetRCA?
A root cause analysis pipeline that runs entirely on one on-premise GPU: it collects Kubernetes pod logs, events and API objects at pod, namespace or cluster scope, de-duplicates log templates, ranks evidence with BM25 into a 25-line prompt, filters runbooks through an ingest guard, and uses Qwen2.5-14B-Instruct to return a root cause with cited evidence lines — with no external network calls.
How accurate is TriFleetRCA?
In the paper's 100-analysis live-cluster evaluation (four injected faults, temperature 0), hit rates were 0.85 at pod, 0.90 at namespace and 0.95 at cluster scope, though the confidence intervals overlap. The entire scope effect comes from the dns-blocked fault, whose cause is a cluster-level NetworkPolicy; median latency was 1.6 seconds per analysis.
Can a poisoned runbook make the model delete a Kubernetes namespace?
In this paper's test, no. A runbook instructing namespace deletion was rejected by the ingest guard every time, and even with the guard disabled the 14B model refused to follow it in all 20 analyses. The authors caution this was one loudly-worded attack and one model — a subtler phrasing could pass the guard's pattern list, so the guard is justified as defence in depth, not as a proven barrier.
Is the TriFleetRCA code and data available?
Yes — the paper states that the pipeline, the fault injector, all per-analysis records including every model output verbatim, and the scripts that generate every table are publicly available at github.com/SPriTLab-iitj/TriFleetRCA.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1 and 2 are referenced but their content cannot be assessed from the extracted text.
- Repository contents (code, raw records, grading patterns) were not independently verified; availability is as stated by the authors.
- References [1], [2], [30] and [31] are 2026 preprints that could not be checked from the provided text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.