Large Language Models (LLMs) for Telecom Root Cause Analysis (RCA): A Structured Reasoning Framework for Evidence-Grounded Diagnosis
Hao Zhou, Mandar Kulkarni, Hao Chen, Yan Xin, Charlie, Zhang
cs.AI
Quick verdict
SEKA-FT structured fine-tuning lifts a 1.5B LLM to 0.942 RCA accuracy on TeleLogs; label-derived supervision and weak baselines temper the claim.
Summary
The paper proposes SEKA-FT, a structured fine-tuning framework for LLM-based root cause analysis (RCA) in 5G telecom networks, framed by a survey of the shift from rule-based and ML approaches to LLM paradigms (CoT, RAG, agentic orchestration, RLVR). SEKA-FT restructures heterogeneous telemetry into canonical evidence blocks, supervises intermediate deterministic diagnostic checks (speed, scheduling, handover, coverage distance), and adds explanations auto-generated from ground-truth labels, check outcomes, and rule templates. Fine-tuning Qwen2.5-1.5B-Instruct with LoRA yields 0.942±0.006 accuracy / 0.937±0.007 Macro-F1 on TeleLogs and 0.647±0.004 / 0.615±0.005 on TelecomTS, versus 0.256 and 0.132 accuracy for structured-input-only SFT; in-context learning stays below 0.13 accuracy even with Qwen3-32B. Results average ten seeds with 95% CIs, McNemar and permutation tests with Holm-Bonferroni correction (p<0.001). Key limitations: because supervision targets derive from labels and deterministic rules, the model largely distills an expert decision tree rather than demonstrating open-ended reasoning; several baselines perform below chance (vanilla SFT 0.007 on 8 classes), suggesting under-tuning; the accuracy gap between the two datasets is not analyzed; and no code accompanies the magazine-format paper, whose figures carry much of the evidence.
Review complete
Assessment at a glance
4
risk flags identified
● high 1
● medium 2
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Vanilla LLMs fail at telecom RCA regardless of scale: ICL accuracy stays at or below 0.126 on TeleLogs and 0.061 on TelecomTS, even for Qwen3-32B (Table I).
- Canonical context structuring is the dominant single component before the full method, lifting TeleLogs accuracy from 0.007 (Vanilla SFT) to 0.256 (SFT+Structured Input, Sec. V-B).
- SEKA-FT reaches 0.942±0.006 accuracy / 0.937±0.007 Macro-F1 on TeleLogs and 0.647±0.004 / 0.615±0.005 on TelecomTS, with Holm-Bonferroni-adjusted p<0.001 against the strongest baseline on both datasets and metrics.
- Structured intermediate checks improve decision-path consistency (Fig. 5d) and reduce the effective decision space from 8 to an average of 2.5 candidates after four checks (Fig. 5f).
Strengths
Exemplary statistical protocol for the venue
Sec. V-A/V-C: ten seeds, 95% Student-t CIs on all Table I entries, exact McNemar's test on Accuracy and paired permutation tests on Macro-F1 against the strongest baseline with Holm-Bonferroni adjustment; all comparisons p<0.001 on both datasets — rare rigor in telecom-LLM literature.
Consistent ablation ordering across two independent datasets
Table I ranking (ICL < LSTM < raw SFT < SFT+Explanation < SFT+Structured Input < SEKA-FT) replicates on TeleLogs and TelecomTS, giving the component decomposition (Sec. IV-A/B/C) external consistency; canonical structuring alone lifts TeleLogs accuracy from 0.007 to 0.256.
Practically significant small-model result
Reusable and interpretable supervision recipe
Honest scope statement
Weaknesses
Supervision targets are label- and rule-derived, so the method largely distills a decision tree
Baselines perform at or below chance, raising tuning-parity doubts
Cross-dataset gap unexplained and TelecomTS task under-specified
Check thresholds are copied from the candidate option text, making the task close to rule matching
No code release and reviewed paradigms never evaluated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This magazine-format paper (accepted by IEEE Wireless Communications Magazine; Samsung Research America) pairs a compact survey with a method. Sections I-III trace telecom RCA from rule-based expert systems and ML classifiers to four LLM paradigms: CoT (III-A), RAG (III-B), agentic orchestration (III-C), and RLVR (III-D). Section IV defines SEKA-FT, a three-stage fine-tuning framework: (1) canonical context structuring normalizes heterogeneous telemetry into fixed evidence blocks (Bottleneck Snapshot, UE Global State, Trajectory-Level Events, engineering parameters; Fig. 3); (2) CoT-enabled decision-path control supervises four deterministic checks (speed >40 km/h, scheduled RBs <160, handover count, UE-to-serving-cell distance); (3) explanation targets are 'automatically constructed from the ground-truth label, deterministic check results, and domain-rule-based hypothesis refinement templates' (Sec. V-B). The headline claim: this evidence-to-path-to-decision supervision lifts a 1.5B model to 0.942±0.006 accuracy on TeleLogs and 0.647±0.004 on TelecomTS, far above ICL and SFT baselines.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is SEKA-FT in this paper?
SEKA-FT (Structured Evidence- and Knowledge-Aligned Fine-Tuning) is a fine-tuning framework from Samsung Research America that trains small LLMs (Qwen2.5-1.5B) for 5G telecom root cause analysis. It restructures network telemetry into fixed evidence blocks, supervises four deterministic intermediate diagnostic checks (UE speed, scheduled resource blocks, handover count, cell distance), and trains on explanations auto-generated from the ground-truth label, check results, and hand-written rule templates.
How accurate is SEKA-FT and how does it compare to baselines?
It reaches 0.942 accuracy / 0.937 Macro-F1 on the TeleLogs 5G dataset and 0.647 / 0.615 on TelecomTS, versus 0.256 and 0.132 accuracy for fine-tuning with structured inputs alone. Plain prompting (in-context learning) stays below 0.13 accuracy even with a 32B model, and the non-LLM LSTM baseline sits near chance.
Is code or data available for this paper?
No code release is mentioned. The evaluation uses two public benchmarks — TeleLogs (associated with the GSMA LLM benchmarks initiative, from Sana et al.) and TelecomTS (Feng et al., ICML 2026) — and the paper reports LoRA fine-tuning hyperparameters, but the explanation-template pipeline and prompts are not published.
What are the main criticisms of this paper?
The training targets are generated from ground-truth labels and deterministic rules whose thresholds appear verbatim in the task prompts, so the model largely distills a rule-based decision tree rather than learning open-ended reasoning. Several baselines perform below chance (vanilla SFT: 0.007 on an 8-class task), suggesting under-tuning, and the accuracy gap between the two datasets (0.942 vs 0.647) is not analyzed.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Graphical data in Figs. 1-5 (curves, bars, diagrams) is not recoverable from the extracted text; judgments about Fig. 5(b)-(f) rest on the paper's prose descriptions, and figure-only values (e.g., exact convergence curves, consistency scores) could not be independently checked.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.