Automating Attack Graph Construction for Agentic Pentesting. Towards Neuro-Symbolic Vulnerability Hunting
Oliver Stevanovic, Jasmin Wachter
cs.CR · cs.AI
Quick verdict
A semi-automated scanner-to-MulVAL pipeline shows feasibility on 54 web CTFs (100% goal reach, 53.7% coverage, 83.9% noise) but no agent-level evidence yet.
Summary
The paper presents a semi-automated pipeline that bridges heterogeneous vulnerability scanners (Trivy, Semgrep, Nmap) and the MulVAL/XSB logic attack-graph engine for agentic penetration testing. Scanner findings are parsed into MulVAL predicates, while an LLM-assisted offline process (Claude Opus 4.5, GPT-4o) generates domain-specific Datalog rules organized in a two-layer architecture: a MITRE ATT&CK-derived top-level rule base connected to web-security (OWASP Top 10) rules via connector predicates. The authors evaluate only the graph-construction infrastructure — not the downstream Hybrid Reasoner agent — on 54 web CTF tasks from CyBench with manually annotated ground-truth vulnerability classes. Every task produced at least one goal-reaching graph; median end-to-end time was 24.9 s (MulVAL reasoning 2.7 s). Quality is weaker: mean ground-truth vulnerability coverage of 53.7%, full coverage on 51.9% of tasks, and a mean noise-path rate of 83.9%; 24 of 54 tasks (44.4%) achieved zero coverage, with xss, auth_bypass, and path_traversal most frequently missing. The authors candidly acknowledge that reachability alone is insufficient, that rule-level validation was not performed, and that no agent-level ablation (unguided vs. graph-guided) is included. Main limitations: no baselines, unquantified manual effort in rule curation, a subjective usefulness component in the noise metric, and CTF-only generalization. Code is available via a public GitHub repository.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A semi-automated pipeline can translate Trivy/Semgrep/Nmap findings plus LLM-assisted Datalog rules into MulVAL fact/rule bases that yield goal-reaching attack graphs for all 54 evaluated web CTF tasks.
- The pipeline is runtime-practical for agentic workflows, with median end-to-end generation of 24.9 s of which MulVAL/XSB reasoning accounts for only 2.7 s.
- Current graph quality is insufficient for reliable agent guidance: mean ground-truth vulnerability coverage is 53.7% (full coverage on 51.9% of tasks; 44.4% zero coverage) and the mean noise-path rate is 83.9%.
- A two-layer predicate/rule architecture (MITRE ATT&CK backbone connected via connector predicates to a domain layer) makes the MulVAL knowledge base modular and customizable without manually rewriting hundreds of rules.
- The contribution establishes infrastructure and methodology only; it does not establish that symbolic attack-graph guidance improves exploitation success, efficiency, or reasoning quality.
Strengths
Honest scope discipline
Sections 1, 1.1 and 6 state explicitly and repeatedly that only graph-construction infrastructure is evaluated, not the agent; weak numbers (53.7% coverage, 83.9% noise) are reported without spin, and Section 7 concedes 'reachability alone is an insufficient quality criterion'.
Real problem, sensible modular architecture
The two-layer predicate/rule design (MITRE ATT&CK top layer joined via connector predicates to an OWASP-domain layer, Sections 3.1-3.2) addresses the documented MulVAL customization pain ([23]) without introducing a new formalism — the right engineering target for scanner-to-symbolic interoperability.
Runtime feasibility demonstrated
Concrete failure analysis
Reproducibility groundwork
Weaknesses
No baselines or ablations
Noise metric is partly subjective
LLM rule generation unvalidated and unquantified
Graph quality insufficient for the stated purpose
Domain-aligned CTF-only evaluation
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This ESORICS 2026 workshop paper (arXiv:2609.15523) attacks a real interoperability problem: contemporary scanners (Trivy, Semgrep, Nmap) do not emit MulVAL predicates, and generic MulVAL rules do not encode web-specific attack patterns. The claimed contribution is explicitly infrastructural — a semi-automated pipeline that (1) parses scanner findings into an initial fact base, (2) uses an LLM-assisted offline process (Claude Opus 4.5, GPT-4o) to author domain-specific Datalog rules under a two-layer architecture (a MITRE ATT&CK-derived top-level rule base joined via connector predicates to an OWASP Top 10 web-security layer), and (3) runs MulVAL/XSB to derive attack paths for a downstream CAI-based 'Hybrid Reasoner' agent. The authors are unusually disciplined about scope: Sections 1, 1.1 and 6 state repeatedly that the agent is not evaluated, that no neuro-symbolic feedback loop exists yet, and that rule semantics were not formally validated. This honesty is commendable — but it also means the paper's practical promise rests entirely on graph quality, which the reported numbers show is not yet high.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is MulVAL and why use it for AI pentesting?
MulVAL (USENIX Security 2005) is a logic-based network security analyzer: it takes a Datalog fact base describing hosts, services, and vulnerabilities plus inference rules, and uses the XSB Prolog engine to automatically derive multi-host, multi-stage attack paths where every edge is a traceable logical derivation. This paper argues such explicit, auditable paths could give LLM pentesting agents structured, inspectable context that prompt-based reasoning lacks.
Did this paper show that attack graphs improve LLM pentesting agents?
No. The authors evaluate only the graph-construction infrastructure on 54 CyBench web CTF tasks. The downstream CAI-based 'Hybrid Reasoner' agent is integrated but explicitly not evaluated; a three-condition agent ablation (no graph vs. raw scanner output vs. attack graph) is left as future work.
How well did the automated attack graph pipeline perform?
Every one of the 54 tasks produced at least one goal-reaching graph, at a median of 24.9 s end-to-end (2.7 s of actual MulVAL reasoning). However, mean ground-truth vulnerability coverage was only 53.7% (51.9% of tasks fully covered, 44.4% with zero coverage) and 83.9% of generated paths were noise unrelated to the required vulnerabilities.
Is the code and data for this paper available?
Yes, per the paper: implementation and source code are in a public GitHub repository (the Hydra Hybrid Reasoner project, reference [26]), the benchmark tasks come from the public CyBench collection, and all scanners used (Trivy, Semgrep, Nmap) are open-source tools.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1 (framework architecture) and 2 (example attack graph) could not be analyzed from the extracted text.
- The public GitHub repository (Hydra) was not inspected; code availability is as stated by the authors.
- The adjudication protocol behind the noise metric's usefulness judgment is not described in the text, limiting assessment of the 83.9% figure's objectivity.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.