Feyospace-v1: How the Cyber Mercury Seven Trained Frontier Cyber Models
Zongjie Li, Alan Z. W, John Nicolas J, Walter H. F, Scott Donald L, Gordon Y. P, Deke X
cs.AI
Quick verdict
Big SFT data pipeline yields real CyberGym gains for open models, but 3 of 5 headline systems went unused and provider-abuse methods raise ethics concerns.
Summary
An independent seven-person team reports a data-centric post-training program that turns open-weight Qwen models into strong agentic cyber operators. A data engine builds resettable, execution-verified environments: 27,502 repository-level coding instances, 69,854 CVE-reproduction environments screened from 343,214 vulnerability records, 9,312 CTF challenges, 12,993 Linux-kernel-history environments, 1,601 exploit-construction cases, and 1,377 firmware or device-backed targets. Teacher rollouts pass a four-layer evidence audit (85% accepted), yielding 164,269 trajectories for SFT-only training of three checkpoints (Qwen3.6-35B-A3B, Qwen3.8-27B, Qwen3.5-122B-A10B). On the full 1,507-task CyberGym suite, verified success rises by 41.54, 8.73, and 21.00 points; Feyospace-s1 reaches 63.24% and rank 10 on the official leaderboard as of September 1, 2026. Pooled CTF success improves by 13.00, 6.70, and 11.78 points. Main limitations: three of the five headline techniques (reasoning-trace recovery, gray-market token arbitrage, jailbreak orchestration) were explicitly excluded from the final training pipeline; evaluations are single-run without confidence intervals; closed-model comparisons reuse developer-reported numbers under different scaffolds; and the paper documents provider-abuse methods and deliberate training-data routing manipulation that raise serious ethical and legal concerns despite impressive engineering.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- SFT-only training on 164,269 execution-verified, audited trajectories improves three Qwen checkpoints by an average of 23.76 points on the full 1,507-task CyberGym suite and 10.49 points across pooled CTF suites.
- Feyospace-s1 reaches 63.24% verified success and rank 10 on the official CyberGym leaderboard as of September 1, 2026, with all three checkpoints claimed first among models at comparable parameter scales.
- A seven-person independent team can execute an end-to-end cyber post-training program, with early self-funded costs of $36,070 recorded in Appendix A.
- Hidden reasoning traces are recoverable from three major commercial providers (92.8% weighted recovery before the August 21, 2026 mitigation) but were excluded from SFT as training targets due to cost and contamination risk.
- EXP-specific SFT lifts exploit capability from vulnerability reproduction (T4) to target-specific primitives (T3) on two V8 cases, while generic primitives and full control remain unresolved.
Strengths
Execution-verified environment construction at unusual scale and rigor
Category A screens 343,214 CVE records to 119,732 candidates and 69,854 buildable environments, retaining a case only when a PoC stably reproduces on the vulnerable revision and, where available, fails on the patched one (§3.2); Category C rebuilds 12,993 kernel-history environments in resettable QEMU with a prefix-completion decontamination test.
Complete, unusually detailed SFT recipe
§4 specifies per-token loss weights (reasoning 0.8; final answers and tool calls 1.0; context masked), 262,144-token packing with block-diagonal causal masks, role-faithful serialization with the V1 interface-mismatch failure documented and slated for release, plus exact optimizer settings and per-model parallelism (Table 11).
Candor about what was not used
Behavioral evidence against reward hacking
Teacher-student reasoning compatibility finding
Weaknesses
No uncertainty quantification
Framing overstates the actual recipe
Confounded closed-model comparisons
Documented provider abuse and fraud infrastructure
Unverifiable internal statistics
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper reports Feyospace-v1, a post-training program by Vera Praxis Lab, a self-described seven-person independent team. The claims: (1) a data engine producing resettable, execution-verified environments — 27,502 coding instances, 69,854 CVE-reproduction environments (Category A, from 343,214 records), 9,312 CTF environments (B), 12,993 kernel-history environments (C), 1,601 exploit cases (D), and 1,377 firmware/device targets; (2) 164,269 audited trajectories after a four-layer evidence gate (85% accept / 9% review / 6% reject); (3) SFT-only training of three Qwen checkpoints with average gains of +23.76 points on the full 1,507-task CyberGym suite and +10.49 points pooled CTF; (4) Feyospace-s1 at 63.24%, rank 10 on the official CyberGym leaderboard (September 1, 2026). A fifth claim — first end-to-end demonstration by a seven-person independent team — is a priority assertion, not a testable result.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Feyospace-v1 and what does it claim?
A preprint describing how a seven-person independent lab (Vera Praxis Lab) post-trained three open-weight Qwen models into cybersecurity agents using 164,269 execution-verified training trajectories, reporting gains of up to +41.54 points on the CyberGym benchmark and rank 10 on its leaderboard for Feyospace-s1 (63.24%) as of September 1, 2026.
Are the Feyospace models, data, or code available?
The paper promises to release the 164,269 training trajectories, the 200-case CyberGym subset task list, and a diagnostic dataset (V1). It does not state that model weights or training code are released, and no repository link appears in the provided text.
Why is this paper controversial?
It documents techniques for extracting hidden reasoning from commercial AI APIs, acquiring fraudulently sourced discounted accounts in gray markets, orchestrating jailbreaks against teacher models, and appending suffixes to conversations so providers exclude them from training data. The authors state three of these systems were not used in the final training pipeline, but publishing the operational details raises legal and ethical concerns.
How strong are the results?
Gains over the starting checkpoints are large for two of three models (+41.54 and +21.00 points on CyberGym; +13.00 and +11.78 pooled CTF) but modest for Feyospace-s1 (+8.73 and +6.70). Closed frontier models remain ahead (GPT-5.5 Cyber 85.6%, Claude Mythos Preview 83.1%). All evaluations are single runs without confidence intervals.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt with five '[...sampling gap...]' breaks; Kreator details (§2.5), §3.3 hardware construction, parts of §3.4, and the conclusion were partially or fully skipped, so judgments rest on the visible text.
- Figure text (e.g., Figures 4, 8-10) was extracted as interleaved axis labels and captions; plotted values quoted here come from readable captions and body text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.