AMDKernelVault: Large-Scale Datasets and Agentic Training for AMD GPU Kernel Optimization
Ji Liu, Saptarshi Majumder, Yiqing Huang, Wenwen Ouyang, Umang Pandey, Zeping Li, Chushi Chen, Zihao An, et al.
cs.CL
Quick verdict
AMD's execution-verified HIP/Triton kernel corpus trains an 8B model that tops correctness on 3 benchmarks; evals are small and semantic overlap unaudited.
Summary
The paper introduces AMDKernelVault, an open corpus and pipeline for AMD GPU kernel generation, targeting the CUDA-centric gap in LLM kernel research. Two agent pipelines, HIPKernelGen and TritonKernelGen, convert PyTorch references into HIP or Triton kernels, compile them under ROCm, validate correctness by execution (torch.allclose with rtol/atol 1e-4 for HIP), and profile latency on MI325/MI350 hardware. The released corpus contains 62,153 execution-verified HIP samples (HIP-CudaAgent 5,388; HIP-GPUMode 22,397 variants; HIP2HIP 34,368), 2,377 QA entries mined from rocBLAS/rocSOLVER, and 39,893 Triton kernels from four sources. As a utility demonstration, the authors fine-tune Qwen3-8B with SFT followed by execution-aware GRPO using a hierarchical reward (0.4 compile + 1.0 correct + capped speedup term) and disabled variance normalization. Under the same GEAK OptimAgent-v2 three-iteration budget, the trained model attains the highest correctness among compared models on PyTorch-to-HIP (34.0% Pass@1, 200 KernelBench tasks), TritonBench-G (33.2% Corr@3, 184 kernels), and ROCmBench (41.94% Corr@3, 31 kernels), but trails Claude Sonnet 4 on compile rate and speed on ROCmBench and sits below GPT-5 on HIP-to-HIP correctness (84% vs 87%). Limitations: no significance testing on small eval sets, unquantified semantic train/test overlap, and non-uniform rejection logging across pipelines.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- AMDKernelVault releases 62,153 execution-verified HIP samples, 2,377 rocBLAS/rocSOLVER QA entries, and 39,893 Triton kernels, all validated on recent AMD CDNA GPUs.
- Under an identical 3-iteration GEAK OptimAgent-v2 budget, SFT+RL Qwen3-8B attains the highest correctness among compared models on PyTorch-to-HIP (34.0% Pass@1), TritonBench-G (33.2% Corr@3), and ROCmBench (41.94% Corr@3), while explicitly not leading compilation or speed everywhere.
- Hierarchical rewards plus disabled variance normalization are necessary for GRPO in the sparse-success AMD kernel regime (ablations: no-SFT 8.5%, single-turn 20.1% vs full 33.2% Corr@3).
- A single local 8B policy can serve as generator, reflector, and optimizer in a GEAK-style loop, reducing reliance on repeated frontier-LLM calls.
Strengths
First large-scale AMD-native, execution-verified kernel corpus
62,153 HIP samples plus 39,893 Triton kernels validated by ROCm compilation, execution-based correctness (torch.allclose, rtol/atol 1e-4 for HIP) and latency profiling on MI325/MI350; per-source provenance and license notes are documented (Appendix A), and subset counts reconcile (Table 1 vs Section 3.3).
Controlled same-scaffold model comparisons
All Triton rows use GEAK OptimAgent-v2 with num_offsprings=1, max_iteration=3, identical generator/reflector/optimizer roles; only the checkpoint changes (Tables 4, 5, 12), cleanly isolating the training effect (base 4.9% → SFT 10.3% → SFT+RL 33.2% Corr@3 on TritonBench-G).
Unusually careful claim hygiene
Mechanistically motivated RL design with ablations
Cross-hardware transfer check
Weaknesses
No statistical uncertainty anywhere
Semantic train/test overlap unquantified
Distillation confound in 'beats GPT-5' results
Incomplete corpus accounting and small protocol gaps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
AMDKernelVault (arXiv 2609.12471, cs.CL) is a resource paper from AMD: an open HIP/Triton kernel corpus for recent CDNA GPUs plus a training demonstration on Qwen3-8B. Three claims are made. First, the largest execution-verified AMD kernel corpus: 62,153 HIP samples (HIP-CudaAgent 5,388; HIP-GPUMode 5,910 tasks / 22,397 variants; HIP2HIP 34,368), 2,377 rocBLAS/rocSOLVER QA entries, and 39,893 Triton kernels from four sources. Second, under identical 3-iteration GEAK OptimAgent-v2 budgets, the SFT+RL model attains the highest correctness among compared models on PyTorch-to-HIP (34.0% Pass@1, 200 KernelBench L1/2 tasks), TritonBench-G (33.2% Corr@3, 184 kernels), and ROCmBench (41.94% Corr@3, 31 kernels). Third, an explicit set of non-claims: the recipe is a corpus-utility test, not a new algorithm, and the model "does not uniformly lead compilation or speed" — Claude Sonnet 4 keeps Comp@3 (67.74% vs 58.06%) and speed (1.82× vs 1.61×) on ROCmBench, and GPT-5 keeps HIP-to-HIP correctness (87% vs 84%). This claim hygiene is notably better than typical in the area.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 4 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is AMDKernelVault?
An open dataset and pipeline from AMD for GPU kernel generation on AMD Instinct hardware: agent loops convert PyTorch code into HIP or Triton kernels, compile them under ROCm, verify correctness by execution, and measure latency. It ships 62,153 verified HIP samples, 2,377 QA entries mined from rocBLAS/rocSOLVER, and 39,893 Triton kernels.
Does the trained Qwen3-8B beat GPT-5?
On correctness metrics under the same 3-iteration agent budget: 34.0% Pass@1 on PyTorch-to-HIP, 33.2% Corr@3 on TritonBench-G, and 41.94% Corr@3 on ROCmBench, versus GPT-5's 24.0%, 15.2%, and 29.03%. It does not dominate everywhere: Claude Sonnet 4 compiles more and runs faster on ROCmBench, and GPT-5 keeps the lead on HIP-to-HIP correctness (87% vs 84%).
Where can I download the dataset and code?
The corpus and documentation are on Hugging Face at huggingface.co/datasets/amd/AIG-Datasets, and the training and kernel-generation code is at github.com/AMD-AGI/hip_kernel_llm_lab, with per-source license and provenance notes (e.g., rocBLAS MIT, rocSOLVER BSD-2-Clause).
Can I use AMDKernelVault on NVIDIA GPUs?
No. The corpus targets AMD CDNA GPUs — gfx942 (MI325X) and gfx950 (MI350/MI355X) — and all validation was done under ROCm; the paper notes results depend on the specific ROCm, Triton 3.3.0, and hardware configurations used.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–3 appear only as captions and axis labels in the extraction; curve values (e.g., the Figure 2 training-reward trajectory) cannot be read from the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.