Recent Developments in Transformer Inference Deployment on FPGA Platforms: A Survey
Arjan Blankestijn, Uraz Odyurt, Amirreza Yousefzadeh
cs.LG · cs.AR
Quick verdict
Systematic 2024–25 survey of Transformer inference on FPGAs (57 papers): transparent method, useful taxonomy, but heterogeneous averages weaken conclusions.
Summary
This survey systematically reviews Transformer inference deployment on FPGA platforms, covering 2024 through May 2025. The authors queried IEEE Xplore, Scopus, and arXiv (411 hits), pruned by duplication, date, abstract keywords, survey removal, and training focus (to 96), then ranked by keyword frequency to select 57 primary articles. They derive a taxonomy spanning implementation (on/off-chip weight storage, RTL vs HLS tooling) and optimisation (quantisation, pruning, PE/systolic design, pipelining, SoftMax restructuring), with comparative tables of resources, throughput, and power. Reported tendencies include: on-chip weight storage correlating with lower average latency and higher throughput (Table 3); RTL designs averaging roughly half the power of HLS designs but lower throughput (Table 5); binary/LUT-based quantised designs dominating energy-efficiency rankings (Tables 7–8); and a small Pareto frontier balancing throughput and GOPS/W (Figure 6). The paper candidly notes that cross-article comparison is undermined by inconsistent baselines, model variants, and reporting, and calls for standardised benchmark models (Section 9.5). Limitations: averages aggregate heterogeneous platforms, models, and workloads; the on-chip/off-chip categorisation is acknowledged as unreliable (ProTEA is described as on-chip in Section 5.2 yet listed off-chip in Table 3); several 2023 works appear in comparison tables despite the 2024+ scope; and the 25-point ranking threshold lacks sensitivity analysis.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Utilising persistent on-chip weight storage generally results in much lower inference latency and higher throughput than off-chip access during inference (Section 5.3, Table 3 averages: 5.64 ms / 4,688.79 GOPS vs 49.92 ms / 1,453.66 GOPS).
- RTL-based designs occupy the lower end of the power spectrum, averaging about half the power of HLS designs (13 W vs 26.885 W, Table 5), while HLS designs show higher average throughput (2,934 vs 2,093 GOPS).
- Extreme quantisation dominates energy efficiency: binary-weight designs [43] (679.76 GOPS/W) and COBRA [38] (448.70 GOPS/W) lead Table 8, and 5 of the top-10 efficiency entries use RTL versus ~30% RTL share overall.
- Fair cross-article comparison of FPGA Transformer accelerators is currently infeasible without standardised benchmark models and reporting practices, which the community should adopt (Section 9.5).
- No design dimension (HLS vs RTL, on- vs off-chip storage, pipelining, quantisation scheme) has an absolute preference; the flexibility-vs-peak-performance trade-off is inherent and application-driven (Conclusion).
Strengths
Reproducible, documented search protocol
Section 3 and Figure 3 give per-step article counts (411 → 335 → 161 → 113 → 108 → 96 → 57), and the Python pruning scripts are publicly archived on Zenodo ([19], DOI 10.5281/zenodo.20185395).
Comprehensive comparative aggregation
Tables 3, 5, 7 and 8 consolidate resource usage, throughput (GOPS) and power across ~40 accelerators; the Pareto frontier analysis (Figure 6) over Tables 7–8 is a genuinely useful synthesis device for a field without standardized benchmarks.
Honest epistemic hedging
Actionable community recommendation
Focused recency window
Weaknesses
Cross-article averages over heterogeneous setups
Unreliable categorization feeding headline conclusions
Scope violations in the evidence pool
Fragile selection heuristics
Accuracy analysis inconclusive by its own admission
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This survey (arXiv:2609.01212, cs.LG/cs.AR) reviews Transformer *inference* deployment on FPGA platforms, restricted to 2024–May 2025 publications across IEEE Xplore, Scopus, and arXiv. From 411 query hits, a documented pruning chain (Figure 3) reduces the pool to 96 articles; a keyword-frequency ranking with a 25-point threshold then selects 57 primary articles. The stated goals (Section 1) are to identify performance indicators, improvement techniques, implementation/resource-management levers, and available tooling with limitations. The paper claims: (i) a reproducible, expandable review protocol; (ii) a taxonomy (Figure 4) splitting the field into Implementation (weight storage, tooling/design level) and Optimisation (compression, computation/SoftMax, PE design, pipelining, ISA); and (iii) trend-level comparisons on throughput (GOPS) and energy efficiency (GOPS/W). Its headline conclusions are that no design dimension (HLS vs RTL, on- vs off-chip, pipelining, quantisation scheme) holds an absolute preference, and that standardized benchmarking is the key missing enabler (Section 9.5).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What time window and sources does this FPGA Transformer survey cover?
It covers articles published in 2024 and January–May 2025 on Transformer inference on FPGAs, collected from IEEE Xplore, Scopus and arXiv. From 411 initial query hits, a documented pruning chain (duplicates, date filter, abstract keyword, survey removal, training-focus removal) leaves 96 articles, and keyword-frequency ranking selects 57 primary articles.
What is the main practical takeaway for FPGA accelerator designers?
Storing all weights on-chip tends to give the lowest latency and highest throughput, extreme quantisation (binary or 3-bit LUT-based MAC) leads the energy-efficiency rankings, and RTL designs average roughly half the power of HLS designs. The authors stress these are broad tendencies, not rules, because studies use different models, FPGA platforms, precisions and metrics.
Does the survey publish code or data?
Yes — the Python scripts used to collect and prune articles are archived on Zenodo (DOI 10.5281/zenodo.20185395, cited as reference [19]). The surveyed accelerator implementations themselves belong to the original papers and are not redistributed.
Why can't the survey directly rank accelerator performance?
Because each article evaluates different Transformer variants (e.g., DeiT-tiny vs BERT-base), different FPGA chips, different bit-widths and different metrics, reported figures reflect model complexity as much as design quality. The authors therefore propose standardized benchmark models and reporting practices (Section 9.5) as a prerequisite for fair rankings, and offer only top-10 lists and a Pareto frontier as cautious comparisons.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1 and 3–7 appear in the extraction only as flattened text fragments (axis labels, counts, legend names); figure-level details such as exact Pareto-frontier membership beyond the named designs were read from the adjacent tables and prose, not from the plotted graphics.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.