HEAT: Faster Fully Homomorphic Inference via Approximations-Weights Co-Adaptation
Alessandro Zirilli, Davide Marincione, Evgenios M. Kornaropoulos, Giuseppe Ateniese, Emanuele Rodolà
cs.CR · cs.AI
Quick verdict
Claims 3.1x fewer iterations & 1.4x faster encrypted GPT-2 decoding via learnable per-site FHE approximation depth; well-evaluated but single-model evidence.
Summary
The paper attacks the latency of fully homomorphic encryption (FHE) inference for language models. Under CKKS, nonlinearities (softmax, layer normalization) must be approximated by iterative solvers (Goldschmidt, Newton-Raphson); each iteration consumes multiplicative depth and triggers costly bootstrapping. Existing systems fix iteration counts uniformly, calibrated to worst-case precision. HEAT makes per-site iteration counts learnable: each approximation site carries a halting distribution (in the style of PonderNet), the network outputs the expected approximation over solver states, and a KL penalty toward a negative-binomial prior pushes counts down while the task loss preserves precision; weights fine-tune jointly so they absorb residual approximation error. After training, each site is frozen to its distribution's mode, keeping execution input-independent (data-dependent depth would leak through latency). Fine-tuning GPT-2 (124M) on OpenWebText takes under seven hours on one A100-64GB (~1.5×10^8 tokens). On encrypted decoding with a FIDESlib/CKKS backend at 128-bit security, iterations per forward drop 712→228 (3.1×), bootstraps per token 513→326 (1.6×), and latency 54.3→37.6 s/token (1.4×); top-1 agreement with the plaintext parent rises from 74.0% to 82.9%. Transplanting learned counts onto other checkpoints breaks them, supporting genuine co-adaptation. Main limitations: evidence covers a single 124M backbone, one modality, and one cryptographic parameterization; the EncryptedLLM comparison uses different hardware.
Review complete
Assessment at a glance
3
risk flags identified
● high 0
● medium 2
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Making per-nonlinearity FHE iteration counts learnable and co-adapted with the weights cuts executed iterations 3.1× (712→228 per forward pass) on encrypted GPT-2.
- The co-adapted circuit reduces bootstraps per token 1.6× (513→326) and end-to-end decode latency 1.4× (54.3→37.6 s/tok) versus a precision-calibrated baseline.
- The shallower HEAT circuit is more faithful to its plaintext parent (pooled top-1 agreement 82.9% vs 74.0%; final-position KL 6.6× lower) than the calibrated baseline is to its own.
- Learned iteration counts are checkpoint-specific: transplanted onto squeezed or pretrained weights they break inference, demonstrating genuine weights-approximations co-adaptation.
- The fine-tuning budget is modest: ~1.5×10^8 tokens (under 2% of the pretraining corpus), under seven hours on a single A100-64GB.
Strengths
Novel training-side lever on a real systems bottleneck
Reframes PonderNet-style halting distributions (Banino et al.) as a differentiable relaxation for per-site FHE approximation depth, and freezes to per-site modes because input-dependent depth would leak via latency (Section 3, Appendix A) — a technically sound adaptation rather than a naive port of adaptive computation.
End-to-end encrypted measurements, not simulations
All latencies come from actual encrypted runs on a FIDESlib/CKKS backend at 128-bit security; Appendix C publishes the full parameter set (N=2^16, 53-bit rescaling primes, 28-limb chain, depth 27, dnum=7) and Table 2 times each CKKS primitive (bootstrap 82.1 ms vs ≤0.58 ms arithmetic), making the bootstrap-dominance argument checkable.
Clean ablation isolation
Fidelity is measured, not asserted
Fine-grained per-site analysis
Weaknesses
Single model, single parameterization
Cross-hardware baseline comparison
Fidelity proxy is self-referential
Teacher-forced evaluation only
Key hyperparameters absent from the text
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
HEAT addresses the dominant cost of non-interactive FHE inference for autoregressive language models: under CKKS, every nonlinearity (Softmax, LayerNorm) is approximated by an iterative solver (Goldschmidt, Newton-Raphson) whose iterations consume multiplicative depth and force expensive bootstrapping (82.1 ms per refresh vs ≤0.58 ms for any arithmetic primitive, Table 2). The paper's central claim is that the per-site iteration count — previously fixed by worst-case calibration — can be made a learnable quantity, co-adapted with the weights during fine-tuning. On encrypted GPT-2 (124M) decoding this yields 3.1× fewer iterations (712→228 per forward), 1.6× fewer bootstraps (513→326 per token), and 1.4× lower end-to-end latency (54.3→37.6 s/tok), while improving fidelity to the plaintext parent (pooled top-1 agreement 74.0%→82.9%). The claimed scope is deliberately narrow — one backbone, one modality, one cryptographic parameterization (Section 5, Limitations) — and the paper is consistent about it.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 3 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is HEAT and how much faster does it make encrypted GPT-2 inference?
HEAT (Homomorphic Encryption-Aware Training) is a fine-tuning method that lets the iteration count of each FHE nonlinearity approximation be learned jointly with the model weights. On encrypted GPT-2 decoding it cuts solver iterations per forward pass from 712 to 228 (3.1×), bootstraps per token from 513 to 326 (1.6×), and latency from 54.3 to 37.6 seconds per token (1.4×), while improving top-1 agreement with the plaintext model from 74.0% to 82.9%.
Why do iteration counts matter so much in fully homomorphic encryption?
Under CKKS FHE, ciphertexts natively support only addition, multiplication, and rotation. Nonlinear functions like softmax and layer normalization must be approximated by iterative methods (Goldschmidt, Newton-Raphson), and every iteration costs multiplications. Multiplications are budgeted by a finite multiplicative depth; exhausting it forces a bootstrapping refresh that costs two to three orders of magnitude more than any other primitive (82.1 ms vs ≤0.58 ms in the paper's Table 2). Fewer iterations therefore mean fewer bootstraps and lower latency.
Does reducing approximation iterations hurt the model's accuracy?
In this paper, no — the opposite. Because the weights are fine-tuned under the same approximations they will face at inference, the shallower HEAT circuit tracks its plaintext parent more closely than the calibrated baseline does: pooled top-1 agreement 82.9% vs 74.0%, and final-position KL divergence 6.6× lower. Caveat: this was measured on one 124M model with teacher-forced decoding, against each model's own plaintext parent.
Is the code or the trained model available?
The paper links a GitHub repository (gladia-research-group/heat) and states that HEAT-trained checkpoints are published on HuggingFace; Appendix C also gives the complete CKKS parameter set. This review did not verify the links (offline review), so availability of the exact modified FIDESlib build is unconfirmed.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure content (Figures 1-5) is only partially recovered from embedded text fragments; plot-level readings rely on the in-text numbers, not the rendered graphics.
- The linked GitHub/HuggingFace resources could not be verified in this offline review; availability claims are as stated in the paper text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.