Variational Quantum Transformer Architecture for Synthetic Language Generation

Summary

The paper proposes a hybrid quantum-classical autoregressive sequence model that preserves the transformer's next-token interface while replacing attention and feed-forward sublayers with variational quantum circuits. A four-token context is angle-encoded into four data qubits; E encoder blocks (two parallel variational heads plus an integration circuit, each reduced to two qubits) process the context, D decoder blocks condition generation via CNOT connector circuits onto two ancilla qubits with a residual path from the original context state, and the diagonal of the final two-qubit density matrix yields the four-token probabilities. Four variants (124-364 parameters, maximum six active qubits) are evaluated in exact PennyLane simulation on two synthetic four-token grammars - a deterministic cycle (AAA BBB CCC)* and a lexicographic language of nondecreasing three-character words - against a 700-parameter classical transformer over three seeds. The classical baseline solves the deterministic task perfectly and achieves a perfect grammar score on the lexicographic task; the best quantum variants reach 0.755±0.213 (deterministic) and 0.828±0.299 (lexicographic) grammar scores, with perfect deterministic generation in single seeds. The authors explicitly disclaim quantum advantage, framing the contribution as a feasible NISQ-scale architecture. Main limitations: only three seeds with very large variances (token accuracy SD up to ±0.384), no parameter-matched classical control, minimal synthetic tasks, exact simulation only, and a partially specified grammar score susceptible to degenerate outputs such as repeated valid words.