From LLM-Generated Specifications to Learned Quadruped Locomotion
Merve Atasever, Keyan Azbijari, Cagan Bakirci, Alfredo Reina Corona, Tolga Izdas, Richard Yang, Erdem Biyik, Jyotirmoy V. Deshmukh
cs.RO · cs.AI
Quick verdict
LLM-drafted STL specs + expert-fit thresholds yield competitive sim quadruped rewards; candid analysis, yet human-written specs still win multi-gait settings.
Summary
This paper asks whether large language models can remove the manual-effort bottleneck in Signal Temporal Logic (STL) reward design for quadruped locomotion. GPT-5.5 and Qwen 3.6 propose parametric STL template structures (command tracking, safety, gait) from a natural-language objective and a constrained grammar; numerical thresholds are fit from 50 expert trajectories per gait regime, and specifications whose median expert robustness is negative are discarded. Retained formulas become smooth finite-history robustness rewards for PPO training of a Barkour vb policy in MuJoCo MJX, evaluated in gait-aware (walking-trot/trot/bound) and gait-agnostic settings against heuristic rewards, the authors' manual STL reward, Text2Reward, and an expert-switching oracle. Results are setting-dependent: in the gait-agnostic setting GPT-STL succeeds across all 0.3–2.1 m/s commands while Qwen-STL fails from 1.6 m/s; in multi-gait, Qwen achieves 100% survival and command success at all speeds but matches the prescribed gait only at ≥1.6 m/s, while Text2Reward collapses to 0% survival at ≥1.9 m/s. The authors' hand-written STL remains the most consistent multi-gait controller, and short robustness horizons (H≤10) outperform long ones. The authors candidly note that gait-agnostic winners learn an unnatural bound at low speeds. Main limitations: simulation only, 20 rollouts without statistical tests, and qualitative gait-match labels.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- LLMs (GPT-5.5, Qwen 3.6) can propose PSTL specification structures that, with expert-fit parameters and median-robustness filtering, train competitive quadruped locomotion policies with PPO in simulation.
- In the multi-gait setting, Qwen-MG achieves 100% survival and command success across 0.3–2.1 m/s with the desired bound gait at high speeds (≥1.6 m/s), whereas Text2Reward-MG has 0% survival at ≥1.9 m/s.
- The relative quality of the two LLMs' specifications flips between settings: GPT-5.5 wins gait-agnostic, Qwen 3.6 wins multi-gait.
- Short robustness horizons (H ≤ 10) yield more stable and successful policies than long horizons across most configurations.
- Command-tracking success alone does not guarantee desirable locomotion: the gait-agnostic winners learn an unnatural bound-like pattern at low speeds.
Strengths
Clean division of labor between LLM and data
Section III.A states explicitly that numerical parameters are 'not generated by the LLM'; thresholds are fit from expert-trajectory quantiles and a median-robustness filter (Eq. 12) discards systematically violated proposals — directly neutralizing the known failure mode of LLM-invented reward constants.
Unusually candid negative-result analysis
Section V openly reports that GPT-GA and Text2Reward-GA's 100% tracking success masks an unnatural high-cadence bound at 0.3 m/s, and states plainly that Human-STL — not any LLM variant — is the most consistent multi-gait controller (Table II).
Broad, well-chosen baseline matrix
Interpretable rewards by construction
Weaknesses
Simulation-only validation
Thin statistical and protocol reporting
Abstract oversells relative to Table II
Missing reproducibility details in text
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (arXiv 2609.07111, cs.RO) proposes a pipeline in which LLMs author the *structure* of Parametric Signal Temporal Logic (PSTL) specifications for quadruped locomotion, while expert trajectories supply all numbers. GPT-5.5 and Qwen 3.6, prompted with the same natural-language objective and a constrained STL grammar, independently propose templates for command tracking, safety, and gait structure (Section III.A). Thresholds are fit from 50 expert rollouts (500 steps) per regime via quantiles; a formula is discarded if its median robustness on expert data is negative (Eq. 12). Retained specifications become smooth trailing-window robustness rewards (soft-min aggregation, tanh normalization, Eq. 10) for PPO training of the Barkour vb policy in MuJoCo MJX. Central claims: (1) LLM-proposed specifications are competitive with, and in some regimes superior to, direct LLM reward-code generation (Text2Reward); (2) specification quality is setting-dependent — GPT-5.5 wins gait-agnostic, Qwen 3.6 wins multi-gait; (3) short robustness horizons (H ≤ 10) outperform long ones.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Can LLMs design reward functions for quadruped robot locomotion?
According to this paper, yes in a structured form: GPT-5.5 and Qwen 3.6 proposed Signal Temporal Logic (STL) specification structures whose numerical thresholds were fitted from expert trajectories, and the resulting rewards trained quadruped policies in simulation (MuJoCo MJX) that were competitive with hand-engineered rewards and with the LLM reward-code tool Text2Reward. However, the authors' manually written STL specification still produced the most consistent multi-gait controller, and all results are simulation-only.
Did the paper test the policies on a real robot?
No. All training and evaluation were done in simulation using MuJoCo MJX with the Barkour vb robot model, with domain randomization over friction and actuator gains. The authors explicitly state that sim-to-real transfer on a physical quadruped remains unverified.
How does the STL-specification approach compare with Text2Reward?
In the multi-gait setting, the Qwen-generated STL specifications achieved 100% survival and command success at all tested speeds (0.3–2.1 m/s), while Text2Reward (using GPT-5.5) reached 0% survival at 1.9 m/s and above. In the gait-agnostic setting both tracked commands across the full range, but visual inspection showed both learned an unnatural bound-like gait even at low speeds.
What is parametric Signal Temporal Logic (PSTL)?
A variant of Signal Temporal Logic in which numerical constants (thresholds and time intervals) are parameters rather than fixed values. In this paper the parameters are fitted from expert demonstration trajectories, and the logic's quantitative robustness — how far a trajectory satisfies or violates the formula — is used as a smooth reward signal for reinforcement learning.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The supplied abstract string is cut off mid-sentence ('...hand-engineered rewards, T'), but the corresponding content appears complete in the extracted body text — this is a metadata excerpt limit, not paper truncation.
- Figures (Fig. 1 robot, Fig. 2 pipeline) are referenced but not recoverable from text extraction; the four tables were parsed from flattened text and all values are legible.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.