CodeTS: Verifiable Text-to-Time Series Generation via Executable Code
Xudong Yuan, Shunyu Liu, Tongya Zheng, Huiping Zhuang, Mingli Song, Kaixuan Chen
cs.LG · cs.AI
Quick verdict
CodeTS turns Text-to-TS into Text-to-Code-to-TS with SFT+RLVR; benchmark wins, but auto-derived descriptions embed target statistics, favoring the method.
Summary
CodeTS reframes text-to-time-series (Text-to-TS) generation as a Text-to-Code-to-TS pipeline: a Qwen2.5-Coder-7B-Instruct backbone translates a natural-language description into a normalized JSON program (structured params plus a generate_ts function), and sandboxed execution yields the series. Training proceeds in two stages: supervised fine-tuning on 500 synthetic Text-Code-TS triplets built rule-based from structured temporal attributes, followed by RLVR with GRPO on 6,300 real Text-TS pairs, using execution-based rewards covering format, executability, length, pointwise error, correlation, and variance. Across eight benchmarks (ETTh1/h2, ETTm1/m2, Weather, Electricity, Exchange, Web) at lengths 96/192/336, zero-shot CodeTS reports averaged MSE 0.0321, DTW 0.0938, and Pearson 0.6808, surpassing supervised baselines trained per dataset (best averages 0.0733, 0.1675, 0.2279) and all zero-shot LLM baselines, with near-100% pass@1 execution success. Ablations attribute the dominant gains to RLVR, and template-perturbation tests reveal residual brittleness to rephrased descriptions. Major caveats: evaluation descriptions are auto-extracted from the target windows themselves and contain exact statistics (mean, std, trend endpoints, period, phase, event positions) that the generated code consumes directly (e.g., target_mean/target_std rescaling), so the task largely reduces to inverting a deterministic attribute pipeline; RLVR directly optimizes the reported metrics; and no error bars or significance tests are reported. Scope is univariate; code is in an anonymized repository.
Review complete
Assessment at a glance
5
risk flags identified
● high 2
● medium 3
● low 0
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Text-to-TS generation can be reformulated as Text-to-Code-to-TS, with executable code as an explicit, inspectable intermediate interface (Sec. 4.1).
- Supervised initialization on 500 rule-built synthetic triplets plus GRPO-RLVR on 6,300 real Text-TS pairs yields a zero-shot generator with ~100% pass@1 execution success (Sec. 4.4, Table 2).
- Zero-shot CodeTS beats per-dataset-trained supervised baselines on averaged MSE/DTW/Pearson (0.0321/0.0938/0.6808 vs best 0.0733/0.1675/0.2279; Table 1) and all zero-shot LLM baselines, reaching 11.9x the strongest baseline Pearson (Sec. 5.2, Table 2).
- The gains stem from the normalized code interface and execution-based reward optimization rather than code-generation ability alone (Sec. 5.3, Appendix F Table 6).
Strengths
Verifiable-by-execution design with fully disclosed rewards
The normalized JSON interface (params + generate_ts, Appendix B) plus staged rewards with exact weights (λfmt=0.05, λexec=0.10, λlen=0.10, λerr=0.30, λcorr=0.30, λstat=0.15; Appendix C, Eqs. 17-22) makes every training signal inspectable and reproducible — a cleaner setup than latent Text-to-TS generators.
Broad, multi-pronged evaluation
Eight benchmarks x three lengths (Tables 1-2) against supervised generators (Diffusion-TS, T2S, VerbalTS, TimeVQVAE), zero-shot LLMs (TimeOmni-1, ChatTime, GPT-4o-mini), and four recent coding LLMs under the identical Text-to-Code-to-TS prompt (Table 6), with ~100% pass@1 SR.
Thorough ablation and robustness program
Strong headline numbers with honest edge-case reporting
Reproducibility-oriented disclosure
Weaknesses
Benchmark circularity: descriptions embed the answer
Missing achievable-ceiling baseline
Train/test objective alignment favors the method
No uncertainty quantification
Ablation claim conflicts with its own Table 8
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
CodeTS (arXiv:2609.15393; Zhejiang Univ./NTU/SCUT) targets Text-to-Time Series generation: given only a natural-language description, synthesize a univariate series of length 96, 192, or 336. The central move is reformulating generation as Text-to-Code-to-TS (Sec. 4.1): a Qwen2.5-Coder-7B-Instruct policy emits a normalized JSON program (structured params plus a generate_ts function, Appendix B), and sandboxed execution produces the series. Claims: (1) first Text-to-Code-to-TS formulation; (2) explicit, verifiable, optimizable generation via execution-based rewards; (3) rule-built Text-Code-TS triplets (500) for SFT plus multi-stage rewards for RLVR on 6,300 real pairs; (4) zero-shot results beating per-dataset-trained supervised baselines and all zero-shot LLM baselines on eight benchmarks. Notably, Appendix D.1 itself concedes the benchmark is a constructed 'text-conditioned synthesis' setting — a scoping fact essential for judging every headline number.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is CodeTS and how does it generate a time series from text?
CodeTS is a framework that turns a natural-language description of a time series into a normalized JSON program — structured parameters plus a Python generate_ts function — and then runs that code in a sandbox to produce the series. It is built on Qwen2.5-Coder-7B-Instruct and trained in two stages: supervised fine-tuning on 500 synthetic text-code-series triplets, then reinforcement learning (GRPO) on 6,300 real text-series pairs with execution-based rewards.
Does CodeTS beat GPT-4o and other LLM baselines?
In the paper's experiments on eight benchmarks, zero-shot CodeTS reports average MSE 0.0321 versus 0.2064 for the strongest zero-shot LLM baseline (ChatTime) and 0.2546 for GPT-4o-mini, with near-100% execution success versus 69-91% for the LLM baselines. It also outperforms four recent coding LLMs prompted with the same code-generation setup (Appendix F, Table 6).
Is the code or data for CodeTS available?
The paper links an anonymized repository (anonymous.4open.science/r/CodeTS-54DA) and documents public data sources for training (18 sources such as MIT-BIH, Binance, and NYC Taxi) and evaluation (eight public time-series benchmarks). The repository's completeness cannot be verified from the paper text alone.
What are the main limitations of CodeTS?
It handles only univariate series and is sensitive to how descriptions are phrased — under reworded descriptions, ETTh1-96 Pearson drops from 0.8136 to 0.6578. A deeper reviewer concern is that evaluation descriptions are auto-extracted from the target windows and include exact statistics the generated code directly consumes, so the reported gains may partly reflect inverting a deterministic extraction pipeline rather than general text-to-series understanding.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Several passages were truncated in extraction (parts of Sec. 5.1, Appendices G.2, H, and checklist Q8); some training-pipeline and compute details are unavailable.
- Figures 1-8 could not be analyzed as images; numeric claims rely on the tables, which appear intact.
- The anonymized code repository's contents could not be inspected.
- arXiv v1 (14 Sep 2026); no peer-review verdict exists yet.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.