arXiv paper· machine learning / databases (relational deep learning, domain-specific languages)· method paper (tool/DSL)

Relational Task Generation Language: A Declarative Specification Framework for Relational Deep Learning

Oleksii Kolesnichenko, Jakub Peleška, Gustav Šír

cs.PL · cs.DB · cs.LG

Quick verdict

RTGL is an open-source PQL-style DSL for leakage-safe RDL task generation; useful tool, credible RelBench bug findings, but downstream evaluation is shallow.

Summary

The paper introduces RTGL, an open-source declarative domain-specific language for defining prediction tasks in Relational Deep Learning (RDL). RTGL is modeled on the proprietary Predictive Query Language (PQL) from Kumo.ai but is an independent implementation targeting open ecosystems such as RelBench and ReDeLEx. The pipeline parses RTGL queries with ANTLR4 into an AST, performs schema-aware semantic validation (including strict temporal checks separating past observation from future prediction windows), and compiles queries into temporally bounded SQL using CTEs. The authors reconstruct several RelBench tasks on rel-f1 and rel-stack and report that the comparison exposed genuine temporal-leakage defects in the official manual SQL: driver-dnf filtered drivers using future activity, and user-engagement/user-post-comment omitted registration-date filters, so baselines included users not yet registered. They also define new binary, multiclass, multilabel, and regression tasks on the enterprise-scale ctu-seznam (Seznam) dataset from ReDeLEx, and train GraphSAGE and HGT models on a selection of tasks, reporting e.g. test accuracy .7219±.0123 (SAGE) on driver-dnf and test AUPRC .6274±.0017 (SAGE) on client-services. Main limitations: downstream evaluation lacks external baselines and does not quantify how the corrected tables change benchmark scores; the number of random seeds behind the mean±std figures is unstated; multi-hop tasks required intermediate views; and no runtime or performance measurements are provided despite acknowledged scalability limits.

Review complete

Assessment at a glance

machine learning / databases (relational deep learning, domain-specific languages)

5

risk flags identified

● high 0

● medium 2

● low 3

Reproducibility readinesshigh
Code & data availableyes
Review confidencehigh
AI text signal0/20 passages flagged
Deep research viableCode-reproducibleNeeds GPU

5 strengths · 5 weaknesses · 5 author questions

AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.

Levels reflect verification complexity — not a quality score. Evolve never rates your paper.

Key claims under review

  • RTGL is an open-source declarative DSL that generates temporally safe RDL training tables by compiling high-level task specifications into anchor-time-bounded SQL.
  • Reconstructing RelBench tasks in RTGL exposed genuine temporal-logic errors in the official manually written SQL: future-activity filtering in driver-dnf and missing registration-date filters in user-engagement and user-post-comment.
  • RTGL expresses diverse new task types (binary, multiclass, multilabel, regression) on the enterprise-scale ctu-seznam dataset within the ReDeLEx framework.
  • GraphSAGE and HGT models trained on RTGL-generated tables converge across task types, confirming the structural and temporal integrity of the generated data.

Strengths

  • Fills a genuine open-source gap

    PQL (ref. [24], Kumo.ai) is proprietary and platform-locked; RTGL (Sec. 1, Sec. 4) is an independent open reimplementation targeting RelBench/ReDeLEx, released at github.com/kolesole/RTGL with task definitions at github.com/kolesole/rtgl-tasks.

  • Concrete, checkable benchmark-corruption findings

    Sec. 6.2 documents specific temporal-logic bugs in official RelBench SQL — driver-dnf lacks an upper time bound (future driver activity used for filtering, even including not-yet-existing drivers), and user-engagement/user-post-comment omit creation-date filters — verified by row-level joins against reference tables (Sec. 6.1).

  • Sound compiler engineering

  • Breadth of task types across two frameworks

  • Honest scoping of limitations

Weaknesses

  • No external baselines or leakage-impact quantification

  • Thin statistical reporting

  • Partial correctness evaluation

  • No performance/scalability data

  • Simplicity claim not substantiated

Full Review Report

$1.99 · one-time

Unlock the complete preliminary analysis Evolve produced for this paper.

Report preview

Scope & Claims

This is a tool/DSL paper (LNCS layout; cs.PL/cs.DB/cs.LG) introducing RTGL, an open-source declarative language for defining Relational Deep Learning (RDL) prediction tasks, modeled on Kumo.ai's proprietary PQL [24] but implemented independently for open ecosystems (RelBench [38, 17]; ReDeLEx [33]). Four claims: (1) high-level task specifications compile to temporally safe training tables or raw SQL; (2) reconstructing RelBench tasks uncovered genuine temporal-leakage defects in the benchmark's manual SQL; (3) RTGL expresses new binary/multiclass/multilabel/regression tasks on the enterprise-scale ctu-seznam dataset; (4) downstream training confirms the generated tables are structurally and temporally valid. Claims (1) and (3) are convincingly demonstrated; (2) is documented qualitatively but never quantified; (4) rests on convergence evidence only, with no external reference points.

The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.

  • All 5 strengths & 5 weaknesses, analyzed in detail
  • 5 risk flags with severity, section & reviewer questions
  • Methods & statistical evidence assessment
  • 5 questions a peer reviewer would ask
  • Full six-dimension evaluation report & recommendations

One-time purchase · this paper only

One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.

Frequently asked questions

What is RTGL and what problem does it solve?

RTGL (Relational Task Generation Language) is an open-source declarative language for defining prediction tasks on relational databases for Relational Deep Learning. Instead of hand-writing complex, time-aware SQL to build training tables — a process prone to temporal data leakage — users specify what to predict, for which entity, and within which past and future time windows; RTGL compiles that specification into temporally bounded SQL.

How does RTGL relate to Kumo.ai's PQL?

RTGL adopts the syntax and conceptual structure of Kumo.ai's proprietary Predictive Query Language (PQL), but it is an independent, fully open-source implementation designed to integrate with open ecosystems such as RelBench and ReDeLEx, whereas PQL is closed-source and tied to a commercial platform.

What errors did RTGL find in the RelBench benchmark?

While recreating RelBench tasks, the authors found that some official SQL definitions leaked future information: driver-dnf filtered drivers by race activity that had not yet occurred at prediction time, and the user-engagement and user-post-comment tasks did not filter users by registration date, so their training tables included users who had not yet registered.

Is the RTGL code and data publicly available?

Yes. The language implementation is at github.com/kolesole/RTGL and the task definitions and experiment code at github.com/kolesole/rtgl-tasks. The experiments use public RelBench datasets (rel-f1, rel-stack) and the Seznam-based dataset from the open-source ReDeLEx framework.

About this review

This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.

  • The RTGL query examples inside Figures 3-6 are rendered as images; only captions survive text extraction, so syntax-level judgments rest on the Sec. 4 prose rather than the literal figure contents.
  • Table 1's row/column layout is flattened in extraction, though all operator names remain individually readable.

Go deeper on this paper

Wrote a paper? Stress-test it before reviewers do.

Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.

Scan your paper
Relational Task Generation Language: A Declarative Specification Framework for Relational Deep Learning | Evolve.Pub