Multitask Reinforcement Learning for Assisting Choice Model Specification
Gabriel Nova, Stephane Hess, Sander Van Cranenburgh
econ.GN · cs.AI
Quick verdict
Delphos reframes discrete choice model specification as multitask RL with a DeepSet-Q set representation; results are promising and arithmetically consistent, but validation rests on only two held-out datasets, single runs, and favourable baseline comparisons.
Summary
Delphos is a multitask deep reinforcement learning assistant for discrete choice model (MNL) specification. Specification is an MDP: the agent adds or changes utility terms (attribute, transformation, taste structure, covariate interaction) and terminates, receiving a reward equal to tanh of per-observation log-likelihood gain over the null model, with a fixed penalty for non-convergence; Apollo estimates candidates. A DeepSet-Q network encodes specifications as unordered term sets plus a dataset context vector, letting one policy span datasets with different variables. Trained jointly on nine transport datasets, the multitask agent beats independently trained single-task DQNs on most mean/max reward, AUC, and convergence-rate metrics (Table 2). Without retraining, within 20 CPU-minutes it finds a Swissmetro specification with LL/N of -0.68 versus -0.72 for the VNS metaheuristic of Ortelli et al. (2021), and on Decisions reaches -0.37 versus -0.36 for an expert MNL (Tsoleridis et al., 2022). Main limitations: behavioural plausibility is not actually encoded in the evaluated reward; transfer is validated on only two held-out datasets; results are single runs without seed variance or significance testing; and there is no random-search or LLM baseline within the same restricted MNL catalogue, so the source of the gains is hard to attribute.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Training a single DeepSet-Q agent jointly on nine transport choice datasets yields specification policies that outperform independently trained single-task DQN agents in reward, learning speed (AUC), and convergence rate.
- The learned policy transfers without any further training to unseen datasets (Swissmetro, Decisions), producing competitive specifications in under 20 minutes on a standard CPU.
- On Swissmetro the transferred agent attains a higher log-likelihood per observation (-0.68) than the VNS assisted-specification metaheuristic (-0.72; Ortelli et al., 2021).
- On Decisions the agent's specification (-0.37 LL/N) is comparable to an expert-developed published MNL specification (-0.36; Tsoleridis et al., 2022).
- Representing utility specifications as unordered sets of modelling terms via a DeepSet-Q architecture, conditioned on a dataset-specific context vector, is sufficient to share a specification policy across datasets with different variables.
Strengths
Well-motivated set-based representation enabling genuine cross-dataset transfer
Sections 3.1-3.2 encode each utility specification as an unordered set of (attribute, transformation, taste structure, covariate) terms (Eq. 4-5) processed by a DeepSet encoder with permutation-invariant pooling (Eq. 8), plus a task context vector (Eq. 9). This cleanly separates behavioural decisions from dataset-specific variable labels, which is the paper's core technical contribution over the fixed-length single-task Delphos of Nova et al. (2025).
Consistent multitask gains across most training datasets
Table 2 shows the DeepSet-Q agent beating the single-task DQN on mean reward, max reward, and AUC in 8 of 9 datasets (e.g., SwissmetroRouteChoice AUC 761 vs 291; LondonModeChoice -716 vs -2086), while exploring fewer novel specifications and achieving higher proportions of estimable specifications (e.g., 88.52% vs 84.62%), supporting the sample-efficiency claim rather than mere exploration broadening.
Headline inference results are arithmetically consistent and practically relevant
Honest experimental accounting and clear task setup
Weaknesses
Behavioural plausibility is central to the motivation but absent from the evaluated objective
No variance, seeds, or significance testing for any RL comparison
Benchmark comparisons are potentially apples-to-oranges
Missing controls: no random search within the same catalogue and no LLM baseline
Transfer evidence rests on two held-out datasets whose catalogues largely overlap training
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Delphos and what does it do?
Delphos is a reinforcement learning assistant for discrete choice model specification in transport research. Given a dataset and a catalogue of allowed modelling components (attributes, transformations, taste structures, socio-demographic interactions), it builds candidate multinomial logit utility functions step by step, estimates them with the Apollo software, and returns well-fitting specifications from a Pareto front for the modeller to review. The modeller keeps final control over diagnosis and selection.
How does Delphos compare to the VNS method for assisted specification?
On the Swissmetro dataset, Delphos reached a log-likelihood of -0.68 per observation versus -0.72 reported for the VNS metaheuristic of Ortelli et al. (2021), using about 20 minutes on a standard laptop CPU. Note the comparison is on final model fit only; the paper does not match the two methods' search spaces, model classes, or parameter counts in detail.
Can the trained agent handle datasets it was never trained on?
According to the paper, yes: after joint training on nine transport choice datasets, the same agent was applied without retraining to the unseen Swissmetro and Decisions datasets. It encodes utility specifications as sets of generic modelling terms (e.g., 'travel time with a log transform and alternative-specific coefficient') plus a context vector describing which variables each dataset offers, so the policy can carry over across datasets with different variables.
Does Delphos replace human choice modellers?
No. It is positioned as an assistant: it automates the trial-and-error search for promising utility specifications and presents Pareto-optimal candidates, but the modeller still defines the catalogue, checks behavioural plausibility and parameter significance, and makes the final model choice. Its reward currently reflects goodness-of-fit and convergence rather than explicit behavioural criteria.
What are the main limitations of the Delphos paper?
The main self-acknowledged limitations are that it was trained on only nine datasets and validated on just two unseen ones; the reward function covers fit and convergence but not behavioural plausibility; and the policy is fixed at inference without adaptation to the new dataset. Reviewers would add: results come from single training runs without variance estimates, there is no random-search baseline in the same restricted specification space, and no code release is stated.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-5 are not analyzable from the extracted text; learning-curve shape, Pareto-front spread, and figure-level detail could not be assessed.
- The abstract in the embedded metadata is truncated mid-sentence ('...expert modellers on Decis'), but the body and conclusion contain the complete claim, so no judgment was affected.
- The embedded text contains extraction artefacts (concatenated reference fields, 'vs 2m', garbled PDF URL in Table 1); these are copy artifacts, not paper defects, and do not affect the review.
- No supplementary material or code repository was available; reproducibility assessment is based solely on the manuscript's stated hyperparameters (Appendix Table 5).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.