GUIDE: Generative Utility Inference and Decision Engine
Anagha Tiwari, Alexander G. Gray, Nick Feamster, Brian Jabarian, Alex Imas, Alex Kale
cs.LG
Quick verdict
GUIDE joins LLM dialogue, Bayesian particle filtering, and symbolic domain priors; in silico only, strong early turns, late-stage rebound, no uncertainty reported.
Summary
GUIDE is an LLM-driven preference-elicitation architecture that maintains a particle-filter posterior over user preference-dimension weights and selects questions across heterogeneous types (pairwise, rule elicitation, dimension dominance, dimension proposal, free-text) by expected information gain, with adaptive corrections for particle degeneracy and transform overuse. An offline pipeline mines the NFCS Investor Survey with ILASP to produce population rules, dimension parameterizations, and per-dimension priors, injected only after a developer-reviewed steering checkpoint. In silico experiments on portfolio optimization use 75 LLM-generated portfolios and five Claude Sonnet 5-simulated personas with hidden linear utilities, 50 runs of 15 turns, against four LLM baselines, three ablations, OPEN, and PEBOL. GUIDE starts at mean regret 0.278 versus 0.678 (OPEN) and 0.651 (PEBOL), reaches 0.038 by turn 5 (over 90% lower), and posts the highest normalized regret reduction on all five personas. However, beyond turn 5 full GUIDE's regret rebounds to 0.081 at turn 15, below the pairwise ablation's 0.027, an instability the authors attribute to transform competition. Main limitations: purely simulated evaluation, linear ground-truth utilities matching the model class, a single domain, no variance or significance reporting, and a backbone mismatch (GUIDE uses Claude Opus 4.7; OPEN/PEBOL use GPT-5.4-mini). No code release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Diverse elicitation question types can be unified on a single particle-filter posterior with EIG as the common selection currency, yielding faster and more consistent early-turn convergence than pairwise-only elicitation.
- Symbolic domain initialization from population survey data (NFCS mined via ILASP, developer-approved) cuts cold-start regret from 0.678/0.651 (OPEN/PEBOL) to 0.278 at turn 1.
- GUIDE achieves the highest normalized regret reduction on all five personas and beats its strongest ablation on four of five.
- Late-stage regret rebound after turn 5 is an inherent trade-off of diverse transform competition, which the architecture's observability features can diagnose.
- The free-text→quasi-likelihood transform is claimed to be a novel form of parametric Bayesian inference driven by natural language.
Strengths
Heterogeneous question types unified on one posterior
All five transforms (pairwise Bradley-Terry T=0.5, rule elicitation, dimension dominance, free-text quasi-likelihood, dimension proposal) share a single additive log-likelihood update on an N=200 particle posterior, and compete on one EIG scale (Supplementals C.1) — a clean architectural answer to OPEN's pairwise-only limitation.
Empirically grounded, auditable domain initialization
The ILASP pipeline mines segment→dimension rules from the real NFCS Investor Survey (deviation ≥10 points on a 0-100 scale, ≥5% support), and every rule passes a validated developer steering checkpoint before reaching a live session (§ Offline Domain Initialization; § Developer Steering Checkpoint) — directly addressing OPEN's LLM-confabulated ontology weakness.
Honest reporting of a negative trade-off
Careful evaluation fairness controls
Actionable observability (R3)
Weaknesses
No uncertainty quantification anywhere
Evaluation circularity favoring the model class
LLM backbone confound
Thin external validity
Heavy untuned hyperparameter load
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
GUIDE (arXiv 2609.12137, cs.LG; University of Chicago / CMU) is a method paper: an LLM-assisted preference-elicitation (PE) architecture organized around three requirements — expressive querying (R1), empirical grounding (R2), and observability (R3). Its central claims are that (1) heterogeneous question types unified on a single particle-filter posterior converge faster than pairwise-only elicitation, (2) offline symbolic initialization from population survey data (NFCS Investor Survey, mined with ILASP) slashes cold-start regret — 0.278 mean regret at turn 1 versus 0.678 (OPEN) and 0.651 (PEBOL), and (3) GUIDE attains the highest normalized regret reduction (NRR) on all five personas, beating the strongest ablation on four of five. The scope is explicitly in silico: portfolio optimization only, five LLM-simulated personas, no human subjects (human evaluation is deferred to future work).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is GUIDE and what problem does it solve?
GUIDE (Generative Utility Inference and Decision Engine) is a research system from the University of Chicago and Carnegie Mellon that infers a user's preferences through conversation. It combines an LLM dialogue interface with a Bayesian particle filter over preference-dimension weights, asks many question types (pairwise choices, rules, dimension comparisons, free text) selected by expected information gain, and seeds its initial beliefs from rules mined from a real investor survey (FINRA's NFCS), with a human developer reviewing every mined rule.
How much better is GUIDE than earlier methods like OPEN or PEBOL?
In the paper's simulated portfolio experiments, GUIDE's first-turn average regret is 0.278 versus 0.678 for OPEN and 0.651 for PEBOL; by turn 5 GUIDE reaches 0.038, over 90% lower than both. Most of this early advantage comes from its data-grounded initialization. After turn 5, however, full GUIDE's regret rebounds (0.081 at turn 15), ending worse than its own pairwise-only ablation (0.027), a trade-off the authors report openly.
Was GUIDE tested with real humans?
No. All results come from simulations in which five LLM-based investor personas (played by Claude Sonnet 5) with hidden linear utility functions answer the elicitation questions. The authors list human-subject evaluation across domains as future work.
Is GUIDE's code, data, or the portfolio catalog available?
The paper text does not state any code or data release. The underlying population data (FINRA's NFCS Investor Survey) is publicly available, but the mined rules, persona definitions, and the 75-portfolio catalog generated from Fidelity and Vanguard data are not published according to the text.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-4 appear only as captions in the extracted text; the underlying regret curves (Figures 2-3) and the observability distributions (Figure 4) could not be inspected.
- Table 2's header labels are partially garbled by extraction; the six numeric columns were mapped to methods by their listed order, so misalignment is possible, though the pattern matches the text's stated claims (best on 5/5 vs prior work, 4/5 vs strongest ablation).
- The main text references Supplemental section G.1, but no section labeled G appears in the provided text (Table 3 may correspond to it).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.