Odds-Ratio Thompson Sampling: A Specification and Design Guide for Contrast-Based Multi-Armed Bandits
cs.LG
Quick verdict
OR-TS carries log-odds contrasts and refits the level each batch; preregistered simulations and ASOS data support it, but real-world gains shown are modest.
Summary
This paper specifies Odds-Ratio Thompson Sampling (OR-TS), a memory rule for batched multi-armed bandits with binary rewards. Instead of carrying per-arm absolute rate posteriors, as Beta-Bernoulli Thompson sampling (Beta-TS) does, OR-TS fits a reference-coded logistic regression at each batch boundary with a fresh intercept for the shared level, marginalizes it out, and carries forward only the joint posterior over log-odds contrasts; a ranking identity shows these contrasts alone determine Thompson allocation. Two operational controls are added: posterior decay λ and allocation aggressiveness γ. Evidence is preregistered and unusually candid. In prespecified synthetic environments (ten arms, forty batches), a common level shock costs Beta-TS five times OR-TS's regret and leaves the best arm below a majority of traffic in 7 of 20 runs, versus none for OR-TS; in stationary conditions the two are equivalent. An excess-standard-deviation analysis of 86 public ASOS A/B series finds the level varies about 25 times more than the contrast in every series; in a replay built from 71 experiments, expected-click differences stay within 0.1% for 58, yet OR-TS ends on the better arm more than twice as often. Contrast drift is the failure case, met with decay. Limitations: real contrasts are tiny, the logit-versus-linear scale is unidentifiable in these data, the Laplace approximation degrades in sparse batches, and no regret theory is given.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Level shifts never reorder arms (arg max p_i = arg max{β_1..β_{K−1},0}), so a batched bandit can discard the fitted level each boundary and carry only the joint log-odds contrast posterior.
- Across 86 public ASOS A/B series the shared level varies about 25 times more than the contrast (registered test: median ratio 25.1, all 86 series above 1), supporting the Level-varying premise.
- In prespecified synthetic common-shock environments, Beta-TS accrues five times OR-TS's regret (6,317 vs 1,246) and loses the best arm's traffic majority in 7 of 20 runs versus none, while the two are equivalent in stationary conditions.
- In an ASOS-based replay of 71 experiments, expected-click differences stay within 0.1% for 58 experiments, yet OR-TS ends with the better arm below a majority in only 11 of 71 versus 27 for Beta-TS.
- Contrast drift is the failure case for full contrast memory; posterior decay addresses it, but data-driven decay selection failed the paper's own registered no-loss criteria.
Strengths
Preregistration with disclosed failures
Hypotheses H1–H24 were registered with fixed seeds/horizons and immutable run IDs (Supplements E–F); failed predictions are reported as failures: H11's one-effect-gap boundary, H20–H22 learned-decay no-loss criteria (failed 3 of 5 cells), H23 event-count batch triggers, H19b recorded as falsified.
Mechanism isolation controls
Full-TS (same logistic model, Beta-TS memory) does no better than Beta-TS under the common shock, attributing the result to the memory rule not the model; the registered fixed-allocation sweep (Figure S2) shows Beta-TS identifies the best arm at every shock size, confirming the allocation-imbalance mechanism of Section 3.
Assumption tested on public data, with a counterexample
Deployment-grade specification
Honest boundary-keeping
Weaknesses
Benefit magnitude at realistic effect sizes
No theory
Approximation and regime limits
Scale choice not identified by data
Single-author, single-domain evidence
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper specifies Odds-Ratio Thompson Sampling (OR-TS), a memory rule for batched binary-reward bandits, positioned explicitly as a specification and design guide building on the author's 2020 preprint (Kim and Kim 2020). Standard Beta-Bernoulli Thompson sampling (Beta-TS) carries each arm's absolute rate posterior across batch boundaries; OR-TS instead fits a reference-coded logistic regression at every boundary with a fresh flat-prior intercept α_t (the shared level), marginalizes it out, and carries only the joint posterior over log-odds contrasts (μ_t, S_t). The load-bearing theoretical device is the ranking identity of Section 2.2: arg max p_i equals arg max of the contrasts plus zero, so a common level shift cannot reorder arms and the marginal contrast posterior suffices for Thompson allocation. Two operational controls complete the agent (Section 5): decay λ, a precision-tempering power prior, and aggressiveness γ over allocation sharpness. Claims are deliberately scoped: Table 2 is labeled by its author as not a validated rule, and the regret theory of the fresh-intercept recursion is listed as open.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Odds-Ratio Thompson Sampling (OR-TS) and how does it differ from standard Thompson sampling?
OR-TS is a Thompson-sampling variant for batched bandits with binary outcomes. Standard implementations (Beta-TS) store each arm's estimated success rate and carry it across updates; OR-TS stores only the relative comparisons between arms (log-odds contrasts) and re-estimates the shared baseline level from scratch at every batch. This keeps comparisons valid when the baseline moves (for example, weekend traffic converting worse) without carrying stale absolute rates into traffic allocation.
When does OR-TS fail or need adjustment?
When the contrasts themselves change over time, not just the baseline. The paper's Contrast-varying simulations show plain OR-TS then adapts slowly. The provided remedy is posterior decay, which forgets a fraction λ of old evidence per batch; it helped most when contrast drift coincided with level shocks. Automatic decay selection from data did not yet meet the paper's own no-loss criteria.
What did the analysis of real experiments find?
Across 86 variant-versus-control series from 71 public ASOS experiments, the shared level's variability (median excess SD 0.34) was about 25 times the contrast's (median 0.013) in every series, supporting OR-TS's premise. In a policy simulation built from those experiments, expected click differences were within 0.1% for 58 of 71, but OR-TS finished with the better arm holding a traffic majority far more often (Beta-TS below a majority in 27 of 71 experiments versus 11 for OR-TS).
Is the code or data available?
Yes. The reference implementation, synthetic runner, and archived results are on GitHub (github.com/sulgik/orts). The ASOS experiments dataset (Liu et al. 2021) and the Open Bandit Dataset are public. The 2020 advertising case study uses proprietary data that cannot be redistributed, so it is recalled rather than reproduced.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The extraction contains three marked omissions (parts of Section 3, Section 5.2, and Supplements E/F); conclusions are drawn from the surrounding complete text.
- Figures are assessed from captions and in-text descriptions only; underlying plotted data cannot be verified here.
- The review covers the embedded arXiv v1 text; the full preregistration ledger (RESEARCH_PLAN.md) lives in the external repository.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.