DeepAffinity: Long-Term Aspect Preference Prediction in eCommerce using Small Language Models
Yotam Eshel, Guy Hadad, Guy Feigenblat, Yuri M. Brovman, Matt Gearhart, Bracha Shapira
cs.LG · cs.AI
Quick verdict
135M SLM with prediction heads beats heuristics, prompted LLMs and 3B SFT at aspect prediction; solid, but private data, weak stats, no sequential baselines.
Summary
The paper introduces Aspect Affinity: predicting the aspect values (brand, size, color, and beyond) of a user's future eCommerce purchases from their time-ordered interaction history, with a 24-hour gap that strips out current-session intent. DeepAffinity fine-tunes SmolLM2-135M on stringified user histories with special-token structure, per-event [CLS] affinity embeddings, category conditioning, and one linear prediction head per aspect, trained jointly with cross-entropy. On proprietary eBay data (six months, tens of millions of users; 1K fashion and 20K all-category test samples), it reports micro-F1 of 0.34 versus 0.28 for personalized heuristics and 0.26 for zero-shot Gemini 1.5 Flash on fashion, and 0.35 versus 0.30 for a 3B fine-tuned generative model on all categories, matching or beating baselines on most of the ten aspects. An offline/online split (cached embeddings, lightweight heads) enabled a production ranker test showing +0.6% Recall@1, +0.4% Recall@3, and +0.3% Recall@5. Zero-shot experiments over Llama/Qwen models show prompting helps but never closes the gap. Main limitations: all data is proprietary with no code release, no statistical significance testing, no comparison against standard sequential recommenders (SASRec/BERT4Rec are cited but not benchmarked), figure-only results for the generative comparisons, and Table 2's narrative (SFT better on two aspects) does not quite match its own numbers (three aspects favor SFT).
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Aspect Affinity — forecasting aspect values of a user's future purchases from time-ordered history with a 24-hour horizon — is a well-defined temporal task distinct from session-intent prediction.
- DeepAffinity, a 135M SmolLM2 with structured stringification and per-aspect linear heads, outperforms MCV/PMCV heuristics, zero-shot Gemini 1.5 Flash, and a QLoRA fine-tuned 3B generative model (mean micro-F1 0.35 vs 0.30 on the all-category set, 0.34 vs 0.28 PMCV on fashion).
- General-purpose open-source LLMs perform poorly without task-specific fine-tuning, and few-shot/CoT/enhanced prompting never closes the gap to the fine-tuned 135M model.
- Adding two DeepAffinity-derived features (predicted brand and size purchase likelihoods) to a production ranker improves Recall@1/3/5 by 0.6%/0.4%/0.3% over the deployed baseline.
Strengths
Novel, broadly-scoped task formulation
Aspect Affinity generalizes prior single-aspect work (size/fit prediction [3,6,13], query filter suggestion [9]) to predicting all aspect values jointly with cross-aspect interactions, framed cleanly as multi-task multi-class classification over P(v|u,c,a) in §3.
Rigorous temporal design against leakage
Strict temporal split (test = a subsequent week), a 24-hour label horizon that explicitly excludes current-session intent, and causal full-sequence training with cross-entropy at every [CLS] position so future events are masked (§3, Training on Entire Sequences).
Strong efficiency result
Wide baseline spectrum
Demonstrated production value
Weaknesses
No statistical significance reporting
Missing sequential-recommendation baselines
No component ablations
Reporting inconsistencies and metric switches
Proprietary data, low reproducibility
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper defines Aspect Affinity as estimating P(v|u,c,a): the probability that user u's next purchase in category c carries value v for aspect a, predicted from a time-ordered history of purchases and queries. Ground-truth labels must fall at least 24 hours after the history prefix (§3) — the paper's operational definition of 'long-term' preference, explicitly separating session intent from durable taste. Three claims are advanced: (1) a fine-tuned 135M SLM with per-aspect prediction heads beats heuristics, zero-shot LLMs, and larger fine-tuned generative models; (2) careful input-output design, not scale, drives the gains; (3) the approach delivers measurable production value in a deployed ranker.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Aspect Affinity in eCommerce?
A prediction task introduced by this paper: given a user's time-ordered history of purchases and search queries, forecast the aspect values (brand, size, color, style, etc.) of items they will buy in a given category in the future. A required 24-hour gap between the history and the predicted purchase separates long-term preference from current-session shopping intent.
What model does DeepAffinity use and how large is it?
It fine-tunes SmolLM2, a 135M-parameter small language model. Each user's event history is converted into a structured string; a per-event [CLS] embedding ('Affinity Embedding') is combined with a category embedding and passed to one linear classification head per aspect. Despite being roughly 20x smaller, it matched a 3B fine-tuned generative model on the paper's all-category benchmark.
Is DeepAffinity's code or dataset publicly available?
No. The paper uses six months of proprietary eBay US interaction data and states that no public dataset contains the needed item aspects and values. No code release is mentioned, so the reported results cannot currently be reproduced outside eBay.
Did DeepAffinity actually improve real recommendations?
The authors report a production-ranker experiment: adding two features derived from DeepAffinity (predicted brand and size purchase likelihoods) improved Recall@1 by 0.6%, Recall@3 by 0.4%, and Recall@5 by 0.3% over the deployed ranker. However, no statistical significance tests are reported for these gains.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 2 and 3 are described qualitatively in the text but their plotted values (e.g., Mean Precision curves, zero-shot LLM scores) exist only as images and are not present in the extracted text.
- The provided text ends with reference [15] and appears complete; no truncation was observed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.