Predicting Social Media Engagement using Machine Learning
Ritwik Singh, Mayukh Majumdar, Subodha Kumar
cs.SI · cs.LG
Quick verdict
Predicts Facebook engagement with LightGBM (R²=0.8178) on 32k furniture-brand posts; solid robustness checks, but brand-clustered split limits generalization.
Summary
The paper builds a predictive model of Facebook post engagement for 162 US furniture brands, using 32,456 image posts (≈200 per brand due to API limits). The target is a log-transformed weighted score (reactions ×1, comments ×5, shares ×10). From each post the authors extract 249 features spanning text (spaCy, TextBlob, VADER, roBERTa), visuals (OpenCV statistics, YOLO objects, CLIP embeddings with PCA), temporal/account variables, and — the stated novelty — brand-relative deviations and z-scores from each brand's own posting history; permutation and SHAP filtering leaves 172 features. Tuned LightGBM wins (test R² 0.8178, RMSE 0.7847) over XGBoost (0.8121), Random Forest (0.8059), and OLS (0.3544); a paired t-test over five CV folds favors LightGBM over XGBoost (t=4.61, p=0.010). Historical posting patterns dominate group importance (51.3%), and a brand-only model reaches R² 0.688, implying content features add ~0.13. Robustness checks (age-adjusted targets, month-demeaning, within-season fits) leave results largely unchanged. Limitations: no paid-boost data, single-industry scope, no image-emotion features, and no code or data release. Evaluation uses a random post-level split despite heavy brand clustering, so generalization to unseen brands is untested, and directional claims rely on signed mean SHAP values without uncertainty estimates.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A tuned LightGBM on 172 joint visual/textual/temporal/brand-history features achieves test R² 0.8178 (RMSE 0.7847) for log-weighted Facebook engagement, versus 0.3544 for an OLS baseline.
- Brand identity alone explains R² 0.688 of engagement variance; content features add roughly 0.13 R² on top.
- Brand-relative deviation and z-score features are among the most informative predictors, with historical posting patterns the dominant group at 51.3% of permutation importance.
- Raw hashtag count is positively associated with engagement while exceeding a brand's own hashtag norm is negatively associated, per signed mean SHAP values.
- Findings are robust to age-adjusted targets, monthly demeaning, and within-season refitting (R² remaining between 0.77 and 0.82 across checks).
Strengths
Joint multi-modality framework at real scale
32,456 posts from 162 brands with 172 final features spanning visual, textual, temporal, and brand-history groups modeled together (Sections 1–2), where the authors argue prior work examined these feature types in isolation.
Brand-relative deviation features with a decomposed baseline
The stated novel contribution (Section 1); the brand-only model (R² 0.688, Section 3) cleanly quantifies the content-feature increment (~0.13), and the within-brand demeaning check (Pearson r 0.029→0.047) directly addresses the brand-size confound.
Leakage-aware evaluation protocol
Extensive, targeted robustness battery
Honest limitation framing
Weaknesses
Post-level random split despite dominant brand identity
Directional claims rest on signed mean SHAP values
Arbitrary target weighting without sensitivity analysis
Reporting inconsistencies and no artifact release
Interpretive overreach in places
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper predicts weighted Facebook engagement for image posts from 162 US furniture brands (32,456 posts) and makes three claims: (1) jointly modeling visual, textual, temporal, and brand-history features improves prediction relative to prior single-modality work; (2) brand-relative deviation features are among the most informative predictors; (3) the resulting rankings support managerial recommendations. The authors are appropriately careful — the Discussion frames all findings as "predictive associations rather than established causal effects," and Section 1 scopes the study as "an in-depth case study of a mature retail category." The headline result: tuned LightGBM explains 81.78% of variance in the log-transformed weighted engagement score (reactions ×1, comments ×5, shares ×10; Table 1), versus 35.44% for OLS.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
How well does the model predict Facebook post engagement?
The tuned LightGBM model reaches a test R² of 0.8178 (RMSE 0.7847) on a log-transformed weighted engagement score (reactions ×1, comments ×5, shares ×10), compared with 0.3544 for an OLS baseline, 0.8121 for XGBoost, and 0.8059 for Random Forest. A brand-only model without any content information already achieves R² 0.688.
What data does the study use?
32,456 image posts collected from the Facebook pages of 162 US furniture brands, roughly 200 posts per brand because of API rate limits. The posts yield 249 candidate features (text, image, temporal, and brand-history), reduced to 172 by permutation importance and SHAP-based selection.
What drives social media engagement most, according to the paper?
The brand's historical posting patterns dominate, contributing 51.3% of permutation importance, followed by visual features (32.4%), textual features (13.4%), and temporal context (2.9%). Posts that deviate sharply from a brand's own usual style — for example, far more hashtags than normal — are associated with lower engagement.
Can managers use these findings to increase engagement?
The authors explicitly frame the results as predictive associations, not causal effects. Raw hashtag counts and all-caps words show positive signed-SHAP associations, but exceeding a brand's own norms shows negative ones, so they recommend treating the suggestions as hypotheses to test (for example via A/B posting experiments) rather than guaranteed tactics. The study also cannot separate organic engagement from paid post boosting.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Fig. 1 (the SHAP beeswarm underlying several directional claims) cannot be inspected in the extracted text, so figure-specific claims are taken on trust.
- Supplementary details are absent: hyperparameter search space, exact collection window, and the full brand list; the 4-brand sample-cap check is footnote-level only.
- No author-supplied code or data; all reproducibility judgments rest on the paper's descriptions alone.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.