Beyond Accuracy: Centroid-Guided Contrastive Loss for Structured Fraudulent Job Posting Detection
Syed Ali Ahmed, Malaika Raza, Muhammad Shoaib Siddiqui, Muhammad Rafi
cs.AI
Quick verdict
Proposes CGCL loss for fake job posting detection claiming 99.2% SOTA on EMSCAD; internal ablations and a 99.9% prior baseline contradict the SOTA claim.
Summary
The paper proposes Centroid-Guided Contrastive Loss (CGCL), a training objective that combines inverse-frequency weighted cross-entropy with a centroid-based contrastive term: per class, the top-k farthest in-class embeddings are pulled toward the class centroid while the top-k closest cross-class embeddings are pushed beyond a margin m. The method is evaluated on EMSCAD (17,014 genuine vs. 866 fraudulent postings) using a six-hidden-layer MLP over GloVe/Word2Vec/TF-IDF text representations, with ADASYN oversampling; the best configuration (GloVe, alpha=0.5, beta=0.7, k=5) reports 99.2% accuracy and 0.987 F1, plus strong clustering metrics (ARI 0.953, NMI 0.903). A five-seed ablation finds that imbalance handling—ADASYN or center loss—drives nearly all of the gain (F1 from 0.718 to ~0.98), while full CGCL achieves the best PR-AUC (0.981) and recall (0.997) but lower F1 (0.971) and higher seed variance than CE CenterLoss. Main limitations: the SOTA claim is contradicted by the paper's own Table 1 (prior accuracies of 99.9% and 99.76%), the ablation backbone differs from the main model (three vs. six hidden layers; 500 vs. 3000 epochs), the margin m of the final model is unreported, and headline results rest on a single 80/20 split without significance testing.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- CGCL unifies class-weighted cross-entropy with a centroid-guided top-k push-pull contrastive loss to jointly optimize decision boundaries and latent-space structure.
- The GloVe-based CGCL model reaches 99.2% accuracy and 0.987 F1 on EMSCAD, which the authors claim is state of the art.
- The learned representations exhibit strong clustering structure (ARI 0.953, NMI 0.903, Silhouette 0.701), supporting the 'beyond accuracy' thesis.
- Ablations attribute most of the performance gain to imbalance handling (ADASYN or center loss), with full CGCL contributing the best PR-AUC (0.981) and recall (0.997) at higher seed variance.
Strengths
Clearly specified loss with pseudocode
Algorithm 1 and Eqs. (1)-(5) fully define the weighted CE, l2-normalized centroid pull/push with top-k selection, per-class skip conditions, and the alpha/beta combination — unusually complete for a loss-function paper and a genuine reproducibility asset.
Candid ablation reporting
Table 7 reports five-seed mean±std including unfavorable results: CGCL full shows the highest variance (F1 ±0.020, MCC ±0.028) and seed 99 collapses to F1 0.931; the discussion explicitly acknowledges sensitivity rather than hiding it.
Evidence beyond accuracy matches the framing
Broad model-level sweeps
Weaknesses
SOTA claim contradicted by the paper's own comparison table
Ablation does not support the core contribution
Ablation protocol mismatch and foreign terminology
Underspecified experimental details
Internal inconsistencies and discarded metadata features
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper introduces Centroid-Guided Contrastive Loss (CGCL), a unified objective (Eq. 5) combining inverse-frequency weighted cross-entropy (Eq. 1) with a centroid-driven contrastive term: for each class, the top-k farthest in-class embeddings are pulled toward the class centroid (Eq. 2) and the top-k nearest cross-class embeddings are pushed beyond a margin m (Eq. 3). The claims are: (i) CGCL simultaneously improves classification and latent structure; (ii) state-of-the-art performance on EMSCAD — 99.2% accuracy, 0.987 F1 with GloVe (alpha=0.5, beta=0.7, k=5; Table 5); (iii) clustering metrics (Table 6: Silhouette 0.701, ARI 0.953, NMI 0.903) certify structured representations. The scope is narrow: one dataset, one architecture (six-hidden-layer MLP, Fig. 3), and three classical text features (GloVe/Word2Vec/TF-IDF); no transformer and no re-run prior-method baselines are considered.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the CGCL method for fake job posting detection?
CGCL (Centroid-Guided Contrastive Loss) is a training loss that adds a contrastive term to weighted cross-entropy: each class's hardest top-k samples are pulled toward the class centroid while the nearest other-class samples are pushed beyond a margin, aiming to make the latent space compact within classes and separable between them.
What accuracy does CGCL reach on EMSCAD?
The paper reports 99.2% accuracy and 0.987 F1 with the GloVe configuration (alpha=0.5, beta=0.7, k=5) on a 20% held-out test split of EMSCAD (17,014 real vs 866 fake postings). Note that earlier works cited in the same paper report up to 99.9% accuracy under their own setups.
Is the code and dataset available?
Yes: the authors link code at github.com/ali-ahmed925/CGCL_code, and EMSCAD is a public Kaggle dataset ('Real or Fake Job Posting Prediction') with 17,014 genuine and 866 fraudulent postings collected 2012-2014.
Does the contrastive loss actually improve over simpler alternatives?
The paper's own five-seed ablation is mixed: cross-entropy with ADASYN (F1 0.981) or with center loss (F1 0.982, accuracy 0.990) matches or beats full CGCL (F1 0.971, accuracy 0.983) on average; CGCL's clearest advantages are recall (0.997) and PR-AUC (0.981), at the cost of higher variance across seeds.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-6 are available only via captions; the visual claims (t-SNE/UMAP substructure, seed-7 confusion-matrix counts, latent-space evolution over epochs) could not be independently checked from the text.
- Section IV-D (ablation) uses terminology foreign to the method ('contrastive temperature', 'contrastive graph objective'); this may be a drafting artifact, but could also mean different experiments than described.
- The paper's preprocessing (dropping all integer columns, deduplication) likely changes the effective dataset relative to prior EMSCAD work, so comparability of Table 1 numbers is uncertain in both directions.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.