Automated Identification of Competing Narratives in Political Discourse on Social Media
Sergej Wildemann, Erick Elejalde
cs.CL · cs.SI
Quick verdict
Unsupervised BERTopic→event→story pipeline with user embeddings surfaces competing German political narratives; promising but evaluation is purely qualitative.
Summary
The paper presents an unsupervised framework for identifying competing narratives in political discourse on social media, applied to German politicians' tweets. From 189,850 tweets (January 2022–June 2023, 786 accounts identified via Wikidata), the pipeline clusters documents into 57 topics with BERTopic (43% discarded as outliers), detects events via affinity propagation over a combined semantic–temporal similarity, links events into stories using a Sliced Wasserstein Distance event graph partitioned by the Leiden algorithm, and splits story posts into user communities obtained by HDBSCAN clustering of global user embeddings. Validation is largely qualitative: user embeddings visually separate the Greens and AfD; XLM-T sentiment analysis and phi-4 LLM summaries contrast community voices in the 2022 energy-crisis debate; representative tweets illustrate divergent framings of a migration-law event. The authors acknowledge that quantitative evaluation is lacking: no baselines, metrics, or significance tests are reported, hyperparameter choices receive no sensitivity analysis, and the case studies are selectively presented. Code release is announced. The framework is a sensible composition of existing techniques and the case studies are instructive, but the claimed demonstrated efficacy exceeds the qualitative evidence.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- An unsupervised multi-stage pipeline (BERTopic topics → affinity-propagation events → SWD/Leiden story graphs → HDBSCAN user communities) can identify competing narratives in political tweets without using party-affiliation information during extraction.
- Global user embeddings built from topic-centroid-relative positions form communities that align with political ideology, evidenced by the clear separation of Greens and AfD members (Figure 3).
- Competing narratives differ both in content — sentiment and framing of the 2022 gas-price debate (Section 5.2) — and in structure, namely which events within a story different communities emphasize or skip (Section 5.3).
- Two qualitative case studies on polarizing issues (energy crisis, migration law) suffice to demonstrate the framework's efficacy, although the authors concede that datasets and evaluation metrics for competing narratives remain open challenges.
Strengths
Coherent, tweet-adapted end-to-end pipeline
Section 4 assembles BERTopic (min cluster size 500), affinity-propagation event detection (S1 = α·Stemp + (1−α)·Scos), SWD-based event graphs with Leiden partitioning, and HDBSCAN user communities; each MTLS component is deliberately reworked for tweet-level rather than sentence-level analysis (Sections 4.2–4.3).
Principled event-similarity design
Section 4.3 justifies Sliced Wasserstein Distance over centroid/cosine similarity because AP cluster centers may not cover an event's breadth, and notes the 15.9 tweets/event average makes SWD computationally efficient — a well-argued technical choice.
Dataset construction is documented and reproducible in principle
Unusually candid limitations section
Multi-signal narrative characterization
Weaknesses
No quantitative evaluation whatsoever
Narrow, potentially circular validation
All hyperparameters asserted, none analyzed
LLM summaries serve as evidence without fidelity checks
Substantial, unexamined data losses
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This Text2Story'25 workshop paper (arXiv 2609.11202) proposes an unsupervised framework that automatically identifies competing narratives in political discourse on social media, instantiated on 189,850 tweets from 786 German politician accounts (of 1,324 accounts / 1,300 politicians identified via Wikidata property filters) collected from 1 January 2022 to 24 June 2023, excluding retweets, replies, and quotes. Central claims: (1) a multi-stage pipeline — BERTopic clustering, affinity-propagation event detection, event linking into stories, and user-community separation — can surface competing narratives without using party labels; (2) the resulting user communities correspond to ideological positions; (3) two case studies (2022 energy crisis; the Residence Opportunities Act migration debate) demonstrate efficacy. The authors are candid in Section 6 that evaluation 'rel[ies] primarily on qualitative' evidence, framing missing annotations and metrics as open challenges.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does this paper's competing-narratives framework actually do?
It is an unsupervised NLP pipeline that takes social media posts (here, German politicians' tweets), groups them into topics, detects events within each topic, links events into coherent stories, and then splits each story's posts by clusters of like-minded users. The per-community versions of the same story are what the paper calls competing narratives.
Did the method need labeled data or party labels to work?
No. The pipeline deliberately ignores politicians' party affiliations during extraction; Wikidata party data is used only afterwards, to check that the discovered user communities align with political ideology — the Greens and the AfD separated most clearly in the embedding space.
What data was analyzed and is it publicly available?
189,850 tweets (excluding retweets, replies, and quotes) from 786 German politician accounts, collected January 2022 to June 2023 using accounts identified through Wikidata. The authors state the pipeline code is publicly released on GitHub, but the tweet corpus itself is not redistributed.
How well was the method evaluated?
Mainly qualitatively: a 2D visualization of user embeddings shows party separation, and two case studies (the 2022 energy-crisis gas price debate and a German migration law) contrast community framings using sentiment analysis and LLM-generated summaries. No quantitative metrics, baselines, or statistical tests are reported — a limitation the authors themselves acknowledge.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–4 appear in the extraction only as captions plus embedded axis/tick text; their visual content (e.g., cluster shapes in Figure 3a, sentiment distributions in Figure 4b) could not be inspected beyond the authors' written descriptions.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.