Issue Bias in Generative AI Writing Assistance: Political Issues and LLMs in the Swedish 2026 Election
Bastiaan Bruinsma, Annika Fredén, Paul Röttger, Moa Johansson, Asad Sayeed
cs.AI · cs.CY · stat.AP
Quick verdict
Six LLMs show issue-level stances in Swedish writing tasks—GPT neutral, Grok divergent—yet no significant party alignment; LLM-judge validation unfinished.
Summary
This preliminary-draft study measures political stance patterns of six commercial LLMs — Claude Sonnet 4.6, DeepSeek V4 Pro, Gemini 2.5 Pro, Mistral Large 2512, GPT Chat Latest, and Grok 4.3 — on Swedish-language writing tasks ahead of the September 2026 Swedish election. Adapting IssueBench, the authors cross 107 policy propositions (97 taken from three Swedish voting advice applications, whose eight parties' positions on a -2 to +2 scale serve as reference) with 77 writing templates and neutral, positive, and negative framings, yielding 24,717 prompts per model and 146,350 eligible responses collected via OpenRouter at temperature 0. Gemini 2.5 Flash assigns each output to a six-way stance taxonomy. Under neutral framing, GPT most often produced neutral text (majority category on 84 of 107 issues), Grok most often exclusively pro text (50 issues) and diverged most on migration, crime, and gender items (e.g., +1.216 versus a -0.034 mean of the other five models on reporting undocumented migrants); Claude, DeepSeek, Gemini, and Mistral clustered tightly. The Social Democrats were the numerically closest party for all six models (mean absolute distance 1.45-1.52 over 97 issues), yet no within-model party contrast survived Holm correction. Key limitations: human validation of the LLM classifier (540-response sample) is unfinished, each prompt ran only once, party comparisons are purely descriptive, and no code or data release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Under neutral framing, the six models differ mainly in consistency and direction: GPT most often produces neutral/ambivalent text (majority category on 84 of 107 issues), Grok most often exclusively pro text (50 issues), and no model ever has a con-majority or refusal-majority issue.
- Claude, DeepSeek, Gemini, and Mistral share closely related issue profiles (mean pairwise correlations 0.786-0.823, Table 4), while Grok diverges most on migration, crime, and gender propositions and GPT differs mainly through neutrality.
- All models comply strongly with explicit directional framings (positive 82.9-93.2%, negative 97.0-99.3%), so default stance tendencies are conditional rather than immutable.
- The Social Democrats are numerically closest to all six models (mean absolute distance 1.45-1.52 across 97 issues), but after Holm correction no within-model party contrast is significant, so no model shows a statistically distinguishable party alignment.
Strengths
Large scale with ecologically valid prompts
The IssueBench adaptation crosses 107 policy propositions with 77 realistic writing templates and 3 framings (24,717 prompts per model; 146,350 eligible responses, Sec. 3), measuring what voters actually ask rather than abstract ideology questionnaires, in a non-English language with real VAA party positions as reference.
Honest null on party alignment
Sec. 5.3 and the Table 5 note explicitly state that none of the 28 within-model party-pair comparisons survives Holm correction and that Table 5 supports only a descriptive statement, actively blocking the overclaim that models 'endorse' the Social Democrats.
Transparent collection audit
Directional framings as manipulation checks
Careful construct discipline
Weaknesses
Unvalidated LLM judge underpins every result
Single run per prompt with no variance estimates
Party proximity may be a scale artifact
Under-specified measurement inputs
Preliminary-draft quality signals
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Review: Issue Bias in Generative AI Writing Assistance (Bruinsma et al., arXiv 2609.15207)
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Did any AI model favor a Swedish political party?
Numerically the Social Democrats were closest to all six models studied (mean absolute distance 1.45-1.52 across 97 policy issues), but after Holm correction for multiple comparisons no within-model difference between parties was statistically significant. The authors therefore conclude that no model shows a clear or statistically distinguishable preference for any party.
Is ChatGPT or Grok politically biased according to this study?
The study finds issue-level tendencies rather than a left-right ideology. On neutrally framed Swedish writing tasks, GPT most often produced neutral or ambivalent text (majority category on 84 of 107 issues), while Grok most often wrote exclusively supportive text (50 issues) and diverged most from the other models on migration, crime, and gender propositions, such as supporting deportation of dual citizens and repealing the Gender Recognition Act.
How large was the experiment?
107 policy propositions were crossed with 77 writing templates and three framings (neutral, positive, negative), giving 24,717 prompts per model and 148,302 total attempts across six models, of which 146,350 (98.7%) were eligible. Responses were collected via OpenRouter at temperature 0 between July 30 and August 12, 2026.
Is this paper peer reviewed and final?
No. It is a September 2026 preliminary arXiv draft, presented at an APSA 2026 panel, with the note 'please do not cite without permission'. The planned human validation of the automatic stance classifier (540 responses, two coders) was not yet complete, so the reported labels and results may change.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 is an image; its plotted detail could not be verified from the extracted text, though the key numbers are restated in Section 5.2 prose.
- The paper self-identifies as a preliminary draft; the classifier's human validation (footnote 4) is incomplete, so reported labels and conclusions may change.
- Appendix tables A and B were extracted in full, but long-table OCR may contain minor diacritic or alignment artifacts in the Swedish text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.