Reference-Based Bias Detection in LLMs via Relative Representations of Hidden States
Marek Jeliński, Jan Dubiński, Maciej Chrabaszcz, Sebastian Cygert
cs.AI
Quick verdict
ΔB, an anchor-based relative-representation bias-shift metric, tracks output bias in 15/18 settings, but is validated only on a controlled merge spectrum.
Summary
The paper proposes a reference-based method for auditing bias shifts in LLMs by comparing hidden-state representations of a fine-tuned model against its base model. Because fine-tuning reshapes embedding geometry, each sentence is encoded through its cosine similarities to a fixed set of 1,000 neutral anchor sentences, yielding relative representations in a shared space; the Representational Bias Shift ΔB measures how target groups move toward negative versus positive attributes relative to the reference. Across Llama-3.1-8B, Mistral-7B and Gemma-3-4B, full and LoRA fine-tuning on WildGuardMix splits with linear merging into seven-checkpoint spectra, and three output benchmarks (WildGuardMix harmfulness, DecodingTrust stereotypes, ToxiGen toxicity), ΔB correlates with output-level bias change in 15 of 18 settings, reaching |r| = 0.84 under full fine-tuning, and thresholding it detects increased-bias checkpoints with ROC AUC 0.65–0.99, beating a SEAT baseline on WildGuardMix and DecodingTrust. Ablations show stability across anchors, attributes, templates, pooling and seeds. Main limitations: validation uses output benchmarks at arbitrary thresholds as ground truth; pooled correlations treat non-independent checkpoint–group pairs as independent; results are confined to a synthetic merge spectrum, English single-axis groups and the final layer; Gemma under LoRA shows no significant correlation. Code is released.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Relative representations over a shared anchor set make hidden-state bias measurements comparable across fine-tuned model variants without fitting any cross-model map.
- ΔB co-varies with output-level bias change in 15 of 18 tested settings, with |r| up to 0.84 (p < 0.001) under full fine-tuning.
- Thresholding ΔB detects increased-bias checkpoints with ROC AUC 0.65–0.99 and outperforms a SEAT baseline on WildGuardMix and DecodingTrust for all three model families.
- ΔB is stable under changes to anchor sets, attribute sets, target templates, pooling strategy and training seeds.
- The method audits a model in about three minutes using 3–50× less compute than the output-level benchmarks considered.
Strengths
Principled cross-model comparison without fitted maps
Relative representations over a shared 1k-anchor set (Sec. 3.3) avoid any cross-model alignment fit. The baseline decomposition in Table 7 makes the attribution clean: Procrustes-SEAT sits at chance (0.567/0.513 on Llama), CKA drift detects but stays below RR (0.864/0.753 vs 0.964/0.949), so the gain comes from the construction itself.
Graded, spectrum-based validation with noise controls
Seven checkpoints per family/regime from linear merges of harmful and benign fine-tunes (Appendix C) give graded bias rather than a single jump, and the seed study (std 0.003, Appendix E.5/Fig. 5) shows ΔB is not training noise.
Honest negative results and explicit scoping
Broad robustness ablations with numbers
Practical cost profile
Weaknesses
Ground truth is one output benchmark at an arbitrary operating point
Pseudo-replication in pooled correlations
Easy, engineered setting
Unresolved Gemma anomaly
Narrow representational and linguistic scope
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper defines a comparative auditing task: given a reference model (typically the base checkpoint) and an audited variant, quantify how much a target social group's hidden-state association with negative versus positive valence has shifted. The core construct is the Representational Bias Shift ΔB (Sec. 3.4), computed in a relative-representation space induced by 1,000 neutral anchor sentences (Sec. 3.3). The authors are careful about scope: ΔB is explicitly a proxy (Sec. 3.4), the paper does not claim representation geometry determines behaviour (Sec. 1), and the method is positioned as complementary to output-based auditing (Abstract, Conclusions). The central empirical claim — ΔB co-varies with output-level bias change in 15 of 18 settings, |r| up to 0.84 under full fine-tuning, ROC AUC 0.65–0.99 — is precisely stated and consistent with Table 1.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the Representational Bias Shift (ΔB)?
ΔB is a number measuring how much a fine-tuned model's internal associations for a social group shifted toward negative (or positive) attributes compared with a reference model such as its base checkpoint. It is computed in a shared comparison space built from each model's similarities to a fixed set of about 1,000 neutral anchor sentences, so hidden states of two model variants can be compared directly without aligning their embedding spaces.
Does ΔB replace output-level bias benchmarks?
No. The authors position it as a complement: it is cheap (about three minutes per model), needs no evaluation dataset or judge model, and flags which checkpoints deserve closer output-level testing. It requires a reference model, cannot certify a model unbiased on its own, and missed the signal in 3 of 18 tested settings (notably Gemma under LoRA).
How well does ΔB detect models whose bias increased after fine-tuning?
In the paper's experiments, thresholding ΔB detected increased-bias checkpoints with ROC AUC between 0.65 and 0.99 across three model families, three benchmarks, and full/LoRA fine-tuning, and it beat a SEAT-based baseline on the WildGuardMix and DecodingTrust benchmarks for all three families. Reliability dropped for the smallest model, Gemma-3-4B, under LoRA.
What data and code are available for this paper?
The auditing code and the anchor, target and attribute sentence sets are open-sourced at github.com/NASK-AISafety/Reference-Based-Bias-Detection. The fine-tuned and merged checkpoints are not released for safety reasons, and the harmful training split is synthetic, derived from WildGuardMix.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 2, 3, 4 and 5–16 are scatter/ROC plots whose underlying point clouds are available only via captions in the extraction; per-threshold and per-anchor-source numeric values inside them could not be independently verified.
- No end-of-text truncation observed: the extraction runs through Appendix H.3 and ends with a complete sentence; the supplied metadata abstract is cut mid-sentence but the full abstract is present in the embedded text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.