Off-Target Effects of Response-Style Alignment in a Korean 27B Language Model
cs.AI
Quick verdict
Claims style-only post-training moves bias exposure and disclosure via answer propensity/verbosity; rigorously controlled but single-model, n=3 seeds.
Summary
The paper post-trains Qwen3.8-27B for Korean response style (verbosity, list/markdown usage, discourse structure, register) and measures two behaviours the objective never targets: abstention on ambiguous KoBBQ items and unprompted disclosure in securities guidance. Both moved, and the changes run through the model's emission policy (how often and how much it answers). Matched target-form controls — identical prompts, recipe, data volume and serving, only target text swapped — show style targets raise the ambiguous answer rate (three seeds, mean +0.82 pp) while neutral targets lower it (mean −1.53 pp), with non-overlapping seed ranges; a length-matched arm sits between, and a hedge-preserving arm is too unstable to identify the responsible feature. An exact algebraic decomposition shows absolute stereotyped-exposure changes are dominated by the answer-propensity term (composition term confined to +0.00 to +0.45 pp across seventeen contrasts). On disclosure, style training cuts detector-identified disclosure ~22 pp; assigning response length reproduces most of that gap, while compliance distillation gains +59.7 to +82.1 pp at every assigned length. The paper also shows conditional-share contrasts are selection-confounded, detector agreement is treatment-dependent (0.44–0.99), and option rotation flips a pre-specified abstention verdict. Limitations: single model and language, at most three seeds per arm (permutation floor p=0.10), unresolved target-form mechanism, unverified endpoint identity for the disclosure arms, and a withheld training corpus.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The direction of the answer-propensity shift on ambiguous KoBBQ items depends on the form of the training target, not the prompt set or recipe alone: three style seeds give positive answer-rate changes (mean +0.82 pp) and three neutral seeds negative ones (mean −1.53 pp), with non-overlapping observed ranges.
- Across seventeen training-checkpoint contrasts, changes in absolute stereotyped exposure are dominated by the answer-propensity term (span −13.42 to +1.47 pp) while the composition term stays within +0.00 to +0.45 pp, and this is an allocation statement about a deployment-visible quantity, not evidence about latent preference.
- Style post-training reduces detector-identified disclosure by about 22 pp, most of which is reproduced by directly assigning response length (residual 3–5 pp at ≥400 characters, indistinguishable from zero at ≤200), while compliance distillation gains +59.7 to +82.1 pp at every assigned length.
- Standard instruments fail under the treatment: between-arm conditional-share contrasts are selection-confounded when answer status is treatment-dependent, disclosure-detector agreement runs 0.44–0.99 across checkpoints, and option rotation flips a pre-specified ≥93% abstention verdict (fails 8 of 10 rotations).
Strengths
Matched target-form control design
§4 holds prompts, recipe, data volume, holdout split and serving fixed and swaps only the target text (neutral, length-matched, hedge-preserving; 3 seeds each); base is re-evaluated on the arm serving configuration, and /v1/models identity is asserted before recording for all KoBBQ arms (§3.3).
Identity-versus-finding discipline
§5.1 states ΔS = L0ΔA + A1ΔL is exact algebra, verifies the composition column numerically, reports allocation in percentage points before ratios, and quotes the 93–116% share only for the 13 of 17 contrasts where ΔS resolves under template clustering.
Cluster-aware inference honestly tiered
Dose-response by assignment, not adjustment
Measurement-validity results elevated to findings
Weaknesses
Very small seed counts with weak per-arm inference
Mechanism unresolved and the fourth arm doubly confounded
Narrow external validity
Disclosure-axis provenance gap
Reproducibility partially limited
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper post-trains Qwen3.8-27B for Korean response style and asks what happens to two behaviours the objective never mentions: abstention on ambiguous KoBBQ items and unprompted securities disclosure. Four RQs structure the results: whether untargeted behaviours move and why (§4), whether changes are carried by emission policy vs conditional content (§5, the main section), whether the instruments survive the treatment (§6), and whether repairs compose and how order matters (§7, descriptive). Central claims: (1) the answer-rate direction is attributable to target form — three style seeds rise (mean +0.82 pp), three neutral seeds fall (mean −1.53 pp), ranges non-overlapping, mean gap +2.343 pp, CI [+1.177, +3.600]; (2) exposure changes are dominated by the answer-propensity term via the exact identity ΔS = L0ΔA + A1ΔL — across seventeen training contrasts the propensity term spans −13.42 to +1.47 pp while the composition term stays within +0.00 to +0.45 pp; (3) the style stage cuts detector-identified disclosure ~22 pp (−22.25 loose / −21.83 strict), mostly reproduced by assigned length (3–5 pp residual at ≥400 chars, ~0 at ≤200), while compliance distillation gains +59.7 to +82.1 pp at every assigned length; (4) instruments break: conditional-share contrasts are selection-confounded, detector agreement spans 0.44–0.99, and rotation flips the pre-specified ≥93% abstention verdict (fails 8/10 rotations).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is an off-target effect in LLM alignment?
A change in a behaviour that a training objective never aimed at. Here, post-training a model only for Korean response style (brevity, formatting, register) changed how often it answers ambiguous social questions and whether it emits legally expected investment disclosures.
Did style alignment make the model more biased?
Not in the sense of preferring stereotypes more. Absolute stereotyped exposure rose because the model answers ambiguous questions more often (+0.57 to +1.24 pp across seeds); the composition of its answers barely moved, and the authors explicitly decline to claim any change in latent preference.
Did style training destroy the model's compliance capability?
The authors frame it as an emission change, not a capability loss: making the base model write short responses via a length instruction reproduces most of the detector-measured ~22 pp disclosure drop, so brevity alone accounts for most of it — sufficiency, not proven causation.
Is code or data available for this paper?
Weights, model cards and evaluation harnesses are released; the paired training corpus is not, so the training arms can be reproduced at the method level but not file-for-file. Every reported number maps to a measurement ledger, and the KoBBQ snapshot hash is published.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The abstract in the supplied metadata is cut mid-sentence, but the complete abstract is present in the extracted full text; no material truncation was detected anywhere in the provided text.
- Tables are linearized into running text with page-break artifacts (e.g., the string '3 Table 1:' inside a sentence); all numbers quoted in this review were legible in the extraction.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.