When Decodability Is Not Enough: Logical Validity Representations, Behavioral Dissociation, and Causal Tests in Language Models
Smitha Muthya Sudheendra, Jaideep Srivastava
cs.CL · cs.LG
Quick verdict
Claims validity is decodable but not behaviorally expressed or causally used in LLMs; controlled study, but small models and synthetic data limit scope.
Summary
This paper asks whether logical validity is represented, behaviorally expressed, and causally used in language models. The authors build a controlled dataset of 800 examples — 400 matched valid–invalid premise–claim pairs spanning five inference families, five semantic domains, and three difficulty levels — and evaluate five open-weight transformers (Pythia-1.4B/2.8B, SmolLM3-3B, Llama-3.2-3B, Mistral-7B). Behavioral verification, scored by likelihood of VALID versus INVALID, stays at or near chance (0.470–0.500 accuracy) with strong model-specific label biases. Linear probes on the final prompt-token hidden state instead reach near-perfect random-split AUROC (1.000) and remain strong under held-out templates (0.963–0.999), domains, and inference families. Exhaustive leave-one-out tests expose exceptions: syllogism is the weakest held-out family for every model (AUROC 0.500–0.762), and the Pythia models degrade on the biological domain. Matched-pair accuracy is 1.000 throughout, and validity remains decodable on behaviorally incorrect examples where that comparison is well defined (mainly Llama-3.2). TF–IDF controls reveal substantial lexical predictability in-distribution (0.970), and interventions along the probe-derived direction change output margins only weakly and inconsistently, comparable to random orthogonal controls, with no prediction flips at the tested strengths. Limitations: synthetic templated data, small quantized models, a single final-token probe/intervention site, and no stated code or data release.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Behavioral verification of logical validity is at or near chance in all five models (0.470–0.500 accuracy) despite strong model-specific answer-label biases (Table 2).
- Logical validity is linearly decodable from hidden states at near-perfect in-distribution AUROC (1.000) and remains strongly decodable under held-out templates (0.963–0.999), domains, and inference families (Table 3), with systematic exceptions — syllogism is weakest for every model (0.500–0.762; Fig. 2).
- Validity remains decodable on behaviorally incorrect examples wherever correctness-conditioned evaluation is statistically well defined (mainly Llama-3.2; Table 10).
- Interventions along the probe-derived validity direction change output margins only weakly and inconsistently — comparable to or smaller than norm-matched random controls — with no prediction flips at the tested strengths (Table 4, App. H).
- Behavioral expression, representational accessibility, and causal use of validity are therefore distinct forms of evidence about what a model knows.
Strengths
Matched-pair design isolates the relational property
Within each pair, premises, family, domain, template, and difficulty are held fixed and only the claim flips (§3.2); pairs are kept together in splits and bootstrap, yielding a within-context test with PairAcc = 1.000 across all five models and all four primary splits (§5.4, App. E).
Unusually thorough control battery
Full-prompt, claim-only, and premises-only TF–IDF, metadata-only classifiers, and 200 shuffled-label permutations (§4.3, App. F). The paper's own controls show the random-split probe AUROC of 1.000 is partly lexical (TF–IDF 0.970) — a level of self-skepticism rare in probing papers.
Honest null on causal interventions with proper controls
Exhaustive leave-one-out analyses add real value
Statistical hygiene
Weaknesses
In-distribution result is heavily lexically confounded
Thin evidence base for the dissociation-on-errors claim
Causal null may be a site/design artifact
Behavioral protocol may underestimate models
Synthetic templated data and no release statement
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks whether logical validity — whether a claim follows from premises — is (i) expressed in model outputs, (ii) linearly accessible in hidden states, and (iii) causally used in the decision. Using 800 deterministically generated examples (400 matched valid–invalid pairs) across five inference families (syllogism, transitivity, set inclusion, causal chain, permission logic), five semantic domains, and three difficulty levels, the authors evaluate Pythia-1.4B/2.8B, SmolLM3-3B, Llama-3.2-3B, and Mistral-7B. Central claims: behavioral verification is at or near chance (Table 2: 0.470–0.500 accuracy with extreme label biases); validity is near-perfectly linearly decodable (random-split AUROC 1.000, Table 3) and largely transferable across held-out templates (0.963–0.999), domains, and families; validity remains decodable on behaviorally incorrect examples where defined (Table 10); interventions along the probe direction are weak and nonspecific (Table 4). The claims are carefully scoped and, in our reading, supported by the presented evidence — with one important qualification about claim three's evidentiary base.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the main finding of this paper?
That in five open-weight language models, logical validity is nearly perfectly decodable from hidden states even though the models' actual yes/no answers are at chance level, and that pushing activations along the probe's validity direction barely changes outputs — so representing validity, expressing it in behavior, and using it causally come apart.
Which language models were tested?
Pythia-1.4B, Pythia-2.8B, SmolLM3-3B, Llama-3.2-3B, and Mistral-7B (v0.3). Causal interventions were run only on Pythia-2.8B, Llama-3.2-3B, and Mistral-7B.
Does high probe accuracy mean the model actually uses validity when answering?
No — that is the paper's central point. Probes reach AUROC 1.000 in-distribution, but interventions along the probe direction shift output margins by only about 0.002–0.004, comparable to random-direction perturbations, and flip no binary predictions at the tested strengths (α up to ±4).
Is the code or dataset available?
The paper does not state a code or data release. The dataset is described as deterministically generated from known inference rules (800 examples, 400 matched pairs), and Hugging Face model identifiers are listed, but the exact templates are not published in the text, so exact replication is not currently possible.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–3 are extracted as scattered axis labels and numbers, so visual details of the layer-wise trajectories and intervention sweeps could not be assessed.
- No concrete dataset items (verbatim premise–claim examples) appear anywhere in the extracted text, so template wording and potential lexical-leakage sources cannot be inspected directly.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.