Psychosis involves a deficit of information compression in connected speech
Samuele Vallisa, Claudio Palominos, Rui He, Emre Bora, Burcu Verim, Cemal Demirlek, Berna Yalincetin, Philipp Homan, et al.
cs.CL
Quick verdict
Finds reduced grammar-based surprisal gain and embedding ID in 106 Turkish psychosis patients; credible, but ID-surprisal effects flip sign by estimator.
Summary
This empirical study tests whether psychosis reflects a deficit of information compression in connected speech, operationalized through language-model metrics. The authors analyze transcribed speech from 144 Turkish speakers — 38 healthy controls and 106 patients with schizophrenia-spectrum disorders (56 chronic schizophrenia, 33 first-episode psychosis, 17 schizoaffective disorder) — elicited via picture description, free speech, and retelling. They define surprisal difference as the gap between word-frequency surprisal and context-based surprisal from Turkish BERT, estimate intrinsic dimensionality (ID) of the embeddings with 2NN and Gride estimators, and combine these with syntactic metrics (dependency optimality, hierarchical and constituency depth) in GEEs controlling for age, sex, education, and word count. Surprisal difference is attenuated in all clinical groups and robust across 50–300-word windows, while frequency surprisal shows no group effect. ID is reduced in SZH (β≈−0.91) and FEP (β≈−1.01) but not SZA, and syntactic complexity predicts surprisal difference. Critically, the ID–surprisal-difference association flips sign between estimators (Gride β≈+0.27; 2NN β≈−0.45). Limitations include no stated multiple-comparison correction, missing confidence intervals, dependence on a single BERT model for all measures, a roughly 90%-male sample, and private data with no code release.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Surprisal difference (frequency minus contextual BERT surprisal) is significantly attenuated in chronic schizophrenia, first-episode psychosis, and schizoaffective disorder relative to healthy controls, independently of word count and robust across 50–300-word windows.
- Intrinsic dimensionality of BERT token embeddings is reduced in chronic schizophrenia (β≈−0.91) and first-episode psychosis (β≈−1.01) but not significantly in schizoaffective disorder.
- Syntactic complexity metrics (dependency optimality Ω, mean hierarchical distance, constituency depth) predict surprisal difference, supporting grammar as the mechanism of contextual uncertainty reduction.
- ID and surprisal difference are associated, but the sign of the association depends on the ID estimator's neighbourhood scale (negative at k=2, positive at larger k up to 256).
Strengths
Cross-diagnostic, clinically stratified sample
106 schizophrenia-spectrum patients split a priori into 56 SZH, 33 FEP and 17 SZA against 38 HC, with full demographics, SAPS/BNSS/PSP/SCIP scores and ethics documentation (Table 1, protocol 8527-GOA), allowing severity gradients (e.g., PSP 86.5 HC → 40.8 SZH) to be read off rather than lumped.
Word-count confound handled twice
Word count enters every GEE as a standardized covariate and is additionally stress-tested via a post-hoc windowed analysis (50–300 words, 25% stride); surprisal difference remains lower in all clinical groups at every window size (Figure 1b), directly answering the known reduction in speech quantity (SZH 94.9 vs HC 131.8 words, Dunn p<0.01).
Informative metric double dissociation
Methodological agnosticism on ID, honestly reported
Model-free triangulation of compressibility
Weaknesses
Shared-instrument circularity
Estimator-dependent sign flip undermines the unifying claim
Multiplicity and precision gaps
Confounded demographic structure
Very small per-participant data for ID estimation
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks whether psychosis involves a general deficit of information compression in connected speech, integrating two previously separate lines of evidence: increased contextual surprisal (prediction error) and 'shrinkage' of the lexical-semantic embedding space (higher cosine similarity, lower intrinsic dimensionality, ID). Its central conceptual move is to operationalize compression through grammar: surprisal difference — the drop from frequency-based surprisal to context-based (BERT) surprisal — should be attenuated in psychosis if grammar-enabled prediction is impaired. The stated claims are: (1) attenuated surprisal difference in all clinical groups (56 SZH, 33 FEP, 17 SZA) versus 38 healthy controls, independent of word count; (2) reduced ID in SZH (β≈−0.91) and FEP (β≈−1.01), not SZA; (3) syntactic complexity (Ω, MHD, mean depth) predicting surprisal difference; (4) an ID–surprisal-difference association whose sign depends on estimator. The claim architecture is coherent and the introduction positions the work honestly against Sharpe et al.'s predictive-processing account and the authors' own prior cross-linguistic work (refs 2, 10, 13, 37).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is 'surprisal difference' in this psychosis study?
It is the gap between how surprising a word is judged from its raw frequency alone and how surprising it is when a contextual language model (Turkish BERT) also accounts for grammar and sentence context. A smaller difference means context reduces prediction error less, which the authors interpret as weaker information compression through grammar in psychosis.
Did all psychosis subgroups show the same pattern?
No. Attenuated surprisal difference appeared in all three clinical groups (chronic schizophrenia, first-episode psychosis, schizoaffective disorder), but reduced intrinsic dimensionality of word embeddings was significant only for chronic schizophrenia (β≈−0.91) and first-episode psychosis (β≈−1.01), not for schizoaffective disorder.
Is the data or code from this paper publicly available?
No. The paper states that dataset access requests should be directed to the clinical collaborator (emre.bora@deu.edu.tr) and no code repository is mentioned. The analysis tooling (wordfreq, Turkish BERT, scikit-dimension, DADApy) is public, but the speech corpus is not.
What are the main limitations of this study?
All language measures derive from a single BERT model; the sign of the ID–surprisal relation flips between estimators (Gride ≈ +0.27 vs 2NN ≈ −0.45); no multiple-comparison correction or confidence intervals are reported; the sample is about 90% male with small word counts (~95–132 per group); and the data are available only on request.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are available only as captions in the extracted text; exact coefficients and p-values for Figure 1a/1c and Figure 4 could not be verified.
- Supplementary Materials (full GEE formulas, TTR analyses, Figure S5a stability checks) are referenced but not included in the extract.
- The abstract is truncated mid-sentence in the source text.
- GEE working-correlation structure and most exact p-values are not stated in the main text, limiting the statistical audit.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.