LLM-Based Schema-Aware Split Learning for Privacy-Preserving Mental Distress Prediction Across Heterogeneous Surveys
Md Khalid Syfullah, Alvi Ataur Khalil
cs.LG · cs.AI · cs.CR
Quick verdict
A split-learning framework that serializes heterogeneous mental-health surveys into text and fine-tunes a split LoRA-adapted LLaMA-3.2-3B averages 0.708 ANLS and beats FedAvg in 8/9 settings, but its privacy claim is only data locality and its metric is lenient.
Summary
This paper proposes a schema-aware split learning (SL) framework that lets institutions collaboratively train a mental-distress predictor without sharing raw survey responses. Each cleaned survey record is serialized into a natural-language instruction sample (feature names, ordinal responses rendered as text, explicit missing-value markers, and a schema header), so four heterogeneous public surveys — OSMI Mental Health in Tech (4,701 rows, 133 columns), DASS-42 (~40,000 rows, 172 columns), Student Depression (27,901 rows, 18 columns) and a general Mental Health dataset (292,364 rows, 17 columns) — are mapped into one shared semantic representation. A LoRA-adapted LLaMA-3.2-3B-Instruct (rank 16, alpha 32 on q/k/v/o/gate/up/down projections) is split across the client-server boundary: clients keep the tokenizer, a LoRA-adapted embedding and a ~100K-parameter, zero-initialized residual projection head, and transmit only cut-layer activations; the server holds the frozen-base transformer trunk and LM head and returns cut gradients. Across a 3x3 grid crossing schema heterogeneity with volume imbalance, with 10 clients and at most 2,000 training samples, SL averages 0.708 ANLS (range 0.660-0.759) and exceeds a sample-weighted FedAvg baseline (0.687) in eight of nine settings, while per-client cost drops from ~158 TFLOPs and 15-25 GB to ~0.08 TFLOPs and ~1.3 GB. A generalist model trained on 10,000 merged samples averages 0.662 ANLS. Main limitations: no significance testing or seed variance, a lenient ANLS metric with 0.5 partial credit for adjacent severity levels, an FT-Transformer reference reaching 0.852 accuracy, and a privacy argument resting purely on data locality with no defence against activation inversion.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Serializing heterogeneous survey records into natural-language instruction samples lets institutions with different schemas share one semantic representation for mental distress prediction.
- Splitting a LoRA-adapted LLaMA-3.2-3B keeps raw responses and identifiers local while reducing per-client compute by roughly three orders of magnitude relative to federated learning.
- Across a 3x3 grid of schema heterogeneity and volume imbalance the framework averages 0.708 ANLS and beats a FedAvg baseline (0.687) in eight of nine settings.
- The approach generalizes to unseen survey datasets: a generalist trained on 10,000 merged samples averages 0.662 ANLS and beats zero- and few-shot prompting on every dataset.
- Schema-aware serialization plus LoRA makes the framework data-efficient, reaching 0.708 ANLS with only 2,000 training samples.
Strengths
Clean respondent-level leakage control in the serialization pipeline
Section IV-A/B convert each respondent row into one instruction sample per outcome, drop the predicted outcome from the input, remove byte-identical (input, output) pairs before splitting, and enforce respondent-disjoint train/val/test and client shards — a real and often-missed safeguard when one respondent yields several training samples.
A well-constructed 3x3 evaluation grid that separates schema heterogeneity from volume imbalance
Section IV-C crosses three schema levels (homogeneous / moderate / fully heterogeneous) with three volume-imbalance levels (symmetric 10% per client, moderate power-law, high with 60% in one client), and Table IV shows performance is stable: average ANLS shifts by under 0.01 across volume levels while peaking at 0.759 under homogeneous schemas.
Honest scoping of the privacy threat model
Concrete, quantified systems evidence for the client-cost claim
Multiple independent generalization checks
Weaknesses
No significance testing, confidence intervals, or seed variance anywhere in the paper
The ANLS metric is lenient and not clinically interpretable
The FT-Transformer reference undercuts the practical value of the LLM approach
The privacy architecture has an unresolved inconsistency about labels
Data-scale and subsampling details are underspecified
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets a real and well-motivated gap: institutions hold mental-health survey data that is both privacy-sensitive (HIPAA, GDPR are cited in Section I) and structurally incompatible, because different surveys use different questions, scales and coding. The authors claim no prior work jointly addresses both, and they build a framework out of three ingredients — schema-aware serialization into natural-language instruction samples, a LoRA-adapted LLaMA-3.2-3B as a shared semantic encoder, and split learning (SL) so that raw responses stay local. Five claims are made explicitly: (i) serialization unifies heterogeneous schemas (abstract, Section IV-A); (ii) SL keeps data local at three orders of magnitude less client compute (Table V); (iii) SL averages 0.708 ANLS and beats FL in eight of nine settings (Table IV); (iv) a generalist trained on 10,000 merged samples averages 0.662 and transfers to unseen datasets (Section V-E); (v) the method is data-efficient, reaching its headline number with only 2,000 training samples. The framing in Section I is unusually careful about what is *not* claimed: cut-layer tensors are not asserted to be non-invertible, and differential privacy is scoped out. That honesty about the privacy boundary is worth crediting up front, because it shapes how the rest of the evidence should be read.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does this paper propose for privacy-preserving mental health prediction?
It proposes a split-learning framework in which each institution converts its survey records into natural-language sentences and keeps the raw responses on site. Only a tokenized, lightweight front end (about 3.68M trainable parameters, ~1.3 GB) runs locally; the heavy 3B-parameter language model backbone stays on a server. Clients send only intermediate activations, never the survey answers, so institutions with different questionnaires can train one shared model.
How accurate is the split-learning model compared with federated learning?
Using Average Normalized Levenshtein Similarity (ANLS), the split-learning setup averages 0.708 across nine configurations and beats the federated (FedAvg) baseline's 0.687 in eight of nine configurations, with an average margin of +0.021. The paper reports single runs on 500-sample test sets with no significance testing, so the margin should be read as suggestive rather than established.
Does split learning reduce the computational burden on participating institutions?
Yes, substantially, at least in compute terms. Each split-learning client trains 3.68M parameters at about 0.08 TFLOPs per step and ~1.3 GB of memory, whereas a federated-learning client runs the full model at about 158 TFLOPs per step and 15–25 GB — roughly three orders of magnitude less client-side compute. The paper does not measure communication volume or wall-clock time, listing those as future work.
Is the privacy guarantee in this framework formally protected, e.g. by differential privacy?
No. The guarantee is data locality only: raw responses and identifiers never leave the institution, and only cut-layer activations and gradients are transmitted. The authors explicitly state they do not claim these tensors are non-invertible and that defending against model-inversion or reconstruction attacks (for example with differential privacy or secure aggregation) is outside the scope of this work.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–5 are described in the extracted text only as fragments with axis-less value lists; the numeric readings used here come from the accompanying prose in Sections V-B to V-F and Tables III–V, not from direct inspection of the plots.
- The paper is an arXiv preprint (arXiv:2609.15871v1) with no code or data release statement, so availability could not be verified beyond the text.
- Table IV's cell-level SL and FL values are reconstructed from a compact table layout in the extracted text; the homogeneous/high-volume cell (SL 0.757 vs FL 0.765) is the one place where FL leads, and it should be re-checked against the published table.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.