arXiv paper2026-09-21· computational biology (bio-NLP)· empirical study

QLoRA Fine-Tuning of Ministral LLM for Sequence-to-Function Protein Annotation

cs.CL · cs.AI · cs.NE · q-bio.QM

Quick verdict

QLoRA-fine-tunes a 3B LLM for protein annotation; transparent analysis, but leaky random splits, no baselines and an unvalidated LLM judge cap the claims.

Summary

The paper formulates protein functional annotation as sequence-to-text generation: a 3B Ministral base model, fine-tuned with QLoRA (4-bit NF4, rank-64 adapters on all attention and MLP projections; 135M trainable parameters, 3.39%) on 67,553 UniProtKB/Swiss-Prot pairs from 34 organisms, must output the source organism plus a curator-style FUNCTION description as JSON. Evaluation uses an 'LLM-as-expert' protocol (a GPT model prompted as a senior curator with an explicit rubric, rationale-before-score and structured decoding) on 1,444 held-out proteins. Results: 40.4% organism accuracy; function quality mean 2.72/10, median 1.0; 19.9% scored at least 7 and 12.7% judged biologically equivalent (score 10); joint success only 8.7%. Scores are bimodal, and the authors quantify fabricated provenance: 96.9% of predictions citing PubMed IDs cite at least one wrong identifier, even among top-scoring outputs. Main limitations, most acknowledged by the authors: a random train/test split that does not control homology (so the high-scoring tail likely reflects memorized near-duplicates), no baselines or ablations, an LLM judge not validated against human curators, a single run without confidence intervals, and an unexplained gap between the 2,090-example test split and the 1,444 proteins actually graded. The paper is candid throughout and provides complete code listings for replication.

Review complete

Assessment at a glance

computational biology (bio-NLP)

risk flags identified

● high 2

● medium 3

● low 1

Reproducibility readinesshigh

Code & data availableyes

Review confidencehigh

AI text signal0/20 passages flagged

Deep research viableCode-reproducibleNeeds GPU

4 strengths · 4 weaknesses · 5 author questions

AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.

Levels reflect verification complexity — not a quality score. Evolve never rates your paper.

Key claims under review

A QLoRA-fine-tuned 3B general-purpose LLM given only the raw spaced amino-acid sequence produces Swiss-Prot-equivalent function annotations for 12.7% and usable (≥7/10) annotations for 19.9% of 1,444 held-out proteins, with 40.4% organism accuracy (Table 2).
The score distribution is bimodal: the model either recovers the curated function almost exactly or emits a fluent annotation for a different protein, making confident hallucination the dominant failure mode (Section 5.2).
The model fabricates provenance: 96.9% of predictions citing PubMed identifiers cite at least one ID absent from the reference, including 80.0% of score-10 predictions with citations (Section 5.3).
Sequence-only generative annotation via QLoRA fits on one 80GB GPU in ~7.2 hours with 3.39% trainable parameters, making the pipeline reproducible by a single researcher (Table 1, Section 5.5).
Practical use mandates an external verification layer; generated annotations are curation drafts, never evidence, and evidence tags should be removed from training targets (Section 5.5).

Strengths

Quantified hallucination forensics
Section 5.3 measures fabricated provenance precisely: 87.0% of predictions carry an ECO tag (vs 88.2% of references), 96.9% of citing predictions include at least one PubMed ID absent from the reference, and 80.0% of score-10 predictions with citations are still wrong — a genuinely useful characterization of citation fabrication.
Exceptional reproducibility detail
Appendices A.1-A.3 give the exact UniProt REST query and date (2026-09-19), filtering thresholds as code, prompt/target construction with loss masking, full QLoRA and TrainingArguments (seed 42), greedy generation code, and the verbatim grader system prompt with its Pydantic schema; Table 1 lists all hyperparameters including the 7.18 h / A100-80GB budget.
Honest failure-mode characterization
Carefully engineered evaluation protocol

Weaknesses

No baselines or ablations whatsoever
Homology leakage uncontrolled
Unvalidated LLM judge rewarding verbatim recall
Reporting gaps and designed-in fabrication

Full Review Report

$1.99 · one-time

Unlock the complete preliminary analysis Evolve produced for this paper.

Report preview

Scope & Claims

The paper reformulates protein functional annotation as sequence-to-text generation. Instead of multi-label classification over the Gene Ontology, a compact general-purpose decoder LLM (Ministral 3 3B base) reads a raw, whitespace-spaced amino-acid string and emits a JSON object containing the source organism and a curator-style UniProt FUNCTION description (Sections 1, 4.2). The authors claim three things: (i) QLoRA adaptation on one 80 GB GPU in 7.18 hours produces annotations judged biologically equivalent to Swiss-Prot curation for 12.7% of held-out proteins and usable (score ≥7/10) for 19.9% (Table 2); (ii) organism accuracy reaches 40.4%; (iii) the characteristic failure is confident hallucination rather than vagueness, including fabricated provenance — 96.9% of predictions citing PubMed identifiers cite at least one wrong ID (Sections 5.2-5.3). Claims are mostly hedged appropriately, and Section 5.5 contains unusually candid self-criticism, which strengthens rather than weakens the paper's credibility.

Methodology Assessment

Data (Section 3, Appendix A.1): 350,000 reviewed Swiss-Prot entries retrieved 2026-09-19; filtered to 34 organisms with ≥500 entries each, ≤700 residues, ≤850-character function comments, 20-letter alphabet, duplicates and uncharacterized targets removed — 72,873 pairs; random 97/3 split (67,553 train / 2,090 test). Training (Section 4, Table 1): QLoRA with 4-bit NF4 and double quantization, LoRA r=64, α=128 on all seven attention and MLP projections (135,004,160 trainable parameters, 3.39%), 3 epochs, effective batch 32, lr 1e-4 with cosine schedule, paged 8-bit Ada

The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.

All 4 strengths & 4 weaknesses, analyzed in detail
6 risk flags with severity, section & reviewer questions
Methods & statistical evidence assessment
5 questions a peer reviewer would ask
Full six-dimension evaluation report & recommendations

One-time purchase · this paper only

One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.

Frequently asked questions

How accurate is the protein annotation model in this paper?

On 1,444 held-out Swiss-Prot proteins, it identified the source organism for 40.4%. Function descriptions scored by an LLM grader averaged 2.72/10 (median 1.0); 19.9% reached a usable level (score ≥7), 12.7% were judged biologically equivalent to the curated annotation, and only 8.7% had both the organism and a usable function correct.

What are the main limitations of this study?

The random train/test split does not control homology, so the best results likely reflect memorized near-duplicates rather than true generalization; there are no baselines or ablations; the LLM judge was never validated against human curators; all numbers come from a single training run without confidence intervals; and only 1,444 of the 2,090 test proteins were actually graded, without explanation.

Does the model hallucinate citations?

Yes. Because training targets end with UniProt evidence tags, the model reproduces the format: 96.9% of predictions that cite PubMed identifiers cite at least one ID absent from the true reference — even among predictions judged biologically perfect. The authors recommend removing evidence tags from training data entirely.

Can I reproduce this experiment on my own hardware?

Largely yes: training uses one 80GB A100 for about 7.2 hours with QLoRA 4-bit quantization, and the paper's appendices contain complete code for the UniProt data download, filtering, prompt construction, QLoRA training configuration (seed 42), greedy generation, and the exact grading prompt used with the GPT API.

About this review

This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.

Figure 1 (loss curves and score histogram) is available only as text embedded in the extraction; curve values were taken from the caption's stated numbers, not read off plots.
The grader model name 'gpt-5.6-terra' cannot be verified externally and its exact behavior is version-dependent.
Appendix code listings are reproduced in the extraction and assumed complete; minor truncation of listing content cannot be ruled out.

Go deeper on this paper

Wrote a paper? Stress-test it before reviewers do.

Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.

Scan your paper