Apollo Restore: A Foundation LLM for Historical Greek Optimized for Fill-in-the-Middle Restoration of Ancient Greek Texts
Hope McGovern, Anna Dolganov, Samuel Belkadi, Guillaume Kunsch, Dimitris Vlitas, David A. Smith
cs.CL
Quick verdict
First 24B FIM model for Ancient Greek gap restoration; SOTA claims over Ithaca/Cullhed look credible, but no significance tests and some figures provisional.
Summary
The paper introduces Apollo Restore, a 24-billion-parameter decoder fine-tuned from Mistral Small that restores lacunae in fragmentary Ancient Greek texts using a fill-in-the-middle objective, removing prior systems' dependence on oracle knowledge of gap length. Training combines documentary papyri (DDB), literary papyri (DCLP), inscriptions (I.PHI), and a ~600M-word open Greek corpus, with synthetic gaps sampled uniformly over 1-25 characters and 15% of length hints deliberately corrupted. On short gaps (1-10 characters) under the prior protocol it reaches top-20 accuracy of 80.6%/54.6%/61.0% on documentary papyri, literary papyri, and inscriptions, exceeding Ithaca and Cullhed by 1.6x/2.6x/1.4x; under the authors' new length-balanced metric the margins grow to 2.3x/3.5x/1.6x. It also beats frontier closed models (Claude Opus 4.8, GPT-5.1) on a 200-gap subsample. A blind study with 20 experts preferred Apollo Restore over the strongest baseline (81% head-to-head on documentary papyri), and its reading of Philodemus' P.Herc. 1667 improves the published edition. Limitations include the absence of statistical significance testing, a composite 77% human-parity figure not directly measured, baseline comparisons confounded by scale and data, some provisional figures, and a single literary-papyrus annotator.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Apollo Restore is the first large-scale decoder LLM for historical Greek, trained on ~600M words of Greek (antiquity to 1900) plus DDB, DCLP, and I.PHI transcriptions.
- Reframing restoration as fill-in-the-middle removes the length-oracle dependency while degrading gracefully with wrong or absent hints (W.T1 0.432 exact to 0.332 no hint on DDB, Table 4).
- Prior evaluation protocols inflate scores via short-gap bias; under length-balanced metrics Apollo Restore's Top-20 advantage over the strongest baselines grows to 2.3x/3.5x/1.6x (Table 3).
- In a blind study, 20 experts preferred Apollo Restore to the strongest baseline (81% tie-excluded win rate on documentary papyri), and its reading of P.Herc. 1667 improves the published edition (Appendix A.3).
- Apollo Restore beats frontier closed models on DDB restoration: best closed model Claude Opus 4.8 at 0.151 Top-1 (exact hint) vs 0.447 for Apollo on a 200-gap subsample (Table 5).
Strengths
Task reformulation removes a genuine research blocker
FIM via suffix-prefix-middle reordering with loss only on the middle span (§3.1) eliminates the length-oracle dependency of Ithaca and Cullhed; delta-len near zero (Table 3: -0.6 DDB, +0.3 IG) vs Cullhed's +1.5 over-generation, and no-hint mode retains 0.332 of 0.432 W.T1 on DDB (Table 4).
Dual-protocol reporting is transparent and the metric critique is quantified
Tables 3 and 12 report unweighted and uniform-weighted metrics side by side; §4.1 shows DDB Top-1 margin moving from 2.0x (0.603 vs 0.302) to 2.8x (0.432 vs 0.155), making the short-gap-bias argument (69% of test gaps are 1-3 characters) auditable rather than rhetorical.
Adversarial, well-designed human evaluation
Cross-collection generalization test for inscriptions
Full training disclosure and honest negative results
Weaknesses
No inferential statistics anywhere
Baseline comparison confounded by scale and data volume
Headline human-parity figure is a derived composite
Provisional figures and internal inconsistency
Uniform-weighted metric aligns with the authors' own training distribution
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Apollo Restore (arXiv:2609.22455v2) is a 24B decoder fine-tuned from Mistral Small Base 2503 with a fill-in-the-middle (FIM) objective for restoring lacunae in Ancient Greek documentary papyri (DDB), literary papyri (DCLP), and stone inscriptions (IG). The paper claims four firsts: the first large-scale decoder model for historical Greek, the first length-hint-free restorer for the domain (suffix-prefix-middle packing, §3.1), a corrected length-balanced evaluation framework (§4.1), and the first AI-assisted restoration of a Herculaneum papyrus (P.Herc. 1667, Appendix A.3). Headline numbers: top-20 accuracy of 80.6%/54.6%/61.0% under the prior 1-10-character protocol (1.6x/2.6x/1.4x over Ithaca and Cullhed), growing to 2.3x/3.5x/1.6x under the authors' uniform-weighted metric (Table 3), plus a 20-expert blind preference study and a frontier-model comparison (Table 5).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Apollo Restore?
A 24-billion-parameter language model fine-tuned from Mistral AI's Mistral Small to restore missing text (lacunae) in fragmentary Ancient Greek papyri and stone inscriptions. It uses a fill-in-the-middle objective, so it generates the missing span directly from surrounding context without needing to be told how many characters are missing.
How much better is Apollo Restore than Ithaca?
On short gaps of up to 10 characters its top-20 accuracy is 1.6x (documentary papyri), 2.6x (literary papyri) and 1.4x (inscriptions) the best published baseline's; under the paper's new length-balanced metric the margins grow to 2.3x/3.5x/1.6x, mainly because baselines collapse on gaps longer than roughly 8-14 characters.
Did human experts prefer Apollo Restore's restorations?
Yes. In a blind study, 20 papyrologists, epigraphists and philologists chose Apollo Restore's readings over the strongest baseline's in 67-81% of head-to-head judgments (ties excluded, depending on corpus). By the paper's derived estimate, its reading is at least as good as a human editor's in about 77% of documentary-papyrus cases, though this figure combines two different measurements.
Are the Apollo Restore model weights or code publicly available?
The paper states that the weights, data, and evaluation code will be released, but the reviewed preprint does not include a link or confirm that they are available yet. Training used access-restricted I.PHI data for inscriptions, which is not redistributed.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-6 could not be inspected; the per-gap-length curves (Fig. 3) and perplexity plot (Fig. 4) are judged from captions and in-text numbers only.
- The paper marks some figures as provisional in the source; these markings are not visible in the extracted text, so the affected table entries could not be identified.
- Review is based on the extracted preprint text; released weights and code could not be verified.
- The abstract appears cut off mid-sentence in the source ('Apollo...'), but the full body and Appendices A-I are present.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.