From Symbolic Perception to Logical Deduction: A Framework for Guiding Language Models in Geometric Reasoning
Weichen Dai, Rafael Medeiros Cabral, Ziyi Shou, Yan Cao, Xin Shen, Dongcai Lu, Yi Zhou
cs.AI · cs.CL
Quick verdict
Claims pure LLM + parser + symbolic guidance rivals Gemini 2.5 Pro in geometry; wins thin (trails on L1), weak stats, no code — promising but under-supported.
Summary
The paper argues that a pure text LLM (DeepSeek-R1), augmented with two specialized modules, can rival leading multimodal models on plane geometry. A Geometric Vision Parser converts diagrams into symbolic form using a fine-tuned YOLOv11 pose model for line segments, YOLO detection for symbols/text, Hough transforms for circles, TexTeller OCR, and greedy feasibility-constrained symbol-to-primitive assignment, followed by rule-based textual alignment where text-derived properties override unreliable visual reads. A Theorem Reasoning Module performs literal expansion, rule-based fact deduction with permutation pruning, and relevance filtering; the filtered deductions are injected into the LLM prompt alongside parsed relations and reference-only pixel coordinates. The authors also introduce ZhongkaoGeo, a three-tier benchmark (89/83/105 problems) from 2023–2025 Chinese Zhongkao exams to reduce contamination risk. Their method scores 92.13/74.30 accuracy on L1/L2 and 88.4% mean scoring rate on L3, versus Gemini 2.5 Pro at 94.38/73.49 and 87.2%; with three-run majority voting it reaches 93.26/78.31. Gains on L2 and L3 are under 1.2 points and the method trails Gemini on L1, so the introduction's "surpass" claim is overstated. Statistical testing is absent, no code or benchmark release is mentioned, the parser is compared only against Qwen-VL-2.5-72B, and the theorem module's effect is shown only via case studies.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A pure LLM (DeepSeek-R1) equipped with a Geometric Vision Parser and Symbolic Solver achieves performance comparable to Gemini 2.5 Pro on complex plane geometry (92.13/74.30 accuracy on L1/L2, 88.4% mean scoring rate on L3, vs 94.38/73.49/87.2%).
- The framework is claimed to be the first to enable a pure LLM to match and surpass leading LMMs on complex, unseen plane geometry problems.
- The ZhongkaoGeo three-tier benchmark (89/83/105 problems from 2023–2025 Chinese Zhongkao exams) provides a more difficult and contamination-resistant testbed than GeoQA or Geometry3K.
- The Symbolic Solver's filtered theorem deductions steer the LLM away from brute-force coordinate calculations toward human-readable deductive solutions (Figures 5/6).
- The diagram parser substantially outperforms Qwen-VL-2.5-72B at recovering implicit geometric relations (all-relations-correct 81 vs 23 of 105 on L3, Table 3).
Strengths
Timely, contamination-aware benchmark design
ZhongkaoGeo's three tiers (89/83/105 problems from 2023-and-earlier, 2024–early 2025, and official 2025 exams) address the saturation and leakage problems the paper documents for GeoQA (avg 1.96 reasoning steps) and Geometry3K, and the multi-sub-question structure (avg 1.281/1.807 per problem, Table 1) better matches real exam difficulty.
Substantial, measurable parsing improvement
Table 3 shows the parser recovers implicit relations far better than a prompted Qwen-VL-2.5-72B on L3: collinearity 85 vs 45, angular relations 86 vs 46, and all-relations-correct 81 vs 23 out of 105 problems — evidence the symbolic representation is genuinely more faithful.
Sensible guard against schematic-diagram noise
Transparent evaluation protocol
Compelling failure-mode analysis
Weaknesses
Headline overclaim contradicted by own results
No inferential statistics
Missing ablations and competitor baselines
Reproducibility not addressed
Internal inconsistencies
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This USTC paper (arXiv 2609.10335, AAAI-2026 copyright notice) targets plane-geometry problem solving and makes three contributions: (1) a framework that—for the first time, per the authors—lets a pure LLM (DeepSeek-R1) match a frontier LMM (Gemini 2.5 Pro) on complex, unseen geometry; (2) ZhongkaoGeo, a three-tier benchmark (89/83/105 problems) from 2023–2025 Chinese Zhongkao exams designed for difficulty and contamination resistance; (3) a Geometric Vision Parser plus Symbolic Solver that steer LLMs toward human-readable deductive solutions. The introduction escalates the abstract's "comparable" to "not only match but surpass," a claim the results do not uniformly support (see Statistical Evidence).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ZhongkaoGeo?
A three-tier benchmark of Chinese middle-school (Zhongkao) plane-geometry problems introduced by this paper: 89 problems (L1, from 2023 and earlier), 83 problems (L2, from 2024–early 2025), and 105 problems (L3, official 2025 exams from provincial capitals and major municipalities). Because the items are recent, the authors argue models are unlikely to have memorized them during pre-training, and the problems are harder and more multi-part than older benchmarks like GeoQA and Geometry3K.
Does the proposed method beat Gemini 2.5 Pro?
Not consistently. On benchmark L1 it trails Gemini (92.13% vs 94.38% accuracy, and still 93.26% with 3-run majority voting). It wins narrowly on L2 (74.30% vs 73.49%; 78.31% with majority voting) and on the L3 mean scoring rate (88.4% vs 87.2%). 'Comparable' is a fair description; the introduction's 'surpass' goes beyond the evidence, since no statistical significance tests are reported.
Why use a pure language model instead of a multimodal model for geometry?
The authors argue that large multimodal models are computationally expensive and their reasoning opaque. Instead, a text-only LLM (DeepSeek-R1) is fed a symbolic description of the diagram produced by a dedicated vision parser, plus automatically deduced geometric facts from a theorem engine. This grounds the LLM in verified facts, reduces hallucination, and yields step-by-step solutions that read like human geometry proofs rather than coordinate-algebra brute force.
Is the code or dataset available?
The paper does not mention releasing its own code, the fine-tuned YOLO parser models, or the ZhongkaoGeo benchmark. The only link provided is to TexTeller, a third-party open-source OCR tool used inside the pipeline. Reproducing the results independently would therefore require rebuilding the parsers and re-collecting exam problems.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–6 are embedded images: only values quoted in the surrounding text and captions (e.g., the L3 means 88.4% vs 87.2% and standard deviations 9.4 vs 14.4) are available, so per-province chart values and figure details could not be independently verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.