CRITICS - Critical Science Without Borders: Language Models to Promote Critical Thinking in Science Education
Rodrigo Agerri, Itziar Aldabe, Elena Cabrio, Mark Cieliebak, Jan Deriu, Mariana Flores, Jurgita Kapociute-Dzikiene, Dovile Kuiziniene, et al.
cs.CL · cs.AI
Quick verdict
CRITICS project paper: a well-grounded proposal uniting LLM-based scientific MT with critical-thinking education; promising plan, but no results yet.
Summary
This paper is a project description (extended abstract) for CRITICS, a three-year CHIST-ERA IV Cofund 2025 project coordinating HiTZ/EHU (Spain) with CNRS/Université Côte d'Azur (France), Vytautas Magnus University (Lithuania), and ZHAW (Switzerland). It proposes an integrated pipeline combining LLM-based machine translation of scientific documents with science-education technology. Three components are planned: (1) document-level MT of scholarly texts with adapter fine-tuning, terminology glossaries, and preservation of citations, equations and layout, targeting English–Basque, English–Lithuanian, English–French, and English–German in natural sciences and AI; (2) a Teaching-Learning Sequence generator that transforms translated literature into curriculum-aligned resources via argumentation-graph extraction; and (3) an automatic assessor producing Critical Questions and competency-based (PISA-style) feedback, with argument mining to detect fallacies and misconceptions. Evaluation will combine COMET/ChrF-style metrics, QGEval, calibrated LLM-as-a-judge, and human expert assessment, targeting TRL 5. Data sources include bioRxiv, PubMed Central, chemRxiv, arXiv, the ACL Anthology, science-education repositories, and 500 in-house competency assessments to be anonymized and released. As a proposal, the paper reports no experimental results; all contributions are future commitments, and the feasibility of closing the translation-assessment improvement loop within three years remains untested.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- LLM-based MT with adapter fine-tuning, terminology glossaries, and structure preservation can surpass conventional MT for document-level scholarly translation into low-resource languages such as Basque and Lithuanian (Section 2.1).
- Translated scientific literature can be automatically transformed into curriculum-aligned Teaching-Learning Sequences via argumentation-graph extraction (Section 1, Component 2).
- CRITICS will provide, for the first time, textual student feedback via automatically generated Critical Questions aligned with competency-based assessment criteria in Basque, French, German, and Lithuanian (Section 2.3).
- Calibrated LLM-as-a-judge evaluation can be made comparable to human judgments for scientific MT and critical-question/feedback generation (Objective 4, Section 2.4).
Strengths
Well-motivated problem with typologically informed language choice
Section 1 and Section 3 target Basque (agglutinative language isolate) and Lithuanian (inflectional) — genuinely underserved in scholarly MT — alongside French/German for comparison; the science-access motivation is tied to concrete linguistic inequity rather than generic multilingualism.
Consortium with directly reusable prior art
Each planned component maps onto existing consortium artifacts: the Latxa Basque LLM and circa 1.4B-token Basque corpus [13], the CaSiMedicos-Arg multilingual medical argumentation dataset [31], the first public Critical Questions benchmark [24], and metric-favoritism tooling (FAVI-score [34]).
Concrete, open data plan
Honest pre-registration of evaluation pitfalls
Weaknesses
Zero empirical support
Core novelty mechanisms underspecified
Thin and possibly mismatched education data
Aggressive scope for 36 months
Drafting inconsistencies
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
CRITICS (arXiv 2609.13942, SEPLN 2026) is not an empirical paper but a project description — an extended abstract for a three-year CHIST-ERA IV Cofund 2025 project ('Science in your own language') coordinated by HiTZ/EHU with CNRS/Université Côte d'Azur, Vytautas Magnus University, and ZHAW. It promises an integrated pipeline of three components (Section 1, Figure 1): (1) LLM-based document-level machine translation of scientific texts with adapter fine-tuning, terminology glossaries, and preservation of citations, equations and layout (Section 2.1), targeting English–Basque, English–Lithuanian, English–French, and English–German in natural sciences and AI; (2) a Teaching-Learning-Sequence (TLS) generator that extracts argumentation graphs from translated corpora to build curriculum-aligned resources; and (3) an LLM assessor generating Critical Questions and feedback aligned with competency-based, PISA-style frameworks, with argument mining to detect fallacies and misconceptions (Sections 2.2-2.3). Central claims are prospective: surpassing conventional MT for low-resource scholarly translation, first-ever Critical-Questions-based automatic student feedback in four languages, and calibrated LLM-as-a-judge evaluation comparable to human ratings (Section 2.4, Objective 4). TRL 5 is the stated target.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the CRITICS project about?
CRITICS is a three-year European research project (CHIST-ERA IV Cofund 2025, call 'Science in your own language') coordinated by the HiTZ center of the University of the Basque Country, with partners in France (CNRS/Université Côte d'Azur), Lithuania (Vytautas Magnus University), and Switzerland (ZHAW). It combines LLM-based machine translation of scientific papers with educational technology to turn translated research into teaching materials and automatic student assessment in Basque, Lithuanian, French, and German.
Does this paper report experimental results?
No. It is a project-description (proposal) paper: it outlines motivation, related work, four objectives, data sources, and a work plan of six work packages, but contains no experiments, baselines, or quantitative results. All claimed advances are future commitments targeting TRL 5 over three years.
Which languages does CRITICS target and why?
English is the source language; targets are Basque and Lithuanian (low-resource) plus French and German (medium-resource). Basque is a language isolate with little scientific material available to students, which motivates the project's science-accessibility and critical-thinking goals; the language set is deliberately typologically diverse (agglutinative, inflectional, Romance, Germanic).
What tools will the CRITICS project deliver?
Two prototypes are planned: a Parallel Translator/Reader of scientific documents (document-level scholarly MT preserving citations, equations, and layout) and a Teacher Evaluator that generates critical questions and feedback for competency-based, PISA-style assessments — both validated to TRL 5 with science-education experts, using data from bioRxiv, PubMed Central, arXiv, the ACL Anthology, and education repositories.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The paper is a project description with no results sections; all assessments of claims are assessments of plans, not of evidence.
- Figure 1 (the three-component architecture) is only available as a caption in the extracted text; component interconnections could not be verified.
- The supplied metadata author list omits five authors present in the full text (Rico, Ruiz-González, Soroa, Vaškevičius, Villata).
- The SEPLN extended-abstract format inherently limits methodological detail; some under-specification is a genre property, not necessarily an author omission.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.