AI in Science: Early Insights
Mihai Codreanu, Alex Imas, Juan Mateos-Garcia, Joseph Emmens, Evalyne Muiruri, Arthur Turrell, Julian Jacobs, Atoosa Kasirzadeh, et al.
econ.GN · cs.AI
Quick verdict
Google-led study of Gemini logs, 2,690 specialist models and 637 scientists finds broad AI adoption and complementarity; rich but descriptive, vendor-tilted.
Summary
This Google/DeepMind-led paper offers a first large-scale empirical snapshot of AI use in science by triangulating three novel data sources mapped onto the MIT FutureTech scientific task taxonomy and OpenAlex fields: ~15 million anonymized Gemini interactions (filtered to ~360,000 science interactions via a three-stage classifier), an inventory of 2,690 specialized AI-for-science models built by agentic search plus Epoch AI data, and a survey of 637 US/UK scientists. Four findings emerge: (1) broad adoption—scientists are over-represented in Gemini usage (SOC 19 occupations 2.7× their employment share); (2) LLMs and specialized models act as complements, with the log-log elasticity between their task shares falling from ~0.6 at the most aggregate level to ~0.2 at the finest; (3) surveyed scientists report ~6.9 hours/week average net time savings, mostly reinvested in research; and (4) bottlenecks shift downstream—44% report their rate-limiting step moved to experimentation/validation, 41% report a growing backlog of untested hypotheses, and 49% say AI pushes them toward safer questions. Main limitations: proprietary single-vendor log data with enterprise traffic excluded; a non-probability survey sample prone to selection bias; LLM-based classification with substantial error at granular task levels; and entirely descriptive, correlational evidence. No code or data release is mentioned.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Scientific occupations use LLMs at roughly 2.7 times their US employment share (SOC 19), and nearly half of surveyed scientists report using AI every day.
- LLMs and specialized AI models are complements with a division of labor: the log-log elasticity of their task shares falls from ~0.6 (aggregate) to ~0.2 (Level-3), slightly negative when zero-share tasks are included.
- Surveyed scientists report average net time savings of ~6.9 hours per week from AI, reinvested primarily in additional research output.
- AI shifts research bottlenecks downstream: 44% report their rate-limiting phase moved toward experimentation/validation, 41% report a growing backlog of untested hypotheses, and 49% say AI pushes them toward safer, more incremental questions (vs 28% toward riskier ones).
Strengths
Unprecedented real-time telemetry
Section 3 analyzes ~15M anonymized Gemini interactions filtered to ~360K science interactions via a three-stage pipeline with reported validation (93.7% work classifier, 96%/97% science classifier on synthetic positive/negative controls, App. 1) — the only large-scale behavioral data on day-to-day AI use in science, and externally corroborated by the Anthropic Economic Index (SOC 19 at ~4.5x, fn.8).
Common-metric triangulation
Placing conversational logs, a 2,690-model inventory and survey perceptions on the same MIT FutureTech taxonomy (12 Level-1 / 114 Level-2 / 2,433 Level-3 areas) plus OpenAlex fields is what makes the novel division-of-labor analysis of Section 4.3 and Figure 8 possible; no prior scientometric dataset supports this comparison.
Candid validation and limitations reporting
Policy-relevant geographic findings
Weaknesses
Vendor monoculture and self-interest
Survey evidence is self-selected
Classification error drives the core finding
Citation results partly by construction
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This paper inaugurates a Google/Google DeepMind research program on AI's impact on science, triangulating three datasets mapped onto one common metric — the MIT FutureTech Scientific Task Taxonomy (12 Level-1 / 114 Level-2 / 2,433 Level-3 areas, derived from 3.8M Lightcast job adverts) and 217 OpenAlex subfields covering 92% of works: (1) Google ATLAS 1.0 telemetry, ~15M anonymized Gemini interactions (April 2026 sample; enterprise and paid-API traffic excluded, fn.4), reduced via a work classifier (93.7% synthetic accuracy), a six-SOC filter, and a new Frascati-based science classifier (96%/97% on synthetic positive/negative controls) to ~360,000 science interactions; (2) an inventory of 2,690 post-2012 specialized AI-for-science models with papers and official code repos (from 5,501 collected), yielding 4,475 tasks extracted from publication abstracts; (3) a screened, non-probability survey of 637 US/UK scientists (More in Common panel, 27 Jul–11 Aug 2026; 379 US / 258 UK; 356 senior, 234 mid-career, 47 early-career). Four headline claims: broad adoption (SOC 19 usage at 2.7× the employment share; LLM use in 195 of 217 subfields; ~half of scientists use AI daily); LLM–specialist complementarity (log-log task-share elasticity 0.6 → 0.5 → 0.2 from aggregate to Level-3, slightly negative including zeros); ~6.9 h/week average self-reported net savings (73% report net savings) mostly reinvested in research; and a downstream bottleneck shift (44% moved downstream vs 14% upstream; 41% report a growing untested-hypothesis backlog; 89% of time-savers spend >10% of savings verifying outputs and 46% spend >25%; 49% drift toward safer questions vs 28% riskier).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
How much time do scientists save by using AI, according to this paper?
In a survey of 637 US and UK scientists, about 73% reported net time savings averaging roughly 6.9 hours per week, with most of the saved time reinvested into more research. This is a self-reported figure from a non-probability online panel, so the authors themselves flag that it is likely an overestimate.
Are chatbots like Gemini replacing specialized scientific AI models such as AlphaFold?
No — the paper argues the two are complements. General LLMs are used for writing, coding, literature review and general data analysis, while specialized models handle domain-specific prediction, data generation and simulation (e.g., protein structures, crystal stability). Their task-overlap elasticity falls from ~0.6 at coarse levels to ~0.2 at the finest task level, indicating little substitution.
Is AI making scientific research more incremental and less risky?
Possibly, per self-reports: 49% of surveyed scientists said AI pushes them toward safer, more incremental questions where benchmarks and data are established, versus 28% who take on riskier questions. The authors frame this as a potential 'streetlight effect', but it is a perception measure, not a behavioral one.
Can this study be replicated — is its data or code public?
No. The Gemini interaction telemetry is proprietary (and excludes enterprise traffic), the scientist survey was run on a commercial panel, and the paper mentions no public release of the model inventory, survey microdata, or classification code, so independent replication is currently impossible.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The extraction omits several passages (parts of Section 2.1, end of Section 3.3, the Section 5.1-5.2 seam, end of Section 6.2, and the tail of Appendix 1, which cuts off mid-sentence on task/field accuracy validation).
- All figures (1-14) are unavailable as images; assessments of visual evidence rest on their text notes alone.
- Table 1 and other tabular material are present only as captions/prose; underlying values could not be checked cell-by-cell.
- Findings are dated to an April-July 2026 snapshot; the paper itself warns the LLM/specialist division of labor may change quickly.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.