Accelerating scientific discovery with Co-Scientist

2025 · Paper

---

1. Motivation

LLM research agents to date either execute code on narrow benchmarks (AI Scientist lineage) or generate unverified text. Hypothesis generation - the upstream, language-based part of science - lacked a system that could (a) search hypothesis space systematically rather than sampling once, and (b) be trusted, because LLMs hallucinate novelty and correctness. Co-Scientist targets this gap with a “structured scientific thinking engine” that keeps a human scientist in the loop and grounds its outputs in literature search, domain databases, and ultimately in vitro experiments rather than in an LLM’s self-judgment.¹

---

2. Methodology

2.1 Search Structure

The system performs iterative “generate, debate, evolve” refinement over hypothesis space - explicitly not brute-force enumeration. A Supervisor agent orchestrates an asynchronous task queue, allocating compute among specialized agents, and periodically recomputes summary statistics to reallocate resources across the pipeline:

scientist goal ──► Supervisor (task queue, compute allocation)
        ┌────────┴─────────┬──────────────┬─────────────┐
   Generation          Reflection     Ranking         Evolution
   (4 strategies)      (review        (Elo tournament (grounding,
                        cascade)       + debates)      combination)
        └──────── Meta-review ◄── tournament & review patterns ──┘
                       │ feedback loop
                       └────────► Generation ("research expansion")

Exploration is guided by a Proximity graph over hypotheses plus tournament outcomes. Five default criteria gate all outputs: alignment with the goal, plausibility, novelty, testability, and safety.¹

2.2 State Representation

The searched objects are natural-language hypotheses and research proposals with summaries and categorizations - not code. A persistent context memory stores agent and system states across long-horizon computation (described as an emergent internal model of the research process). The scientist’s goal is parsed into a research-plan configuration, and the system can ingest hundreds of prior publication PDFs.¹

2.3 Generation Mechanism

A Generation agent uses four strategies: (1) literature exploration, building a knowledge base of facts; (2) simulated scientific debates - multi-turn self-critique/self-play; (3) iterative assumptions identification, aggregating conditional reasoning hops into hypotheses; and (4) research expansion, which consumes Meta-review feedback to push into unexplored regions of hypothesis space. An Evolution agent then refines candidates via search-grounded grounding, coherence/feasibility fixes, inspiration from top-ranked hypotheses, and combination of hypotheses - a language-space analog of crossover.¹

2.4 Evaluation & Grounding

Evaluation is a cascade of increasing depth, run by a Reflection agent: initial review → full review with web search and tools → deep verification (decomposing hypotheses into sub-assumptions) → observation review against long-tail experimental results → simulation review (step-wise mechanism simulation) → recurrent, tournament-adaptive review. A Ranking agent runs an Elo tournament (initial rating 1200), with multi-turn scientific debates for top hypotheses to mitigate ordering bias and single-turn pairwise matches lower down. External grounding comes from web search, domain databases (Open Targets, DepMap), private repositories, and specialized models such as AlphaFold - and, at the top of the pyramid, wet-lab in vitro validation.¹

2.5 Memory / Information Flow

Two structures carry information: the persistent context memory, and a Meta-review agent that synthesizes tournament and review patterns into research overviews and feedback, which is fed back into Generation (“research expansion”) and Evolution - a closed reflective loop over the whole search history. Expert feedback in natural language can be injected at any point, including seeding the search with the scientist’s own hypotheses.¹

2.6 Selection & Propagation

Selection is Elo-tournament-based: match scheduling is prioritized by Proximity-graph similarity and by hypothesis novelty/rank; top hypotheses earn debate matches; the Proximity agent clusters near-duplicates to organize exploration. Final candidate selection remains human - e.g., 30 top-ranked acute myeloid leukemia (AML) candidates were narrowed by expert oncologists to 5 for wet-lab testing.¹

2.7 Anti-Slop / Anti-Hallucination Mechanisms

Co-Scientist provides the theme’s cleanest ablation evidence that grounding controls work. A Reflection agent without search tools rated known, published (non-novel) ideas as novel - score 6.14/10 versus 2.38/10 with search - and search raised correctness from 7.4 to 8.46 (GPQA AUC 0.643 → 0.651, Gemini 2.0 Flash backend). Deep verification decomposes hypotheses into sub-assumptions to catch subtle protocol errors; debate prompts reduce positional bias; Meta-review improved correctness-review AUC from 0.521 to 0.597 on a constructed dataset. An Evolution-agent ablation raised GPQA precision from 70.9% to 75.4% and research-hypothesis auto-score from 4.7 to 5.6. Safety filtering and initial ethics screening gate outputs.¹

---

3. Results

• Test-time compute scaling: Elo of top-10 and best hypotheses improves across 10 temporal buckets over 203 research goals (figure-reported trend).¹
• Model comparison: on 15 expert-curated goals, Co-Scientist’s Elo exceeds Gemini 2.0 Pro, Gemini 2.0 Flash Thinking, OpenAI o1, o3-mini-high, DeepSeek-R1, and expert “best guess” hypotheses - caveat: auto-evaluated Elo, not ground truth.¹
• Blinded expert evaluation (11 goals): average preference rank 2.36 (best of compared systems), novelty 3.64/5, impact 3.09/5; LLM-as-judge panels also preferred it on all 15 goals.¹
• AML drug repurposing: from a curated space of 2,300 approved drugs × 34 cancer types, 3 of 5 expert-selected candidates inhibited viability (Binimetinib IC50 as low as 2 nM in AML lines); among 3 autonomously proposed novel candidates, KIRA6 (an IRE1α inhibitor) showed IC50 10 nM in KG-1a cells versus 180 nM in TK6 controls - an 18-fold selectivity window. Synergistic combinations (e.g., Palbociclib + Selinexor) were confirmed in two AML lines.¹
• Liver fibrosis: 3 novel epigenetic targets proposed; 2 drugs showed anti-fibrotic activity in human hepatic organoids without toxicity, including the repurposing opportunity Vorinostat.¹
• AMR mechanism: in ~2 days the system independently recapitulated the then-unpublished cf-PICI/phage-tail interaction mechanism for broad host-range gene transfer, matching a co-timed experimental study (Penadés et al., Cell 2025). Expert time per task was ~3 hours, versus “days and even weeks” of scientist time.¹

---

4. Limitations & Assumptions

The system depends on open-access literature, so paywalled prior art and negative results are systematically missed, with a risk of propagating irreproducible claims. Elo and auto-evaluation are proxies, and the expert evaluations are small ($n$ = 11–15 goals). Wet-lab validations are preliminary viability checks, “not a replacement for rigorous pre-clinical and clinical assessment.” Source code is not publicly available (“not publicly available” per the paper; pseudocode and prompts appear in Supplementary Notes), and validation is primarily on Gemini-2.0 backends. The authors flag risks of bias, homogenization of research directions, and worsening the reproducibility crisis if outputs bypass peer review.¹

---

5. Critical Analysis

• Distinctive strength: It is the only mission-list system whose hypotheses are grounded physically - wet-lab in vitro confirmation, including a blind recapitulation of an unpublished mechanism - and the only one with ablation evidence (6.14 → 2.38 novelty score) that its anti-hallucination gate detects non-novelty.
• Structural weakness / trade-off: Everything above the wet lab rests on LLM-relative evaluation (Elo, auto-scores), and the system is deliberately not autonomous - a human configures goals, injects feedback, and selects final candidates - so it complements rather than replaces the full-autonomy pipeline of the AI Scientist lineage; closed code blocks independent replication.
• Connections: Its Elo tournament and evolution-with-combination are a natural-language analog of the program-database selection in FunSearch (Nature, 2024) and AlphaEvolve (arXiv:2506.13131) - tournament selection over hypotheses instead of execution-scored programs, trading verifiability for breadth.²³ AgentRxiv (arXiv:2503.18102) cites Co-Scientist as evidence that LLM-generated hypotheses can be validated in real biomedical settings, while The AI Scientist-v2 (arXiv:2504.08066) represents the opposite grounding endpoint: automated execution and peer review rather than physical experiments.⁴⁵

---

6. References

• [Gottweis et al., 2025] “Accelerating scientific discovery with Co-Scientist” - https://arxiv.org/pdf/2502.18864 (abs: https://arxiv.org/abs/2502.18864); reported Nature version (2026, treat cautiously): https://doi.org/10.1038/s41586-026-10644-y
• [Romera-Paredes et al., 2024] “Mathematical discoveries from program search with large language models (FunSearch)” - https://doi.org/10.1038/s41586-023-06924-6
• [Novikov et al., 2025] “AlphaEvolve: A coding agent for scientific and algorithmic discovery” - https://arxiv.org/pdf/2506.13131
• [Schmidgall & Moor, 2025] “AgentRxiv: Towards Collaborative Autonomous Research” - https://arxiv.org/pdf/2503.18102
• [Yamada et al., 2025] “The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search” - https://arxiv.org/pdf/2504.08066

Footnotes

1. Co-Scientist v2 PDF, arXiv:2502.18864 - https://arxiv.org/pdf/2502.18864 ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹² ↩¹³ ↩¹⁴ ↩¹⁵

2. FunSearch, Nature 2024 - https://doi.org/10.1038/s41586-023-06924-6 ↩

3. AlphaEvolve, arXiv:2506.13131 - https://arxiv.org/pdf/2506.13131 ↩

4. AgentRxiv PDF - https://arxiv.org/pdf/2503.18102 ↩

5. AI Scientist-v2 PDF - https://arxiv.org/pdf/2504.08066 ↩