Notes — all entries

FormulaCode is a live, repository-scale benchmark that evaluates agentic performance engineering against human-expert patches on 957 real bottlenecks mined from 245,477 PRs across 70 ASV-instrumented scientific-Python repositories, using correctness rollback, statistical significance testing, and an expert-relative Advantage metric with a built-in contamination probe.

A self-evolving multi-agent framework that repairs three failure modes of MLE agents - inter-branch information isolation, memoryless search, and one-shot generation - via Progressive Monte Carlo Graph Search, Retrospective Memory, and Hierarchical Planning with Adaptive Code Generation, reaching a 65.3% medal rate on MLE-Bench at half the standard budget (12 h vs. 24 h) and the best result on 11 of 15 AlphaEvolve math tasks.

MARS treats automated AI research as cost-constrained search over a space of whole software repositories, combining budget-aware MCTS with an efficiency-shaped reward, a Design-Decompose-Implement modular construction pipeline, and a comparative reflective memory whose lessons are distilled from diffs against the previous best solution.

Three independent open-source derivatives show that AlphaEvolve-style program evolution can be made reproducible and radically more sample-efficient - GigaEvo by engineering the unspecified infrastructure, ShinkaEvolve by principled parent/LLM selection (new circle-packing SOTA in 150 samples), and ThetaEvolve by batch generation plus test-time RL (new best-known bounds from a single 8B open model).

FM Agent combines LLM-driven mutation and crossover with a multi-island evolutionary architecture, cold-start population generation, domain-specific evaluators, and a Ray-based asynchronous infrastructure, achieving state-of-the-art results across MLE-Bench, ALE-Bench, KernelBench, and classical mathematics problems.

The survey argues that "Agentic Science" - large language model (LLM) agents autonomously running the full hypothesis–experiment–analysis–refinement cycle - constitutes a distinct stage of AI for Science, and unifies the field through a three-layer framework of five foundational capabilities, four core processes, and four application domains.

The survey defines "deep research" as a paradigm beyond retrieval-augmented generation (RAG), in which agents iteratively plan, retrieve, and synthesize web-grounded analytical reports, and organizes the field by a capability-centric four-stage pipeline - planning, question developing, web exploration, report generation - rather than by whole-system enumeration.

ML-Master integrates parallelized UCT-MCTS exploration with a reasoning model fed a selectively scoped parent-and-sibling memory (1.0), and ML-Master 2.0 extends this to day-scale autonomy via Hierarchical Cognitive Caching (HCC) that differentiates experience → knowledge → wisdom across tasks.

An evolutionary coding agent that orchestrates a Gemini LLM ensemble to make grounded, evaluator-verified code changes, thereby discovering provably correct algorithms - including the first improvement in 56 years over Strassen for 4×4 complex matrix multiplication (48 scalar multiplications) - and deployed optimizations in Google's production infrastructure.

An open-source reimplementation and extension of AlphaEvolve - an asynchronous pipeline (prompt sampler → LLM ensemble → evaluator pool → program database) that evolves entire code files in any language under MAP-Elites quality-diversity plus island migration - claiming "2–3× speedups on real hardware" and "state-of-the-art circle packing" (self-reported).

R&D-Agent splits machine learning engineering between a Researcher agent that converts performance feedback into ideas and a Developer agent that converts error feedback into code, runs multiple heterogeneous exploration traces in parallel, and fuses their partial results into composite solutions; v2 generalizes this into a two-phase, six-component formalization of the whole MLE workflow.

By removing human-authored code templates and adding a four-stage experiment manager with parallelized agentic tree search and vision-language model (VLM) feedback, The AI Scientist-v2 produced the first entirely AI-generated manuscript to pass genuine blind peer review (average score 6.33/10) at an ICLR 2025 workshop.

This 13-page survey argues that agentic AI - LLM systems capable of reasoning, planning, and autonomous decision-making - is transforming every stage of scientific research, but that the field's near-term value lies in augmenting rather than replacing human researchers, with literature review empirically the weakest and most failure-prone stage of current autonomous pipelines.

A centralized preprint server that lets autonomous agent laboratories upload and retrieve each other's research papers, enabling cumulative, collaborative progress - agents with access to prior agent-generated research improve 11.4% (relative) over baseline on MATH-500 while isolated agents plateau, and three parallel laboratories reach 13.7%.

Symbolic regression becomes far more sample-efficient when an LLM proposes executable equation skeletons conditioned on a natural-language problem specification while numerical optimizers fit the placeholder parameters, with a FunSearch-style islands experience buffer maintaining diverse high-scoring hypotheses.

AIDE frames machine learning engineering (MLE) as a code optimization problem over Python scripts and operationalizes trial-and-error as a greedy best-first tree search whose every node is grounded in an executed validation metric.

A Gemini-2.0 multi-agent system that generates, debates, and evolves natural-language scientific hypotheses under an Elo-ranked tournament with literature-grounded verification, outperforming state-of-the-art reasoning models and expert "best guesses" on expert-curated goals - and, uniquely among autonomous research systems, having three of its outputs (acute myeloid leukemia drug repurposing, liver-fibrosis targets, and an antimicrobial-resistance mechanism) validated in wet-lab experiments.

Dolphin closes the research loop that The AI Scientist left open - feeding experimental outcomes back into idea generation so that idea quality improves across cycles - and, evaluated on standard benchmarks rather than toy templates, produces methods comparable to state-of-the-art on tasks such as 3D point-cloud classification.

A pipeline of role-specialized LLM agents (PhD, Postdoc, ML Engineer, Software Engineer, Professor) plus two solver tools (mle-solver, paper-solver) turns a human-provided research idea into a literature review, experiments, and a LaTeX report for as little as $2.33 per paper - an 84% cost decrease versus prior autonomous research methods - with optional human "co-pilot" checkpoints that measurably improve output quality.

The first comprehensive framework in which large language model (LLM) agents autonomously generate research ideas, implement and run experiments, and write full LaTeX papers with a simulated peer-review loop, at under $15 per paper - with an automated reviewer that approaches human-level agreement on ICLR 2022 review data.

Reflective Evolution integrates LLM-generated "verbal gradients" - short-term comparative reflections between parent heuristics and long-term distilled design knowledge - into genetic programming over open-ended heuristic code, yielding state-of-the-art combinatorial-optimization heuristics within 100 evaluated candidates.

Pairing a frozen pretrained code LLM with a deterministic evaluator in an island-based evolutionary loop ("FunSearch" = searching in function space) yields new verifiable scientific discoveries - cap sets beating 20-year-best constructions and online bin-packing heuristics beating first-fit and best-fit - while the evaluator guards against LLM confabulation.

Language agents can be improved across trials without any weight updates by converting environment feedback into natural-language self-reflections stored in a bounded episodic memory, which acts as a "semantic gradient" conditioning subsequent attempts.

An evolutionary algorithm in which an LLM mutates both a population of task-prompts and the mutation-prompts that generate those mutations - a self-referential loop that sustains diversity and escapes the diminishing returns of one-shot automatic prompt engineering.

MAP-Elites maintains a discretized archive over user-chosen feature dimensions and keeps the best-performing solution found so far in each cell, thereby illuminating the fitness potential of the whole feature space - and, by exploring more, often finding a better overall solution than optimization-focused search.