ML-Master / ML-Master 2.0: Integrating Exploration and Reasoning, then Cognitive Accumulation

19 Jun 2025 (1.0), 25 Feb 2026 (2.0) · Paper 1.0 · Paper 2.0

---

1. Motivation

LLM agents for machine learning engineering (MLE) typically fail to reuse exploration experience during reasoning: greedy tree agents such as AIDE select nodes by a hard-coded rule with no exploration/exploitation balance, and ReAct-style agents drown in ever-growing context.¹² ML-Master 1.0 couples balanced multi-trajectory exploration (MCTS) with steerable reasoning - DeepSeek-R1 whose “think” channel is fed a curated memory.¹ ML-Master 2.0 identifies a different bottleneck: on day-scale horizons the context saturates. It reframes context management as cognitive accumulation - raw experience structurally differentiated over time into stable knowledge and cross-task wisdom rather than merely retained.³

---

2. Methodology

2.1 Search Structure

1.0: a tree searched with the four canonical MCTS phases - selection, expansion, verification, backpropagation - executed iteratively and in parallel (2.6).¹ 2.0: departs from rule-based MCTS toward plan-driven phased exploration: at each phase boundary the agent proposes a hierarchical research plan ($m$ exploration directions × $q$ concrete implementation suggestions), executes suggestions in parallel, consolidates the phase, and repeats until budget exhaustion.³

2.2 State Representation

1.0: a node is a solution tuple (code, evaluation metric, defect flag) with per-node visit count $N_v$ and accumulated reward $Q_v$.¹ 2.0: a formal event-sequence model: history $E_t = \{e_0..e_t\}$ is partitioned into environment events $U$ and agent events $A$; context $C_{t-1} = g(E_{t-1})$; actions are sampled $e_t \sim \pi(\cdot \mid C_{t-1})$; the final solution is extracted from the terminal history.³

2.3 Generation Mechanism

1.0: expansion reuses AIDE’s action triplet - Draft (new runnable solution), Debug (fix errors), Improve (preprocessing/architecture/optimization refinements) - chosen by a deterministic rule (no solution → Draft; buggy → Debug; valid → Improve). DeepSeek-R1-0120 generates plans and code.¹ 2.0: the LLM emits code patches, shell commands, and plans; the first phase targets a “buggy-free initial code” before planning.³

2.4 Evaluation & Grounding

1.0: verification reward

$$R(v) = \begin{cases} -1 & \text{if } v \text{ has defects} \\ r_q + r_d + r_s & \text{otherwise} \end{cases}$$

where $r_q = \mathbb{1}[M(v) > M^*]$ (beats best-so-far metric), $r_d = \mathbb{1}[\text{parent's fault fixed}]$, $r_s = \mathbb{1}[\text{improvement stopping criterion met}]$. Rewards backpropagate to the root, updating $N$ and $Q$.¹ 2.0: the environment executes actions and returns terminal output, error messages, and metric logs (kept raw in L1); a task metric $F(\cdot)$ scores the extracted final solution.³

2.5 Memory / Information Flow

1.0 - adaptive memory. The memory injected into the reasoning model is

$$M_t = \{(r_{t-1}, f_{t-1})\} \cup \{(r_t^{(s)}, f_t^{(s)}) \mid s \in S_t\},$$

i.e., distilled reasoning insights $r = \varepsilon(c)$ (extracted from the full R1 reasoning trace) plus execution feedback $f$ (validation metrics, logs, error diagnostics) drawn from the immediate parent (continuity) and parallel siblings at the same depth (contrast). Crucially, this memory is embedded in the reasoning model’s “think” component, not the instruction prompt.¹

2.0 - Hierarchical Cognitive Caching (HCC), the core contribution. Three tiers:³

• L1 Evolving Experience (working memory): raw traces for the active phase plus all plan-boundary events plus a bootstrap segment: $L_1(t) = E_{t_0-1} \cup P_{p-1} \cup E_{t_{p-1}+1:t}$.
• L2 Refined Knowledge (mid-term): one compact summary $\kappa$ per completed phase - key judgments (“feature X is harmful”), insights (“CV leakage under split Y”), decision rationale.
• L3 Prior Wisdom (long-term, cross-task): an embedding-keyed store $\{(h_n, w_n)\}$ where $h_n = E(d_n)$ embeds an LLM-written task descriptor and $w_n$ is distilled transferable wisdom (model templates, preprocessing pipelines, hyperparameter priors); persistent across tasks.

2.6 Selection & Propagation

1.0 - UCT + pruning + branch parallelism. Child selection by

$$\mathrm{UCT}(v) = \frac{Q_v}{N_v} + C\sqrt{\frac{\ln N_{\text{parent}}}{N_v}}.$$

Two pruning rules: improvement-based termination (node terminal if $\sum \mathbb{1}[\Delta_i < t] > \tau_{\text{improve}}$ over $K$ improve attempts) and a debug-depth constraint ($> \tau_{\text{debug}}$ consecutive debugs → terminal). Asynchronous branch-parallel MCTS: workers jointly expand the root; the top-$k$ UCT children become independent search threads; a finished thread returns to the root and picks the best UCT child not occupied by another thread. Reported hyperparameters: $t = 0.001$, $\tau_{\text{improve}} = 3$, $\tau_{\text{debug}} = 20$, parallelism degree 3, 12 h/task, 1×A100 per agent.¹

2.0 - context migration protocol. (a) Prefetching: retrieve L3 wisdom by cosine similarity above threshold $\delta$ to build the initial context $e_0$; (b) context hit: raw events fetched from L1 when present, otherwise fall back to L2 summaries (completed phases collapse to one refined unit); (c) promotion $P = (P_1, P_2)$: $P_1$ is a phase-level LLM retrospective abstraction into L2; $P_2$ distills task-level wisdom into L3 on completion.³

2.7 Anti-Slop / Anti-Hallucination Mechanisms

1.0: sibling-contrastive memory “introduces contrastive signals, allowing the model to recognize and avoid producing reasoning paths that mirror those already explored”; grounding the think-channel in executed metrics rather than free text; stagnation pruning halts unproductive refinement; the $-1$ defect reward suppresses broken-code lineages.¹ 2.0: structural separation of transient detail from validated knowledge - only repeatedly validated experience is promoted; retrieval-gated wisdom blocks irrelevant context injection; the design targets context saturation as a long-horizon error source.³

---

3. Results

ML-Master 1.0 (MLE-Bench, 75 tasks, mean ± SEM over 3 seeds): valid submission 93.3% ± 1.3, above median 44.9% ± 1.2, any medal 29.3% ± 0.8, gold 17.3% ± 0.8 with DeepSeek-R1 - versus AIDE (o1-preview) 16.9%, AIDE (DeepSeek-R1, single run) 14.7% same-model, and R&D-Agent (o1-preview) 22.4% any-medal.¹²⁴ Medium-complexity tasks: 20.2% ± 2.3 any-medal, reported as more than 2× prior SOTA; achieved in 12 h, half the 24-h budget of baselines.¹ An independent re-evaluation (ArchPilot) measured lower figures (gold+ 0.107, normalized rank 0.6535); treat the self-reported numbers with that caveat.⁵

ML-Master 2.0 (MLE-Bench, 24-h budgets, DeepSeek-V3.2-Speciale): average medal rate 56.44% (Low 75.8, Medium 50.9, High 42.2); valid submission 95.6% ± 1.2; above median 63.1% ± 1.2. Same-table baselines: FM Agent (Gemini-2.5-Pro) 43.6, R&D-Agent (GPT-5) 35.1, ML-Master 1.0 29.3.³ HCC ablation (MLE-Bench-Lite, single run): full HCC 72.7% any-medal; without L1 22.7%; without L2 59.1%; without L3 54.5%.³ Comparability caveat: 2.0’s L3 was warmed with 407 Kaggle competitions as prior-wisdom training tasks, unlike AIDE, AIRA, or MARS, which use no warm-up.⁶

---

4. Limitations & Assumptions

• 1.0 defers ablations; the AIDE-R1 baseline was run once; hardware was “slightly inferior” to the MLE-Bench reference.¹
• 2.0’s ablations are single-run on MLE-Bench-Lite only; heavy dependence on the 407-competition warm-up corpus; the v4 text twice calls the system “ML-ACE” (a naming artifact); no public 2.0 code as of July 2026.⁷³
• Both versions are DeepSeek-model-centric; cross-model generality is not established (contrast MARS’s Claude 4.6 Sonnet check).⁶

---

5. Critical Analysis

• Distinctive strength: injecting contrastive parent+sibling memory into the reasoning model’s think channel (1.0), and the explicit experience→knowledge→wisdom promotion protocol (2.0) - memory as a typed, migratable structure rather than a transcript.
• Structural weakness / trade-off: 2.0’s strongest results lean on a large external warm-up corpus and one model family, making its gains partly a data-scale effect; 1.0’s deterministic action rule and un-ablated design leave the source of its improvement under-attributed.
• Connections: inherits and UCT-ifies AIDE’s Draft/Debug/Improve triplet, beating AIDE 29.3% vs. 14.7% any-medal under the same DeepSeek-R1 backend.¹² Uses R&D-Agent (22.4%) as its strongest 1.0 baseline; 2.0 compares against R&D-Agent (GPT-5, 35.1%).⁴ MARS adopts the same MCTS skeleton but adds cost-shaped rewards and repository-level states, and flags 2.0’s warm-up as a setup disparity when comparing MARS+ (62.7%) to 2.0 (56.4%).⁶ 2.0 groups FM-Agent with island-based evolution approaches that treat knowledge as homogeneous - exactly what HCC’s tiering rejects.³⁸

---

6. References

Footnotes

1. Liu et al., 2025, “ML-Master: Towards AI-for-AI via Integration of Exploration and Reasoning” - https://arxiv.org/abs/2506.16499 (full text: https://arxiv.org/html/2506.16499v1) ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹² ↩¹³

2. Jiang et al., 2025, “AIDE: AI-Driven Exploration in the Space of Code” - https://arxiv.org/abs/2502.13138 ↩ ↩² ↩³

3. Zhu et al., 2026, “Toward Ultra-Long-Horizon Agentic Science: Cognitive Accumulation for Machine Learning Engineering” - https://arxiv.org/abs/2601.10402 (full text: https://arxiv.org/html/2601.10402v4) ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹²

4. Yang et al., 2025, “R&D-Agent: Automating Data-Driven AI Solution Building Through LLM-Powered Automated Research, Development, and Evolution” - https://arxiv.org/abs/2505.14738 ↩ ↩²

5. ArchPilot (independent ML-Master re-evaluation on MLE-Bench), 2025 - https://arxiv.org/abs/2511.03985 ↩

6. Chen et al., 2026, “MARS: Modular Agent with Reflective Search for Automated AI Research” - https://arxiv.org/abs/2602.02660 ↩ ↩² ↩³

7. ML-Master code and 2.0 release status - https://github.com/sjtu-sai-agents/ML-Master ↩

8. FM Agent Team (Baidu AI Cloud), 2025, “The FM Agent” - https://arxiv.org/abs/2510.26144 ↩