Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution

2023 · Paper

---

1. Motivation

Chain-of-thought prompting showed that an LLM’s reasoning is steered by its prompt, and automatic prompt engineers (APE, OPRO) showed prompts can be optimized iteratively. But such iterative methods hit diminishing returns: a single fixed meta-prompt proposing mutations converges onto one phrasing style and stops exploring.¹ Promptbreeder’s answer is to make the search self-referential - the system “is not just improving task-prompts, but it is also improving the mutation-prompts that improve these task-prompts.”¹ The design explicitly imports quality-diversity thinking (the lineage of MAP-Elites) into LLM text space, making this paper DeepMind’s conceptual bridge between archive-based illumination and the program-evolution systems (FunSearch, AlphaEvolve) that followed.¹²

---

2. Methodology

2.1 Search Structure

Population-based evolutionary search over prompt strategies rather than solutions. The population consists of 50 “units of evolution,” each typically containing 2 task-prompts + 1 mutation-prompt plus an evolving few-shot context list.¹ Search runs typically 20–30 generations; each generation pairs all units for binary-tournament competition (following Harvey’s microbial GA framework).¹

2.2 State Representation

The genome is natural language. A unit is $\{\text{task-prompt}(s), \text{mutation-prompt}, \text{few-shot context of correct workings-out}\}$. Above the population sits a fixed meta-level hyper-mutation-prompt $H$ (“Please summarize and improve the following instruction:”) that mutates mutation-prompts: $M' = \text{LLM}(H + M)$; task-level mutation is $P' = \text{LLM}(M + P)$.¹ Initialization: the LLM continues from problem description $D$ + a randomly sampled “thinking-style” string $T$ + a randomly sampled initial mutation-prompt.¹

2.3 Generation Mechanism

Nine mutation operators in five classes; exactly one is sampled uniformly per replication event:¹

1. Direct mutation. Zero-order: fresh hint regenerated from $D$ via a “A list of 100 hints:” prompt - re-anchors search to the problem description (akin to uniform resampling). First-order: standard asexual mutation via the unit’s own mutation-prompt.
2. Estimation-of-distribution (EDA) mutation. EDA: continue a BERT-cosine-diversity-filtered list (similarity $\le 0.95$) of population prompts. EDA rank-and-index: fitness-ascending list where the LLM is deliberately “lied to” that the order is descending, exploiting recency bias to favor diversity. Lineage: chronological elite history headed “GENOTYPES FOUND IN ASCENDING ORDER OF QUALITY” - a verbal gradient from bad to good prompts, anticipating FunSearch’s best-shot prompting with sorted program versions.¹²
3. Hyper-mutation. Zero-order and first-order mutation of mutation-prompts - evolution of evolvability.
4. Lamarckian mutation. Reverse-engineer a task-prompt from a correct working-out (“I gave a friend an instruction… The instruction was:”) - a phenotype→genotype mapping related to STaR/APE.
5. Prompt crossover + context shuffling. With 10% probability replace a task-prompt with one from another unit (fitness-proportionate donor choice); few-shot contexts keep only workings-out that produced correct answers, with 10% whole-context resampling.¹

2.4 Evaluation & Grounding

Fitness = accuracy of the unit’s two-stage prompt strategy on a random batch of training data re-sampled at every evaluation - this discourages overfitting to a fixed dev set.¹ Final reporting uses held-out test sets of GSM8K, SVAMP, MultiArith, AddSub, AQuA-RAT, SingleEq (arithmetic), CSQA, StrategyQA (commonsense), instruction-induction tasks, and ETHOS hate-speech classification. Ground truth is dataset labels only; no LLM-as-judge.¹

2.5 Memory / Information Flow

The population itself is the memory. Three auxiliary channels feed information back: per-unit lineage lists (historical elites) consumed by the lineage operator; few-shot contexts accumulating correct workings-out (a Lamarckian channel from phenotype into genotype); and the diversity-filtered population listing consumed by EDA operators. A documented anti-pattern shaped the design: when given raw fitness values, the LLM “did not understand these fitness values and resorted to generating copies” - hence all fitness information reaches the LLM through indirect representations (rank order, chronological order) rather than numbers.¹

2.6 Selection & Propagation

Steady-state binary tournament: sample two units, mutate the higher-fitness one, and let the mutant overwrite the loser. Crossover donors are chosen fitness-proportionately. Diversity is maintained by BERT-embedding filtering inside EDA operators and by zero-order re-anchoring to the problem description.¹

2.7 Anti-Slop / Anti-Hallucination Mechanisms

The main controls target premature convergence, not falsehood: diversity-filtered listings, fresh random training batches per fitness evaluation, and zero-order re-grounding. There is no execution-based verification - fitness is pure text-in/text-out accuracy - so evolved prompts can encode idiosyncratic styles rather than verified reasoning; this residual slop risk is unaddressed and is exactly what FunSearch later closed by executing candidates.¹²

---

3. Results

Zero-shot test accuracies (Table 1; PaLM 2-L backbone; baselines re-run by the authors on text-davinci-003, with PS/PS+ also on PaLM 2-L): Promptbreeder (PB) beats Problem-Description+ (PS+) on 7 of 8 benchmarks - MultiArith 99.7 vs 92.5, SingleEq 96.4 vs 94.7, SVAMP 90.2 vs 86.3, StrategyQA 71.8 vs 50.1, CSQA 85.4 vs 73.3, AQuA-RAT 62.2 vs 39.4, GSM8K 83.9 vs 60.5 (and above APE 77.9 / OPRO 80.2 on GSM8K) - losing only on AddSub (87.8 vs 92.2 for text-davinci-003 PS+).¹ On ETHOS, an evolved two-prompt strategy reaches 89% vs 80% for a hand-designed prompt.¹ Ablations (Appendix L) find removing any self-referential operator harmful in nearly all settings, with the largest benefit from the initial re-description of task-prompts.¹

---

4. Limitations & Assumptions

Prompt topology is fixed: “we only adapt the prompt content not the prompting algorithm itself” - reasoning structure is not evolved.¹ The method assumes datasets with verifiable answers and a training split; fitness noise from small random batches is unquantified. Critically for the evidence base: no code was released, the paper exists as a single arXiv version, and results are not independently replicated; gains were demonstrated on PaLM 2-L and may not transfer to smaller models.¹

---

5. Critical Analysis

• Distinctive strength: the self-referential operator layer (mutation-prompts that mutate mutation-prompts) plus the EDA/lineage operators is a principled answer to prompt-search convergence, and the “lie about the ordering” trick is an honest, empirically motivated account of how LLMs consume fitness information - insight later reused in best-shot prompting.¹
• Structural weakness / trade-off: unverifiable text fitness plus no released code makes the results hard to audit; the system optimizes phrasing, not executable correctness, so nothing prevents a prompt that scores well for spurious stylistic reasons.¹
• Connections: Promptbreeder explicitly cites quality-diversity - the MAP-Elites lineage of archive-based illumination³ - as the inspiration for its diversity filtering, and AlphaEvolve inherits the “LLM as breeder over an evolving population” framing wholesale while adding execution-grounded fitness and a MAP-Elites-inspired database.²⁴ ReEvo applies the same LLM-as-variation-operator principle to heuristic code, swapping mutation-prompts for Reflexion-style comparative reflection.⁵

---

6. References

Footnotes

1. Fernando et al., Promptbreeder (PDF v1): https://arxiv.org/pdf/2309.16797.pdf ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹² ↩¹³ ↩¹⁴ ↩¹⁵ ↩¹⁶ ↩¹⁷ ↩¹⁸ ↩¹⁹ ↩²⁰ ↩²¹ ↩²²

2. Romera-Paredes et al., FunSearch, Nature 625:468–475: https://www.nature.com/articles/s41586-023-06924-6 ↩ ↩² ↩³ ↩⁴

3. Mouret & Clune, MAP-Elites: https://arxiv.org/abs/1504.04909 ↩

4. Novikov et al., AlphaEvolve: https://arxiv.org/abs/2506.13131 ↩

5. Ye et al., ReEvo, NeurIPS 2024: https://arxiv.org/abs/2402.01145 ↩