DecAEvolve: Decompose, Adapt, Evolve
Three Pillars of LLM-based Equation Discovery

Motivation

Where LLM-based equation discovery breaks.

Existing LLM-SR systems treat the LLM as a fixed hypothesis generator and a scalar MSE as the only feedback. Two of the strongest signals available to the search are simply discarded.

Method

Three coupled mechanisms.

Stage 1 (adaptation): for N iterations, generate, score, decompose, and update the LLM with GRPO. Stage 2 (search): freeze the adapted policy and run T more iterations of decomposition-guided evolutionary search.

1 — Decompose: programs → structured feedback

Each candidate function body is parsed with Python's ast module, intermediate assignments are inlined into the return expression, and the AST is traversed under three rules:

For each atom u_m, DecAEvolve constructs an ablated program f_\um, re-optimizes the remaining BFGS parameters on the data, and computes a marginal contribution:

The same construction over pairs (u_m, u_n) yields interaction contributions Δ_um,un, exposing redundancy versus synergy. Re-optimization is essential — with frozen parameters, removing a term can look catastrophic only because the rest of the equation was never allowed to compensate. Contributions are serialized as inline Python comments above the return statement, so executable semantics are unchanged but the next prompt now reads structural evidence.

2 — Adapt: GRPO at test time, with LoRA

Adaptation is reinforcement learning over a deterministic MDP whose state is a (prompt, prefix) pair, action is the next token, and reward is the bounded validation score r(p, h) = exp(−MSE(h, 𝒟)) (floored at 0.01 for invalid completions). Each prompt samples G=64 completions, GRPO computes a per-prompt baseline b(p) and group-relative advantages A_i = r_i − b(p), and optimizes:

LoRA adapters (rank 16, α=16, dropout 0.05) carry all trainable parameters; the base model stays as the frozen reference π_ref. β=0.05 regularizes drift; together with the per-prompt baseline this produces low-variance, single-step updates that are safe to take during search itself.

3 — Evolve: annotated populations, multi-island buffers

The buffer 𝒫 = ⋃_i 𝒫⁽ⁱ⁾ is sharded into independent islands that diverge over time and prevent premature convergence. A new program is admitted to its source island only if it strictly beats that island's best score. Every 4 hours the worst-performing half of islands is overwritten with copies from a surviving island. Within an island, programs are clustered by score signature, sampled by Boltzmann weights with an annealing temperature, and ranked within a cluster by length-and-score. Each prompt is a hierarchical sample — island → cluster → program — concatenated into a structured few-shot context. The few-shot examples carry their decomposition annotations, so the model conditions on component-level success.

Results

Up to two orders of magnitude lower OOD error.

DecAEvolve consistently wins on the OOD splits — the regime where static search overfits or stalls — and the gains transfer across all six backbones from 1.5B to 8B parameters.

Normalized MSE (lower is better), averaged over five runs; column-best in accent. SINDy wins Oscillator 2 because its assumed sparse library happens to match the system — and pays for that prior with a 3.5×10¹⁵ blow-up on real materials data, illustrating why baked-in libraries are not free.

Cite

BibTeX

@inproceedings{behzadifar2026decaevolve,
  title     = {DecAEvolve: Decompose, Adapt, and Evolve, or, Three Pillars
               of Effective LLM-based Scientific Equation Discovery},
  author    = {Behzadifar, Pouya and Shojaee, Parshin and Kabra, Sanchit
               and Meidani, Kazem and Reddy, Chandan K},
  booktitle = {Proceedings of the 43rd International Conference on Machine Learning},
  address   = {Seoul, South Korea},
  publisher = {PMLR},
  volume    = {306},
  year      = {2026}
}