What if the key to evolving better code isn’t bigger models or elaborate heuristics, but simpler search?

Recent works like AlphaEvolve demonstrated how language-model-based search pipelines enable mathematical and algorithmic discovery. The search is done using code evolution, where an evolutionary algorithm is run using a language model as a mutation or recombination operator. Although based on simple principles, in practice code evolution pipelines can be very complicated, built on a myriad of heuristic design choices.

In this talk I will argue that most existing design choices are unnecessary, as very simple search baselines work well across a variety of domains. Using these simple baselines allows uncovering what actually matters for code evolution, revealing shortcomings in the field. I will discuss how to mitigate some of these shortcomings, how code evolution might not be as open-ended as it’s thought to be, and implications for how to build code evolution systems that matter.

Paper: https://arxiv.org/pdf/2602.16805

Blog post: https://yonatan.gideoni.com/blog/what_matters_for_code_evo/

About the Speaker: Yonatan is a DPhil student at Oxford developing fundamental methods in machine learning. His research investigates the limits of existing learning paradigms, aiming to understand where they break down and how to design methods that go beyond them. His recent work explored these questions in multimodality and code generation. Previously, Yonatan received his master’s in computer science from the University of Cambridge and worked on maps for autonomous vehicles at Mobileye. His PhD is funded by the AIMS CDT and a Rhodes Scholarship.