What I'm most curious about is how this translates to messy, real-world codebases without well-defined metrics. Most production software isn't chip design or kernel optimization - it's business logic with unclear success criteria. The infrastructure story is impressive, but I'd love to see how they handle domains where the evaluation function itself is ambiguous.
Most people that I have spoken to that are at Google have complained about the internal Gemini agent and seem to believe it's gotten significantly worse lately. Things like it completely forgetting how to call the tools and eventually giving up having wasted a bunch of time, or the agent completely ignoring code style guidance in an AGENTS.md-esque file.
My experience running Gemma 4 locally has been similar: after maybe one or two tool calls it starts making tool calls however it feels like. Just yesterday, I watched it redefine a tool like read_file(start, end) into read_file(start, number_of_bytes) and it refused to even consider that it was wrong.
I would be interested to see how exactly the agent helped. How was it used, where did it lead to the given improvement and in how far would it have taken a human to come to the same solution.
This reminds me of Antirez's "Don't fall into the anti-AI hype" [0]
In a sentence: These foundation models are really good at optimizing these extremely high level, extremely well defined problem spaces (ie multiply matrices faster). In Antirez's case, it's "make Redis faster".
There have been two reactions: "Oh it would never work for me" and "I have seen months of my life accomplished in an hour", and I think they're both right. I think we should be excited for Antirez, (who has since been popping off [1]), and I think the rest of us should rest easy knowing that LLM's can't (and maybe were never meant to) tackle the tacit-knowledge-filled, human-system-centric, ambiguously-defined-problem-space jobs most mortals work.
"... tackle the tacit-knowledge-filled, human-system-centric, ambiguously-defined-problem-space jobs most mortals work."
sounds like jobs involving legalese, politics, corruption and more generally involving pretending you don't understand something for which your income depends on not openly understanding something...
How many times we have to hear again about Erdös problems? :) It sounds like a great achievement for humanity at first, but after a while they keep coming back!
AlphaEvolve couples map-elites with LLMs. It's an key step in machine learning, in the vein of DQN for reinforcement learning.
AE brings diversity from the genetic algorithms community to large scale optmized deep learning and RL models.
It is a mandatory step for moving forward. The approach is clean and simple, while generic.
The only caveats is the per optimization problem definition of the map élites dimensions. But surely, this will get tackled somehow over the next few years.
If you don't know about map-elites, go look up Jean-Baptiste Mouret' s work and talks, it's both very interesting and universal.
I wish that Google would focus on bringing their Gemini 3.x models to GA, and provide enough capacity such that one not constantly has to fight with 429 errors.
It often feels like they do not want me to develop applications for corporate clients using their Vertex API. It is just such a shame, given that their models were so great for document analysis etc.
From the comments it seems that this community (mostly career software people) is starting to move into a new phase of grief about the median software engineer losing their hoped for permanent place in society.
-2021-2024 was Denial
-2024-2025 was Anger and Bargaining
-2026 seems to be some combo of anger, bargaining and acceptance depending mostly on your class/age
We went from 'AI will replace programmers' to 'AI will help programmers' to 'AI writes code while other AI reviews it' in about 18 months. At this rate the humans are just providing the electricity.
> In advertising and marketing, WPP used AlphaEvolve to refine AI model components, navigating complex, high-dimensional campaign data and achieving 10% accuracy gains over their competitive manual model optimizations.
Ah good, we're getting closer and closer to Venus, Inc. every day. /s
The AI CEOs love to pontificate about AI curing cancer, but it seems like DeepMind is the only one actively working on these research problems, while OpenAI/Anthropic largely chase enterprise/coding revenue.
All the *Evolve publications have very impressive results but from the time I’ve spent on the information published I feel that the attention goes to the LLMs and the AI side of things, although the outcomes reported are in almost all cases the result of very well designed environments for both the LLM and the evolutionary algorithm to work well.
This paper here is a great example of that and it’s worth a reading.
29 comments
[ 1.7 ms ] story [ 43.5 ms ] threadDo we have other examples of AI being used to improve the LLMs, apart for the creation of synthetic data and the testing of the models?
My experience running Gemma 4 locally has been similar: after maybe one or two tool calls it starts making tool calls however it feels like. Just yesterday, I watched it redefine a tool like read_file(start, end) into read_file(start, number_of_bytes) and it refused to even consider that it was wrong.
In a sentence: These foundation models are really good at optimizing these extremely high level, extremely well defined problem spaces (ie multiply matrices faster). In Antirez's case, it's "make Redis faster".
There have been two reactions: "Oh it would never work for me" and "I have seen months of my life accomplished in an hour", and I think they're both right. I think we should be excited for Antirez, (who has since been popping off [1]), and I think the rest of us should rest easy knowing that LLM's can't (and maybe were never meant to) tackle the tacit-knowledge-filled, human-system-centric, ambiguously-defined-problem-space jobs most mortals work.
[0] https://antirez.com/news/158 [1] https://antirez.com/news/164
sounds like jobs involving legalese, politics, corruption and more generally involving pretending you don't understand something for which your income depends on not openly understanding something...
AE brings diversity from the genetic algorithms community to large scale optmized deep learning and RL models.
It is a mandatory step for moving forward. The approach is clean and simple, while generic.
The only caveats is the per optimization problem definition of the map élites dimensions. But surely, this will get tackled somehow over the next few years.
If you don't know about map-elites, go look up Jean-Baptiste Mouret' s work and talks, it's both very interesting and universal.
It often feels like they do not want me to develop applications for corporate clients using their Vertex API. It is just such a shame, given that their models were so great for document analysis etc.
-2021-2024 was Denial
-2024-2025 was Anger and Bargaining
-2026 seems to be some combo of anger, bargaining and acceptance depending mostly on your class/age
How do I access AlphaEvolve?
Ah good, we're getting closer and closer to Venus, Inc. every day. /s
https://github.com/google-gemini/gemini-cli/issues/22141
[1]: https://news.ycombinator.com/noobstories
There are only 3 companies doing this to date: Google, Sakana AI and Autohand AI.
This paper here is a great example of that and it’s worth a reading.
Magellan: Autonomous Discovery of Novel Compiler Optimization Heuristics with AlphaEvolve https://arxiv.org/abs/2601.21096