As an author of the blog, I'll note that this was one of the easiest applications of ADRS. Bowen, who was leading this effort, got things running within a day or two and the initial runs were with free Google credits! It was exciting to see how quickly these kinds of frameworks could be applied to real-world engineering and algorithmic challenges.
Nice result, but the snake pattern is pretty obvious and intuitive even for a human who just glances over the problem. It kinda breaks if there is huge variance (if the top load expert is orders of magnitude higher than #2 it probably should just get its own GPU), but I'm not familiar enough with MoE to know if that's a realistic possibility.
Really cool to see the AI-discovered algorithm is not just a theoretical result but is actually in a PR for vLLM.
My question is about the code itself. Was the Python/PyTorch generated by OpenEvolve directly usable, or did it require significant human cleanup to make it readable, maintainable, and conform to the project's coding standards? I'm curious about how close we are to AI generating production-ready, human-editable code for complex algorithms.
this feels less like Copilot and more like AlphaGo for systems programming. it's not just finding patterns in existing code, but discovering novel and more efficient strategies in a given problem space. Very cool.
The final code might be fast, but is it understandable? The evolution process shows it tried a bunch of things that didn't work. The final result is a heuristic that won out based on a specific simulator and fitness function.
i'm skeptical this generalizes beyond problems that can be expressed as "rearrange tensors faster". it feels like a solution that only works for a very narrow and convenient class of problems.
We've found that these frameworks do well for systems performance problems and expect that the range of problems for which they apply will increase as they models and frameworks improve. See our paper (https://arxiv.org/pdf/2510.06189) for more discussion about this
So, if I got this right, this is just about re-implementing an existing load balancing algorithm faster...? If so, this is really dumb. As you guys checked out, yes most load balancing algorithms are slow/dumb:
>First, we evaluate DeepSeek's open-source EPLB implementation. This employs a greedy bin-packing strategy: experts are sorted by load in descending order, and each is placed onto the least-loaded GPU that has capacity (Figure 3a, Example 1). While simple, the solution is slow because it written in Python and uses a for-loop to performs linear search for finding the best-fit GPU choice.
This is because when considering a load balancing algorithm, unless the work being done (in this case by the GPU) lasts only a few ms, the load balancing algorithm being fast will never be the bottleneck. The post does not mention whether this is the case at all.
Also, I don't want to sound rude, but if all they managed to get is a 5x increase over a simple python algorithm, I don't think this is impressive at all...? Any rewrite of the 'dumb' algorithm in a language with more memory control and cache continuity should result in much better results.
The idea that AI can discover anything is ridiculous. It can propose algorithms like it creates any piece of text, but only the human researcher is capable of analyzing the algorithm, proving that it works, understand what it is doing, i.e., pretty much everything that we call a new "discovery". I would have zero confidence in an algorithm "discovered" by an AI in isolation.
This is quite cool, but I must note that the 5x reported in the headline is the _runtime_ of the load balancing algorithm itself, not the load factor or throughput of the system or what have you.
> On average, it takes about 540 ms to re-balance the experts and achieves a load balance factor of 0.66 (calculated as the ratio of average to maximum tokens generated per GPU).
> ...
> We also consider a non-public reference implementation from a frontier lab that we have access to. This implementation avoids explicit iteration and reduces the rebalancing algorithm runtime to 19.6 ms while achieving the same balance factor as the open-source algorithm.
> ...
> The resulting algorithm matches the load balance factor of the other baselines while reducing runtime to just 3.7 ms, yielding a 5.0x speedup over the internal reference implementation.
That's a good point! The load balancing of the original algorithm was already quite good so our goal was to try to get something that could achieve similar results but could run faster since runtime was also a concern.
Relatedly, I've been working on discovering search ranking algorithms. Starting with a primitive, what can an agent do to generate search ranking + query understanding code that best optimizes a ground truth:
Neat, thanks for the link! Seems like the agent feedback loop made a big difference for your use case. Would be interesting to see how much of a difference more advanced methods (like the genetic algs you mentioned) would make
27 comments
[ 3.1 ms ] story [ 53.3 ms ] threadhttps://arxiv.org/pdf/2402.02447
>First, we evaluate DeepSeek's open-source EPLB implementation. This employs a greedy bin-packing strategy: experts are sorted by load in descending order, and each is placed onto the least-loaded GPU that has capacity (Figure 3a, Example 1). While simple, the solution is slow because it written in Python and uses a for-loop to performs linear search for finding the best-fit GPU choice.
This is because when considering a load balancing algorithm, unless the work being done (in this case by the GPU) lasts only a few ms, the load balancing algorithm being fast will never be the bottleneck. The post does not mention whether this is the case at all.
Also, I don't want to sound rude, but if all they managed to get is a 5x increase over a simple python algorithm, I don't think this is impressive at all...? Any rewrite of the 'dumb' algorithm in a language with more memory control and cache continuity should result in much better results.
> On average, it takes about 540 ms to re-balance the experts and achieves a load balance factor of 0.66 (calculated as the ratio of average to maximum tokens generated per GPU).
> ...
> We also consider a non-public reference implementation from a frontier lab that we have access to. This implementation avoids explicit iteration and reduces the rebalancing algorithm runtime to 19.6 ms while achieving the same balance factor as the open-source algorithm.
> ...
> The resulting algorithm matches the load balance factor of the other baselines while reducing runtime to just 3.7 ms, yielding a 5.0x speedup over the internal reference implementation.
[0]: https://blog.toolkami.com/alphaevolve-toolkami-style/
https://softwaredoug.com/blog/2025/10/19/agentic-code-genera...