In December 2024, during the frenzied adoption of LLM coding assistants, we became aware that such tools tended—unsurprisingly—to produce Go code in a style similar to the mass of Go code used during training, even when there were newer, better ways to express the same idea. Less obviously, the same tools often refused to use the newer ways even when directed to do so in general terms such as “always use the latest idioms of Go 1.25.” In some cases, even when explicitly told to use a feature, the model would deny that it existed. [...] To ensure that future models are trained on the latest idioms, we need to ensure that these idioms are reflected in the training data, which is to say the global corpus of open-source Go code.
PHP went through a similar effort a while back to just clear out places like Stackoverflow of terrible out of date advice (e.g. posts advocating magic_quotes). LLMs make this a slightly different problem because, for the most part, once the bad advice is in the model it's never going away. In theory there's an easier to test surface around how good the advice it's giving is but trying to figure out how it got to that conclusion and correct it for any future models is arcane. It's unlikely that model trainers will submit their RC models to various communities to make sure it isn't lying about those specific topics so everything needs to happen in preparation of the next generation and relying on the hope that you've identified the bad source it originally trained on and that the model will actually prioritize training on that same, now corrected, source.
I definitely see that with C++ code
Not so easy to "fix", though. Or so I think. But I do hope still, as more and more "modern" C++ code gets published
They're particularly bad about concurrent go code, in my experience - it's almost always tutorial-like stuff, over-simplified and missing error and edge case handling to the point that it's downright dangerous to use... but it routinely slips past review because it seems simple and simple is correct, right? Go concurrency is so easy!
And then you point out issues in a review, so the author feeds it back into an LLM, and code that looks like it handles that case gets added... while also introducing a subtle data race and a rare deadlock.
I'd prefer we start nuking the idea of using LLMs to write code, not help it get better. Why don't you people listen to Rob Pike, this technology is not good for us. Its a stain on software and the world in general, but I get it most of ya'll yearn for slop. The masses yearn for slop.
battle of my life. several times i’ve had to update my agent instructions to prefer modern and usually better syntax to the old way of doing things. largely it’s worked well for me. i find that making the agents read release notes, and some official blog posts, helps them maintain a healthy and reasonably up-to-date instructions on writing go.
Its tooling like this that really makes golang an excellent language to work with. I had missed that rangeint addition to the language but with go fix I'll just get that improvement for free!
There have been many situations where I'd rather use another language, but Go's tooling is so good that I still end up writing it in Go. So hard to beat the build in testing, linting, and incredible compilation.
I just searched for `for` loops with `:=` within and hand-fixed them. I found a few forms of the for loops and where there was a high number, I used regexp.
I think tooling that can modify your source code to make it more modern is really cool stuff. OpenRewrite comes to mind for Java, but nothing comes to the top of my mind for other languages. And heck, I into recently learned about OpenRewrite and I've been writing Java for a long time.
Even though I don't like Go, I acknowledge that tooling like this built right into the language is a huge deal for language popularity and maturity. Other languages just aren't this opinionated about build tools, testing frameworks, etc.
I suspect that as newer languages emerge over the years, they'll take notes from Go and how well it integrates stuff like this.
biome kind of does this stuff for me in the typescript world. for example, it recommends for...of instead of forEach. paired with ultracite, it really improves my workflow because it is super easy to add to any project, just one dependency. breath of fresh air compared with the eslint ecosystem.
i now have an agents.md file that says to run biome fix after every modification. i end with much nicer code that i don't have to go and fix myself (or resave the file to get biome to run). speeds things up considerably to not have that step in my own workflow.
Not even mentioned in the article, my favorite capability is the new `//go:fix inline` directive, which can be applied to a one-line function to make go fix inline it's contents into the caller.
That ends up being a really powerful primitive for library authors to get users off of deprecated functions, as long as the old semantics are concisely expressible with the new features. It can even be used (and I'm hoping someone makes tooling to encourage this) to auto-migrate users to new semver-incompatible versions of widely used libraries by releasing a 1.x version that's implemented entirely in terms of thin wrappers around 2.x functions and go fix will automatically upgrade users when they run it.
Incidentally I saw the Wes McKinney podcast who said that Go was the perfect language now because of the fast compile-run cycle with strong types and built in multi thread safety was perfect for Coding Agents. Gave me a whole new interest. https://www.youtube.com/watch?v=1VfzDXeQRhU
Not related with go: I recently tried to learn Python beyond the classical example code from the web. After discovering that there are more or less 4 different ways to do a thing with no clear guide what best practice is. I come from C and there one is happy if there is ONE way to do a thing :). Is Go at this stage? Im Intrested in learning Go but not to a point where i need a LLM to determin if my code follows best practice.
22 comments
[ 2.8 ms ] story [ 41.5 ms ] threadIn December 2024, during the frenzied adoption of LLM coding assistants, we became aware that such tools tended—unsurprisingly—to produce Go code in a style similar to the mass of Go code used during training, even when there were newer, better ways to express the same idea. Less obviously, the same tools often refused to use the newer ways even when directed to do so in general terms such as “always use the latest idioms of Go 1.25.” In some cases, even when explicitly told to use a feature, the model would deny that it existed. [...] To ensure that future models are trained on the latest idioms, we need to ensure that these idioms are reflected in the training data, which is to say the global corpus of open-source Go code.
And then you point out issues in a review, so the author feeds it back into an LLM, and code that looks like it handles that case gets added... while also introducing a subtle data race and a rare deadlock.
Very nearly every single time. On all models.
Real kudos to the golang team.
This tool is way cooler, post-redesign.
Even though I don't like Go, I acknowledge that tooling like this built right into the language is a huge deal for language popularity and maturity. Other languages just aren't this opinionated about build tools, testing frameworks, etc.
I suspect that as newer languages emerge over the years, they'll take notes from Go and how well it integrates stuff like this.
It's used a lot to migrate old codebases. The tool is using itself to downgrade[2] it so that it can run on older PHP versions to help upgrades.
[1] https://getrector.com
[2] https://github.com/rectorphp/rector-downgrade-php
i now have an agents.md file that says to run biome fix after every modification. i end with much nicer code that i don't have to go and fix myself (or resave the file to get biome to run). speeds things up considerably to not have that step in my own workflow.
That ends up being a really powerful primitive for library authors to get users off of deprecated functions, as long as the old semantics are concisely expressible with the new features. It can even be used (and I'm hoping someone makes tooling to encourage this) to auto-migrate users to new semver-incompatible versions of widely used libraries by releasing a 1.x version that's implemented entirely in terms of thin wrappers around 2.x functions and go fix will automatically upgrade users when they run it.