> When I upgraded to Sonnet 4.5, it became less often that it gave phantom answers. But it still sometimes wasn’t able to handle some complex problems. We’d go back and forth and I’d try to give it hints. But with Opus 4.5, that happens much less often now.
The real annoying thing about Opus 4.5 is that it's impossible to tell most people "Opus 4.5 is an order of magnitude better than coding LLMs released just months before it" without sounding like a AI hype booster clickbaiting, but it's the counterintuitive truth. To my continual personal frustration.
> One thing I used to hope for is better interop between C++ and Rust … But after closely following discussions in the Rust committee, I do not think this is likely to happen soon.
Interesting. I thought C++ interop was one of the top priorities right now.
It’s one of the top items mentioned in recent language progress reports, the Rust foundation received a million dollar grant to work on it, and there was a talk at the most recent RustConf about how Google is doing Rust/C++ interop.
Curious to know what discussions led to that conclusion.
> Then I thought: how hard is it to write this C++ static analyzer? Conceptually, I think it’s not hard. It requires going through the AST. And since the static analysis is mostly statically scoped, it doesn’t require heavy cross-file analysis.
How do you handle function lifetimes then? Those are generally non-local to infer, and Rust requires annotating functions with informations for that. I tried taking a look at the mako db's refactor but I didn't see any lifetime annotation being added there.
Anytime that a coder modifies a function, the safe/unsafe-ness of the function will have to be audited.
People complain about comments getting out of sync with the code - seems like the same thing will occur with safe/unsafe comments attached to functions unless the developers are diligent enough to verify nothing has changed on any PR.
My other quibble from the article concerns:
...It requires going through the AST. And since the static analysis is mostly statically scoped, it doesn’t require heavy cross-file analysis. It can be single-file based, very limited scope.
The large C++ codebases I've seen have not been diligent wrt object ownership. Objects may get accessed in different files - not saying this is correct, just that it happens. Objects can be accessed/modified by other non-owning objects in inconsistent ways which lead to inconsistent behaviour especially when errors occur.
The most impressive C++ static analyzer I've seen is Intrinsa's PREfix product, bought by Microsoft back in the 1990s. They parsed the C++ code using a purchased C++ frontend parser (can't recall the company name, but there are only a handful of companies that sell this stuff) and stored the data references in a database. Then they'd do dataflow analysis of the codebase looking for bugs.
They came out with PREfast which does simpler realtime static analysis (more like lint really) and VC contains a version of this. I think the MS DDK also includes a static code analyzer based on this.
But considering the effort put into guiding the AI versus rolling your own code in your spare time and having to reload the context for your static analyzer while dumping out work-related information, we're taking baby steps into a new age/paradigm for software development.
Just think if this article had been posted five or ten years ago. The technology isn't perfect and it has a long ways to go. Let's hope we don't go down too many wrong paths.
Wrapping every pointer in a smart pointer is very bad style; that suggests you simply have the bad level of abstraction in your code.
That problem seems even more prevalent in Rust, where I see Arc used everywhere, presumably as a cop-out not to have to figure out how to satisfy the borrow checker in smarter ways.
1. There are tons of static analyzers for C++. Does none of them support type system augmentation?
2. Is the proposed solution really more practicable than "typescript for c++" [0]?
Honestly, I found this piece depressing. Life is too short and precious to waste on crappy software.
So often the question ai related pieces ask is "can ai do X?" when by far the more important question is "should ai do X?" As written, the piece reads as though the author has learned helplessness around c++ and their answer is to adopt a technology that leaves them even more helpless, which they indeed lament. I'd challenge the author to actually reflect on why the are so attached to this legacy software and why they cannot abandon it if it is causing this level of angst.
OP here: Ah, thanks for all these comments! Didn't expect so much passion.
Many comments are right this is prototype and there isn't any code guarantee! It is purely test case driven.
But this prototype sort of proves that to have Rust-equivalent memory safety, you don't really need to completely ditch C++, and all those "rewrite in Rust" clones of C++ repos. The time I spent on this project is very limited. I did maybe half of the dev on my phone through Happy. If Microsoft or Google who has lots of C++ code is willing to put some serious resources on this idea, I am sure they can have something a lot more solid. And they don't have to give up C++ (they shouldn't, it is very unclever engineering wise).
To me personally this prototype is a "usable" alternative to Circle C++. It saved me a lot of hard debugging time.
19 comments
[ 3.1 ms ] story [ 45.2 ms ] threadSimple and elegant solution.
The real annoying thing about Opus 4.5 is that it's impossible to tell most people "Opus 4.5 is an order of magnitude better than coding LLMs released just months before it" without sounding like a AI hype booster clickbaiting, but it's the counterintuitive truth. To my continual personal frustration.
Interesting. I thought C++ interop was one of the top priorities right now.
It’s one of the top items mentioned in recent language progress reports, the Rust foundation received a million dollar grant to work on it, and there was a talk at the most recent RustConf about how Google is doing Rust/C++ interop.
Curious to know what discussions led to that conclusion.
I don't know if they later changed their minds. From the meetings notes it seemed they didn't want implement a C++ frontend in rustc.
How do you handle function lifetimes then? Those are generally non-local to infer, and Rust requires annotating functions with informations for that. I tried taking a look at the mako db's refactor but I didn't see any lifetime annotation being added there.
Anytime that a coder modifies a function, the safe/unsafe-ness of the function will have to be audited.
People complain about comments getting out of sync with the code - seems like the same thing will occur with safe/unsafe comments attached to functions unless the developers are diligent enough to verify nothing has changed on any PR.
My other quibble from the article concerns:
The large C++ codebases I've seen have not been diligent wrt object ownership. Objects may get accessed in different files - not saying this is correct, just that it happens. Objects can be accessed/modified by other non-owning objects in inconsistent ways which lead to inconsistent behaviour especially when errors occur.The most impressive C++ static analyzer I've seen is Intrinsa's PREfix product, bought by Microsoft back in the 1990s. They parsed the C++ code using a purchased C++ frontend parser (can't recall the company name, but there are only a handful of companies that sell this stuff) and stored the data references in a database. Then they'd do dataflow analysis of the codebase looking for bugs.
They came out with PREfast which does simpler realtime static analysis (more like lint really) and VC contains a version of this. I think the MS DDK also includes a static code analyzer based on this.
But considering the effort put into guiding the AI versus rolling your own code in your spare time and having to reload the context for your static analyzer while dumping out work-related information, we're taking baby steps into a new age/paradigm for software development.
Just think if this article had been posted five or ten years ago. The technology isn't perfect and it has a long ways to go. Let's hope we don't go down too many wrong paths.
However, looking at the recent commits it doesn't quite look like the most solid foundation: https://github.com/shuaimu/rusty-cpp/commit/480491121ef9efec...
… which then 30 minutes later is being removed again because it turns out to be completely dead code: https://github.com/shuaimu/rusty-cpp/commit/84aae5eff72bb450...There's also quite a lot of dead code. All of these warnings are around unused variable, functions, structs, fields:
That problem seems even more prevalent in Rust, where I see Arc used everywhere, presumably as a cop-out not to have to figure out how to satisfy the borrow checker in smarter ways.
Rust has inherited mutability, while I believe const in C++ is shallow. I don't think it's a perfect match.
[0] https://github.com/hsutter/cppfront
So often the question ai related pieces ask is "can ai do X?" when by far the more important question is "should ai do X?" As written, the piece reads as though the author has learned helplessness around c++ and their answer is to adopt a technology that leaves them even more helpless, which they indeed lament. I'd challenge the author to actually reflect on why the are so attached to this legacy software and why they cannot abandon it if it is causing this level of angst.
Many comments are right this is prototype and there isn't any code guarantee! It is purely test case driven.
But this prototype sort of proves that to have Rust-equivalent memory safety, you don't really need to completely ditch C++, and all those "rewrite in Rust" clones of C++ repos. The time I spent on this project is very limited. I did maybe half of the dev on my phone through Happy. If Microsoft or Google who has lots of C++ code is willing to put some serious resources on this idea, I am sure they can have something a lot more solid. And they don't have to give up C++ (they shouldn't, it is very unclever engineering wise).
To me personally this prototype is a "usable" alternative to Circle C++. It saved me a lot of hard debugging time.