In the end the overall design looks like a semi-reified asynchronous dataflow programming API based on coroutines.
I think that it would be useful to consider the existing (very very very large) body of research on dataflow PL and APIs; e.g. check this paper from 1982 which talks about dataflow, coroutines, for handling file operations:
My main concern with this design, even if the code looks quite neat, is that the actual dataflow graph is not reified, but exists "invisibly" as part of the program source through the way co_await calls are sequenced.
This means that it is not possible to do as an user, a static analysis and scheduling of the tasks : it relies on the compiler to perform this.
But, this problem is in my experience quite often very use-case-specific and ridden with informations compilers do not possess. Not having the way tasks are interdependent reified in an actual user-manipulable "graph" object means that users won't have the possibility, to, say, perform a topological sort or flow propagation on their async graph in order to extract information on it that could be necessary to perform a domain-specific optimization of said graph (for instance if we know that a set of tasks are dependent on each other in ways that aren't visible on the source code, but are known to the programmers, it would make sense to schedule them on a queue in their own thread).
Hum, in principle you can do arbitrary compile time transformations to the datagraph before submitting it via metaprogramming.
If you really want you could even convert the graph (fully or partially) to a runtime representation.
The core of the proposals are not about specific classes or functions other than some default algorithm implementations.
Instead they are mostly about defining concepts and protocols so that independent async domains can coexist and iteroperate in the same application.
This is similar to the STL where the most important part are the various iterator concepts and the algorithm interfaces, not the actual algo implementations and the containers.
without actually "running" the code first to go through the co_await, which are the only way to acquire the information of what is to run ?
for instance, let's say that I want to compute "statically" (not in the "at compile time" sense but in the "after knowing the inputs to the program and before actually running the scheduler") how many [write] calls there are to schedule (assuming no cycles / infinite loops).
The proposed async "framework" happens to work well with coroutines, but they are not required (and infact most of the papers on the topic deal with them only in passing). You can build the async graph as an expression template and if you want, each operation (including the while loop) can be expressed as a different type.
I don't think coroutines provide (yet) enough introspection to break them down in the same way.
> if we know that a set of tasks are dependent on each other in ways that aren't visible on the source code, but are known to the programmers, it would make sense to schedule them on a queue in their own thread
Why it would make sense?
In my experience, dedicated threads only makes sense in very specific use cases. Like dealing with APIs which have thread affinity, or when you want OS to prioritize stuff. Generally, throwing tasks into a shared thread pool is the way to go.
> In my experience, dedicated threads only makes sense in very specific use cases. Like dealing with APIs which have thread affinity, or when you want OS to prioritize stuff.
well, yes, those are things that happen fairly often. e.g. there's a lot of work on optimizing code for various arm big.LITTLE things, where you have some algorithm that you want to make sure runs on the "big" cores.
Can’t you use two shared thread pools, with threads affinity set to the big and little parts of the CPU, respectively?
I think for most use cases runtime scheduling is better than compile-time one. Because other processes sharing the hardware, and because tasks take unpredictable time, depends on almost everything including ambient temperature. Not to mention compilers don’t generally know count of cores, outside of a few niches like embedded, iOS and consoles games.
There’s only one C++ library I’ll use for I/O and networking and that’s POCO C++. I’ve used it in a lot of project over the last 10 years and I haven’t seen a better solution.
its open source version has Boost license which is fine, not sure what's the difference between this and its POCO Pro product, it's better than QT license model for sure, but worse than than those "here is _all_ the code, pay me if you need professional service but not the code"
C++'s networking support is one of its greatest shortcomings.
If I had the choice, I'd never use ASIO again and just use libuv.
Coroutines look interesting, I just need to get my head around them. All code examples I've seen, just as with ASIO, deal with the happy path only. For example, there's hardly a mention of timeouts and the faff ASIO makes you do to get them to work. If there's no 'read(..., timeout)' function then the API is broken as far as I'm concerned.
Then you'll (probably) need to write your own HTTP parsing code which, when you're half-way through, will make you realise that HTTP is actually not trivial at all to implement correctly. Best of luck writing your own HTTP2/3 engine. Boost.Beast seems like a great attempt at doing it right, but I've never used it. (HTTP needs to die, but that's a rant for another day)
IMO, it's too late for C++ and networking. If you need to read from something simple like a serial device then fine, it's probably ok. If you actually need to talk across the Internet then there are other languages that will do it better than you can.
Having said all this, personally, I know my limits and could never come up with anything better than ASIO.
Boost.Beast has its own HTTP parser[0], during the development of which Vinnie Falco (the principle author of Beast) found many bugs/inconsistencies in Node.js's own parser[1].
Personally I'd rather use this than something based on Node.js http-parser.
We actually use the original node.js parser code (http_parser), which was originally based _I believe_ on the nginx parser. The Node people have since moved on to generating the C parser code with Typescript because the original code was too difficult to maintain (llparse).
Like I said, human-readable HTTP needs to die, but it's far too late.
llhttp different from the original http-parser. It has lesser bugs because it's written in Typescript which transpiled to C and it's faster than the original http-parser
> If I had the choice, I'd never use ASIO again and just use libuv.
Same here. I ported a legacy project from linux to windows, and continued to use ASIO (i.e., wrote new code) by inertia since the project already used it. Worst decision ever. If I could start from scratch, I'd just use raw Win32 APIs.
And I agree on HTTP(S). It has become an overcomplicated transport protocol on top of another transport protocol (TCP) and the only (though relevant) problem it solves is that it's allowed through most firewalls by default.
Boost Beast, which is built on ASIO, has support for timeouts[0], as well as HTTP 1.1 and Websockets. The examples in the Beast tree do do some pretty complex stuff elegantly, like a multi-client chat server[1], so it's possible.
There's also nghttp2_asio, built on nghttp2, which supports HTTP/2[2].
Overall I agree though that ASIO is a very complex and very low level, and it's a lot of work to use it well.
We actually changed from libcurl to ASIO because libcurl creates a thread for every request, and it didn't handle concurrency well (I forget the exact details but it would crash if you added a curl handle to a curl_multi session if that session was already in progess)
Our use case sends thousands of requests per second per machine.
Nah, that place I leave for C, the number one reason we keep having memory corruption issues during the last 50 years of computer history, and having taited every language that has some level of compatibility with it.
> <sarcasm>Too bad those shitty engineers in the 70s didn't think of Rust...</sarcasm>
True, Rust didn't exist back then, and yes, some (few!) features in Rust weren't invented and worked out then, like the ownership model.
OTOH the corollary to that statement isn't that C was some shining beacon of good design taking advantage of the then state of the art programming language knowledge. See e.g. https://pastebin.com/UAQaWuWG
That pastebin leaves out the fact that C's spartan approach and extreme simplicity allowed C to become the most popular programming language in the world already in the time of K&R, a book still renowned for it's clarity and low mental baggage required to onboard onto a programming language.
The pastebin also left out that Bjarne Stroustrup developed C with classes, and the C, as a backward compatible superset of C to take advantage of C's unrivaled popularity.
We all can agree that the science and art behind programming language design progressed a lot in the four decades where we enjoyed C. However, it feels a kin to rewriting history to depict C as a subpar language that was poorly designed and had little to no redeeming qualities, when facts speak for themselves.
Just like real world programming in C is hardly memory corruption free, and only around 10% of survey answers keep asserting to use any kind of memory assurance tooling.
> However, it feels a kin to rewriting history to depict C as a subpar language that was poorly designed and had little to no redeeming qualities, when facts speak for themselves.
It was subpar compared to the state of the art then. It's redeeming quality was that it worked on the PDP-11, whereas the better languages required mainframe level resources. The tragedy isn't that C was invented, it fitted it's niche back then. The tragedy is that as HW evolved the world remained stuck with C rather than switching to better languages.
Actually, computer science in general and the theory of programming languages in particular were already very advanced back in the 60s. It’s just people thought little of languages that would have needed gigabytes of memory (or days of run time) for the compiler to do its job.
That's exactly the point the pamphlet I linked to makes. C was far from the state of the art when it was invented, it's only major redeeming quality was that it worked on the shit low-end computers that were available to the inventors back then.
The tragedy wasn't that C was invented, it was fit for purpose then. The tragedy was that as HW evolved beyond the PDP-11 level, unix programming remained stuck at C.
They thought many ideas of it (see Lisp, Smalltalk) but computers where not powerful enough for complex compilers. Instead, the burden of ensuring correctness had to be left as the cognitive burden for the developer for decades to come.
> They thought many ideas of it (see Lisp, Smalltalk) but computers where not powerful enough for complex compilers.
That's probably true for borrower-checking specifically, but I don't think it accounts for all language-design breakthroughs. There were decades between the development of functional programming languages, and widespread hybrid languages which incorporated those ideas (e.g. modern Python and C#).
No one can seriously argue that C's language design was inherently advantageous over, say, Pascal. If Lisp, ML, Simula, Algol, or Ada received the amount of optimization and porting investment that C did, history would be very different.
C succeeded because a.) Unix, and b.) there were a zillion platforms with a zillion different compilers that could process something like C. They all did weird things and you couldn't actually write portable code for decades, but you could kind of fake it.
Sure, with more or less effort. C's semantics seem very easy to port, with very little abstraction over the commonalities that all (most?) assemblies share: computer as a giant array with a CPU and registers.
There's a gaping hole in the abstraction when it comes to word size, but that's about it, and it's obvious to diagnose at least.
Compared to the list of languages above with their more advanced features, it's super easy to port a C compiler and start compiling code. Worse is better.
So easy that to actually make C portable, UNIX had to be made into a C standard library via POSIX, otherwise portable C code is basically just logic and data structures.
Sure, but the flipside of that is that UNIX itself is fairly easy to port to a new CPU, due to being mostly written in C.
So port your compiler, figure out a syscall interface on your CPU, patch it into unix/libc and compile them, and.. it's not like that's a small amount of work but it's way easier than bootstrapping any other environment from scratch on a new architecture.
There's surely some amount of historical accident here as well, but no other language/OS combo ran on like 12 different architectures in the 80s.
Actually I'd bet a dollar or two that linear threading of resources existed even before C, it just wasn't neither a common practice nor a theoretical framework at the time.
The late pioneer in compiler optimization--Frances Allen--agrees.
2006: "..but something else had happened--and that was back in 1973--[something] which I was very, very unhappy about--and that was the advent of C! It derailed, as far as I was concerned, the advances that I foresaw in compilers. One cannot let the user play with the addresses, and use procedures, and play with pointers--which is pointing to the data--and be able to understand how one can transform the program to run more optimally."
"There was an interesting debate at the SIGPLAN conference in that period--about the relative need for optimization. It was put on the table that it wasn't going to be needed anymore! So. Anyway. You can have your own opinion on that, but I know I have mine, beyond a doubt that it was a huge setback for--in my opinion--languages and compilers--and the ability to deliver performance--easily--to the user."
She's exactly right. I'm really not sure how we came to accept the worse-is-better argument that high performance required exposing machine details into the source language, and letting compiler writers just "do whatever is fastest", regardless of safety. It gave rise to today's situation where none of application writers, library writers, kernel writers, or compiler writers are quite sure what will happen if a new optimization is introduced. Will it break programs? By how much? Well, let's roll the dice! This is a direct result of allowing--no, even expecting--programmers to peer behind the curtains to see machine details that they frankly, shouldn't. It's why porting programs from one machine to another is even a thing, as opposed to just selecting a different output from a compiler, or running a different VM.
Proper abstractions define exactly what happens in every situation, and design unobservable details out. It is the basis for optimizations that fundamentally cannot observed (except side-channels, like timing and other resource usage). Proper abstractions is why we have fast processors. The ISA is a hard boundary.
> One cannot let the user play with the addresses, and use procedures, and play with pointers--which is pointing to the data--and be able to understand how one can transform the program to run more optimally."
Well 50 years on we still haven't seen any of these "sufficiently smart compilers", at best they can outsmart C compilers in a few cases with things like generics, so the idea that higher level languages would be faster has been wrong for at least 5 decades. At least you can forgive them for thinking they would 50 years ago, but now it's just ignoring 50 years of reality.
Unfortunately, this assumes everyone is a master at C and that we are comparing straight line speed. In the real world, higher level languages have a much better comparison since everyone's C program has their own slow, buggy "standard" library instead of the hyper optimized ones of higher level languages. One of the major reasons why that is the case is because C, at a language design level, makes building those kinds of high level, interesting, performant, and safe APIs hard
> In the real world, higher level languages have a much better comparison since everyone's C program has their own slow, buggy "standard" library instead of the hyper optimized ones of higher level languages.
In the real world we've pissed away 30 years of hardware improvements primarily through switching to higher level languages. In the real world higher level languages are still not used we're performance counts (like games). In the real world low level languages like c and C++ are still powering our most important and fundamental projects.
Have been programming in C (and then C++), like, forever. Hardly any memory corruption issues. Yes, you have to be a little more disciplined and pay attention to things. Being distracted certainly does not help.
I'm less optimistic. Even major C++ projects that do everything right, still have issues that would have been prevented had they used a safe language. My go-to example is Chromium.
Rust doesn't prevents logical bugs. Chromium had plenty of those. Most are capable of RCE. They were more dangerous than memory ones because they bypass sandbox in one-click.
As for Windows, The Russians exploited a logical bug in kernel for privilege escalation.
Rust also doesn't prevents overflows, DoS, UaF, OOB.
For example, see CVE-2018-1000657
Another dangerous thing about Rust is Crates. Crates doesn't audit packages for malware and you will face far worse than NPM like situation in future.
> Rust doesn't prevents logical bugs. Chromium had plenty of those.
Right, but that's not the goalpost we're discussing. We're talking about languages that can guarantee safety - the absence of undefined behaviour - not languages that can fully guarantee correctness (e.g. SPARK).
> Rust also doesn't prevents overflows, DoS, UaF, OOB.
In Rust, integer overflow does not cause undefined behaviour. In Safe Rust, undefined behaviour cannot arise from buffer overflows, use-after-free, or out-of-bounds array access. Safe Rust precludes all undefined behaviour, after all. Unsafe Rust may be 'more safe' than C++ in degree, but not in category: it's an unsafe language, as you say.
It wasn’t undefined behavior four years ago either. RFC 560, from 2014, defines this behavior. As far as I can recall, this was mostly a codification of what was already in place; I’m pretty sure it was never UB.
Here's a source. [0] In release builds, overflow results in two's complement wrapping. In debug builds, it results in a panic at runtime. It never results in undefined behaviour.
Safe Rust really is a safe language. That's really what's remarkable about Rust: it has high ambitions for safety and usability and performance, and it's succeeding in achieving them.
(I hear there are some who want to dilute the safety guarantees of Safe Rust. I'm optimistic that they will continue to be ignored.)
> Relying on integer overflow’s wrapping behavior is considered an error
Your doc also says this.
> I hear there are some who want to dilute the safety guarantees of Safe Rust
Unfortunately, I don't use Rust at work. I can't talk about it anymore, either. I can't use an informal reference to reason about its actual behavior. At the end of the day, C++ puts food on the table. I try to improve C++ as much as possible, knowing that it is an imperfect language. C++ is heading into safe direction, and I'm sure C++26 will be able to provide more features to write code safely.
Rust seriously need to add Function Overloading, Generics.
> I can't use an informal reference to reason about its actual behavior.
I don't follow here. You can do that. There's a more formal reference too though. [0]
> Relying on integer overflow’s wrapping behavior is considered an error
Good point, I'd missed that. So, Rust strongly discourages relying on the wrapping behaviour of release builds. It seems to take the opposite approach that Java uses.
In Java, you get wrapping arithmetic 'by default' (i.e. when using the arithmetic operators), and there are special functions that give throw-on-overflow behaviour (which almost no one uses). [1] In Rust, the arithmetic operators handle overflow either by wrapping or by panic, depending on build configuration, and if you deliberately want to overflow, there are special functions for that (i32::wrapping_add) which behave identically on Debug and Release builds.
I believe Ada does something similar.
> At the end of the day, C++ puts food on the table. I try to improve C++ as much as possible, knowing that it is an imperfect language.
Sure, I use C++ too. I really only know about Rust from a distance, I'm not a Rust programmer. C++ still has considerable advantages: excellent first-class support on all major platforms, a wealth of libraries available, mature tooling (static analysis, dynamic analysis, top-notch IDEs). Of course, some major software frameworks are natively C++, so you're strongly encouraged to stick with C++ (Qt, Direct3D).
> C++ is heading into safe direction, and I'm sure C++26 will be able to provide more features to write code safely.
The C-style footguns will probably still be there in 20 years. Yes, they've given us std::vector and std::array which allow us to manage arrays more safely than raw C arrays, but read-before-write is still undefined behaviour, dereferencing null is still undefined behaviour, divide-by-zero is still undefined behaviour. They gladly add more functionality to the standard library, but they're very reluctant to tweak the core of the language. At least we have two's complement guaranteed by the C++ standard now.
> Rust seriously need to add Function Overloading, Generics.
I wasn't aware Rust lacked generics. That sounds frustrating.
I have mixed feelings on function overloading. It makes it harder to reason about what function is being called.
> read-before-write is still undefined behaviour, dereferencing null is still undefined behaviour, divide-by-zero
You rarely do these in Modern C++.
It's your responsibility to check the input of program before doing anything with it.
Structured Exception Handling aka SEH Exceptions can catch things like divide-by-zero, read-before-write, dereferencing null
> The C-style footguns will probably still be there in 20 years.
It's your job to know these footguns. Actually, it will take 2-3 months for a new programmer to separate C++ and C and understand what Modern C++ actually about.
> I have mixed feelings on function overloading. It makes it harder to reason about what function is being called.
It doesn't because overloading depends on the arguments not on the function names
As I said before, one way or another, undefined behaviour is still a major problem in C++ codebases. This isn't really up for debate. pjmlp provided a good link on this. Most of Chromium's serious security bugs are due to memory safety issues. [0]
> It's your responsibility to check the input of program before doing anything with it.
C++ makes it the programmer's responsibility to avoid undefined behaviour. Programmers are not capable of doing this reliably. We're now at the point that it's no longer credible to argue otherwise. Even after all these decades it's still a problem for C++.
> Structured Exception Handling aka SEH Exceptions can catch things like divide-by-zero, read-before-write, dereferencing null
Sure, and with GCC/Clang, there are equivalent flags to enable trap-on-error for these errors. They're rarely used in production though, so we still face the consequences of undefined behaviour.
> It's your job to know these footguns.
Again, we've seen that C and C++ programmers simply are not capable of avoiding undefined behaviour. Even Google is unable to keep UB out of their prized Chromium codebase, despite their considerable investment.
> It doesn't because overloading depends on the arguments not on the function names
That's the definition of overloading, not a refutation of my point.
In a language like C with no overloading, the function identifier is enough to uniquely identify the function. You don't need to check if there are overloads, the identifier is enough. If you introduce overloading, that property no longer holds. You might work under the mistaken impression there are no other overloads of a function, or it might be non-trivial to determine the types being passed, especially in the presence of type-inference.
It's resolved at compile time, rather than at runtime, but my point stands.
> The thing with memory bugs is that you need another bug for Sandbox to fully exploit the browser.
The quote was Around 70% of our high severity security bugs are memory unsafety problems. Doesn't sound like sandboxing is effective there.
> Here is a real world logical exploit that knock sandbox and Rust won't prevent this stuff,
I'm not sure I see your point here. No one is arguing that Safe Rust makes all logic bugs go away. It's not a formal methods framework, it's a safe programming language. If Chromium had been written in Safe Rust (assuming that's practical), presumably at least 70% of its high-severity security bugs would have been avoided.
You're right that they still wouldn't have perfection. The only shot at perfect security is through formal methods, following SeL4, but as things stand, formal methods cannot scale to a problem as large as a modern web browser. Sadly, even TLS is beyond what we can implement with formal methods.
> It's actually really hard to get modern C++ wrong
That's not true. The language is full of footguns, from its basic expressions (signed overflow, integer division-by-zero, and read-before-write are all undefined behaviour) up to concurrency (data-races are undefined behaviour) and everything in between (unsafe arrays, unsafe memory management, alignment, the list goes on). The high-level constructs in modern C++ help, but you're never further than one misstep from undefined behaviour.
Even high-profile codebases by highly skilled teams using all the modern tools, still run into trouble. Even Chromium has security issues arising from the dangers of C++. The push for Rust is out of pragmatism, not trend-chasing or theoretical purity.
Of course, C codebases like the Linux kernel also continue to have these issues.
If powerhouses like ARM, Apple, Google, Microsoft and even Linux kernel admit not being able to, now even pushing for hardware memory tagging as the only remaining way to fix C and its influence on C++/Objective-C, you are indeed a top expert.
Yeah, "orange site" comments on Twitter are an interesting entertainment genre in general, if you're into "a visit to the bizarro world" type of humor :).
I happen to disagree, C++ is the best thing after sliced bread. (Just wanted to balance the conversation, really, with an equally meaningless, albeit honest, statement.)
I would hesitate to embrace a "universal" I/O abstraction that is architected around a monolithic kernel. Linux and Windows are not the only game in town. This design make make for a nice third-party library for some applications, but it has no place being cemented into the standard library.
> Even though network libraries like NFS and SMB let you treat files on remote machines “as if” they were local, sometimes the connection becomes very slow or goes down, and the file stops acting like it was local, and as a programmer you have to write code to deal with this. The abstraction of “remote file is the same as local file” leaks.
87 comments
[ 3.3 ms ] story [ 147 ms ] threadI think that it would be useful to consider the existing (very very very large) body of research on dataflow PL and APIs; e.g. check this paper from 1982 which talks about dataflow, coroutines, for handling file operations:
- https://scholarship.claremont.edu/cgi/viewcontent.cgi?articl...
More recent papers in the field:
- https://hal.archives-ouvertes.fr/hal-02865894/document
- https://library.oapen.org/bitstream/handle/20.500.12657/3772...
and older general overviews of the field:
- http://www.cs.ucf.edu/~dcm/Teaching/COT4810-Spring2011/Liter...
- http://210.47.10.86:8032/200009-3/65492.pdf#page=141
My main concern with this design, even if the code looks quite neat, is that the actual dataflow graph is not reified, but exists "invisibly" as part of the program source through the way co_await calls are sequenced.
This means that it is not possible to do as an user, a static analysis and scheduling of the tasks : it relies on the compiler to perform this. But, this problem is in my experience quite often very use-case-specific and ridden with informations compilers do not possess. Not having the way tasks are interdependent reified in an actual user-manipulable "graph" object means that users won't have the possibility, to, say, perform a topological sort or flow propagation on their async graph in order to extract information on it that could be necessary to perform a domain-specific optimization of said graph (for instance if we know that a set of tasks are dependent on each other in ways that aren't visible on the source code, but are known to the programmers, it would make sense to schedule them on a queue in their own thread).
If you really want you could even convert the graph (fully or partially) to a runtime representation.
The core of the proposals are not about specific classes or functions other than some default algorithm implementations.
Instead they are mostly about defining concepts and protocols so that independent async domains can coexist and iteroperate in the same application.
This is similar to the STL where the most important part are the various iterator concepts and the algorithm interfaces, not the actual algo implementations and the containers.
I would find that to be great, but I don't understand how.
In e.g.
how can you get a meaningful "graph" object out of that, e.g.[wait 100] -> [write 9] -> [wait 100] -> [write 8] -> ...
without actually "running" the code first to go through the co_await, which are the only way to acquire the information of what is to run ?
for instance, let's say that I want to compute "statically" (not in the "at compile time" sense but in the "after knowing the inputs to the program and before actually running the scheduler") how many [write] calls there are to schedule (assuming no cycles / infinite loops).
I don't think coroutines provide (yet) enough introspection to break them down in the same way.
Why it would make sense?
In my experience, dedicated threads only makes sense in very specific use cases. Like dealing with APIs which have thread affinity, or when you want OS to prioritize stuff. Generally, throwing tasks into a shared thread pool is the way to go.
well, yes, those are things that happen fairly often. e.g. there's a lot of work on optimizing code for various arm big.LITTLE things, where you have some algorithm that you want to make sure runs on the "big" cores.
I think for most use cases runtime scheduling is better than compile-time one. Because other processes sharing the hardware, and because tasks take unpredictable time, depends on almost everything including ambient temperature. Not to mention compilers don’t generally know count of cores, outside of a few niches like embedded, iOS and consoles games.
Gesundheit!
Coroutines look interesting, I just need to get my head around them. All code examples I've seen, just as with ASIO, deal with the happy path only. For example, there's hardly a mention of timeouts and the faff ASIO makes you do to get them to work. If there's no 'read(..., timeout)' function then the API is broken as far as I'm concerned.
Then you'll (probably) need to write your own HTTP parsing code which, when you're half-way through, will make you realise that HTTP is actually not trivial at all to implement correctly. Best of luck writing your own HTTP2/3 engine. Boost.Beast seems like a great attempt at doing it right, but I've never used it. (HTTP needs to die, but that's a rant for another day)
IMO, it's too late for C++ and networking. If you need to read from something simple like a serial device then fine, it's probably ok. If you actually need to talk across the Internet then there are other languages that will do it better than you can.
Having said all this, personally, I know my limits and could never come up with anything better than ASIO.
As for HTTP parsing, llhttp does great job. It however lack multipart parsing.
Personally I'd rather use this than something based on Node.js http-parser.
[0] https://www.boost.org/doc/libs/develop/libs/beast/doc/html/b...
[1] https://github.com/nodejs/http-parser/issues?q=is%3Aissue+au...
Like I said, human-readable HTTP needs to die, but it's far too late.
Same here. I ported a legacy project from linux to windows, and continued to use ASIO (i.e., wrote new code) by inertia since the project already used it. Worst decision ever. If I could start from scratch, I'd just use raw Win32 APIs.
And I agree on HTTP(S). It has become an overcomplicated transport protocol on top of another transport protocol (TCP) and the only (though relevant) problem it solves is that it's allowed through most firewalls by default.
There's also nghttp2_asio, built on nghttp2, which supports HTTP/2[2].
Overall I agree though that ASIO is a very complex and very low level, and it's a lot of work to use it well.
Sometimes it's easiest just to use libcurl.
[0] https://www.boost.org/doc/libs/1_76_0/libs/beast/doc/html/be...
[1] https://www.boost.org/doc/libs/1_70_0/libs/beast/example/web...
[2] http://nghttp2.org/documentation/libnghttp2_asio.html
Our use case sends thousands of requests per second per machine.
https://twitter.com/Cor3ntin/status/1383875999033565194?s=20
<sarcasm>Too bad those shitty engineers in the 70s didn't think of Rust...</sarcasm>
True, Rust didn't exist back then, and yes, some (few!) features in Rust weren't invented and worked out then, like the ownership model.
OTOH the corollary to that statement isn't that C was some shining beacon of good design taking advantage of the then state of the art programming language knowledge. See e.g. https://pastebin.com/UAQaWuWG
The pastebin also left out that Bjarne Stroustrup developed C with classes, and the C, as a backward compatible superset of C to take advantage of C's unrivaled popularity.
We all can agree that the science and art behind programming language design progressed a lot in the four decades where we enjoyed C. However, it feels a kin to rewriting history to depict C as a subpar language that was poorly designed and had little to no redeeming qualities, when facts speak for themselves.
It was subpar compared to the state of the art then. It's redeeming quality was that it worked on the PDP-11, whereas the better languages required mainframe level resources. The tragedy isn't that C was invented, it fitted it's niche back then. The tragedy is that as HW evolved the world remained stuck with C rather than switching to better languages.
The tragedy wasn't that C was invented, it was fit for purpose then. The tragedy was that as HW evolved beyond the PDP-11 level, unix programming remained stuck at C.
That's probably true for borrower-checking specifically, but I don't think it accounts for all language-design breakthroughs. There were decades between the development of functional programming languages, and widespread hybrid languages which incorporated those ideas (e.g. modern Python and C#).
C succeeded because a.) Unix, and b.) there were a zillion platforms with a zillion different compilers that could process something like C. They all did weird things and you couldn't actually write portable code for decades, but you could kind of fake it.
Just marketing speak.
There's a gaping hole in the abstraction when it comes to word size, but that's about it, and it's obvious to diagnose at least.
Compared to the list of languages above with their more advanced features, it's super easy to port a C compiler and start compiling code. Worse is better.
So port your compiler, figure out a syscall interface on your CPU, patch it into unix/libc and compile them, and.. it's not like that's a small amount of work but it's way easier than bootstrapping any other environment from scratch on a new architecture.
There's surely some amount of historical accident here as well, but no other language/OS combo ran on like 12 different architectures in the 80s.
https://www.bell-labs.com/usr/dmr/www/chist.html
2006: "..but something else had happened--and that was back in 1973--[something] which I was very, very unhappy about--and that was the advent of C! It derailed, as far as I was concerned, the advances that I foresaw in compilers. One cannot let the user play with the addresses, and use procedures, and play with pointers--which is pointing to the data--and be able to understand how one can transform the program to run more optimally."
"There was an interesting debate at the SIGPLAN conference in that period--about the relative need for optimization. It was put on the table that it wasn't going to be needed anymore! So. Anyway. You can have your own opinion on that, but I know I have mine, beyond a doubt that it was a huge setback for--in my opinion--languages and compilers--and the ability to deliver performance--easily--to the user."
https://en.m.wikipedia.org/wiki/Frances_Allen
https://www.youtube.com/watch?v=NjoU-MjCws4&t=2450
Proper abstractions define exactly what happens in every situation, and design unobservable details out. It is the basis for optimizations that fundamentally cannot observed (except side-channels, like timing and other resource usage). Proper abstractions is why we have fast processors. The ISA is a hard boundary.
Well 50 years on we still haven't seen any of these "sufficiently smart compilers", at best they can outsmart C compilers in a few cases with things like generics, so the idea that higher level languages would be faster has been wrong for at least 5 decades. At least you can forgive them for thinking they would 50 years ago, but now it's just ignoring 50 years of reality.
I also happen to remember languages like Fortran and Ada do exist.
In the real world we've pissed away 30 years of hardware improvements primarily through switching to higher level languages. In the real world higher level languages are still not used we're performance counts (like games). In the real world low level languages like c and C++ are still powering our most important and fundamental projects.
Source, replacing GUI and distributed applications written in C and C++ with managed languages (AOT and JIT) for the last 20 years.
https://isocpp.org/files/papers/CppDevSurvey-2021-04-summary...
Chromium is a good example actually, I bet they would rather follow the "Custom C++ libraries" and "Hardware mitigations" than the Firefox approach.
https://www.chromium.org/Home/chromium-security/memory-safet...
Apple did something similar recently, their iBoot firmware uses a custom safe C dialect.
> The Chromium project finds that around 70% of our serious security bugs are memory safety problems.
I'll point to this next time this topic crops up.
As for Windows, The Russians exploited a logical bug in kernel for privilege escalation.
Rust also doesn't prevents overflows, DoS, UaF, OOB.
For example, see CVE-2018-1000657
Another dangerous thing about Rust is Crates. Crates doesn't audit packages for malware and you will face far worse than NPM like situation in future.
Right, but that's not the goalpost we're discussing. We're talking about languages that can guarantee safety - the absence of undefined behaviour - not languages that can fully guarantee correctness (e.g. SPARK).
> Rust also doesn't prevents overflows, DoS, UaF, OOB.
In Rust, integer overflow does not cause undefined behaviour. In Safe Rust, undefined behaviour cannot arise from buffer overflows, use-after-free, or out-of-bounds array access. Safe Rust precludes all undefined behaviour, after all. Unsafe Rust may be 'more safe' than C++ in degree, but not in category: it's an unsafe language, as you say.
> Another dangerous thing about Rust is Crates.
Again, sure.
Source please?
I ask this since Rust doesn't have a formal specification and I can't keep up with it's inner changes.
It did cause undefined behavior in my case but that was 4 years ago.
Safe Rust really is a safe language. That's really what's remarkable about Rust: it has high ambitions for safety and usability and performance, and it's succeeding in achieving them.
(I hear there are some who want to dilute the safety guarantees of Safe Rust. I'm optimistic that they will continue to be ignored.)
[0] https://doc.rust-lang.org/book/ch03-02-data-types.html#integ...
Your doc also says this.
> I hear there are some who want to dilute the safety guarantees of Safe Rust
Unfortunately, I don't use Rust at work. I can't talk about it anymore, either. I can't use an informal reference to reason about its actual behavior. At the end of the day, C++ puts food on the table. I try to improve C++ as much as possible, knowing that it is an imperfect language. C++ is heading into safe direction, and I'm sure C++26 will be able to provide more features to write code safely.
Rust seriously need to add Function Overloading, Generics.
I don't follow here. You can do that. There's a more formal reference too though. [0]
> Relying on integer overflow’s wrapping behavior is considered an error
Good point, I'd missed that. So, Rust strongly discourages relying on the wrapping behaviour of release builds. It seems to take the opposite approach that Java uses.
In Java, you get wrapping arithmetic 'by default' (i.e. when using the arithmetic operators), and there are special functions that give throw-on-overflow behaviour (which almost no one uses). [1] In Rust, the arithmetic operators handle overflow either by wrapping or by panic, depending on build configuration, and if you deliberately want to overflow, there are special functions for that (i32::wrapping_add) which behave identically on Debug and Release builds.
I believe Ada does something similar.
> At the end of the day, C++ puts food on the table. I try to improve C++ as much as possible, knowing that it is an imperfect language.
Sure, I use C++ too. I really only know about Rust from a distance, I'm not a Rust programmer. C++ still has considerable advantages: excellent first-class support on all major platforms, a wealth of libraries available, mature tooling (static analysis, dynamic analysis, top-notch IDEs). Of course, some major software frameworks are natively C++, so you're strongly encouraged to stick with C++ (Qt, Direct3D).
> C++ is heading into safe direction, and I'm sure C++26 will be able to provide more features to write code safely.
The C-style footguns will probably still be there in 20 years. Yes, they've given us std::vector and std::array which allow us to manage arrays more safely than raw C arrays, but read-before-write is still undefined behaviour, dereferencing null is still undefined behaviour, divide-by-zero is still undefined behaviour. They gladly add more functionality to the standard library, but they're very reluctant to tweak the core of the language. At least we have two's complement guaranteed by the C++ standard now.
> Rust seriously need to add Function Overloading, Generics.
I wasn't aware Rust lacked generics. That sounds frustrating.
I have mixed feelings on function overloading. It makes it harder to reason about what function is being called.
[0] https://doc.rust-lang.org/reference/behavior-not-considered-...
[1] https://docs.oracle.com/en/java/javase/16/docs/api/java.base...
You rarely do these in Modern C++.
It's your responsibility to check the input of program before doing anything with it.
Structured Exception Handling aka SEH Exceptions can catch things like divide-by-zero, read-before-write, dereferencing null
> The C-style footguns will probably still be there in 20 years.
It's your job to know these footguns. Actually, it will take 2-3 months for a new programmer to separate C++ and C and understand what Modern C++ actually about.
> I have mixed feelings on function overloading. It makes it harder to reason about what function is being called.
It doesn't because overloading depends on the arguments not on the function names
Without overloading, things become ugly.
As I said before, one way or another, undefined behaviour is still a major problem in C++ codebases. This isn't really up for debate. pjmlp provided a good link on this. Most of Chromium's serious security bugs are due to memory safety issues. [0]
> It's your responsibility to check the input of program before doing anything with it.
C++ makes it the programmer's responsibility to avoid undefined behaviour. Programmers are not capable of doing this reliably. We're now at the point that it's no longer credible to argue otherwise. Even after all these decades it's still a problem for C++.
> Structured Exception Handling aka SEH Exceptions can catch things like divide-by-zero, read-before-write, dereferencing null
Sure, and with GCC/Clang, there are equivalent flags to enable trap-on-error for these errors. They're rarely used in production though, so we still face the consequences of undefined behaviour.
> It's your job to know these footguns.
Again, we've seen that C and C++ programmers simply are not capable of avoiding undefined behaviour. Even Google is unable to keep UB out of their prized Chromium codebase, despite their considerable investment.
> It doesn't because overloading depends on the arguments not on the function names
That's the definition of overloading, not a refutation of my point.
In a language like C with no overloading, the function identifier is enough to uniquely identify the function. You don't need to check if there are overloads, the identifier is enough. If you introduce overloading, that property no longer holds. You might work under the mistaken impression there are no other overloads of a function, or it might be non-trivial to determine the types being passed, especially in the presence of type-inference.
It's resolved at compile time, rather than at runtime, but my point stands.
> Without overloading, things become ugly.
It's a tradeoff.
[0] https://news.ycombinator.com/item?id=26862330
Here is a real world logical exploit that knock sandbox and Rust won't prevent this stuff,
https://bugs.chromium.org/p/chromium/issues/detail?id=386988
The quote was Around 70% of our high severity security bugs are memory unsafety problems. Doesn't sound like sandboxing is effective there.
> Here is a real world logical exploit that knock sandbox and Rust won't prevent this stuff,
I'm not sure I see your point here. No one is arguing that Safe Rust makes all logic bugs go away. It's not a formal methods framework, it's a safe programming language. If Chromium had been written in Safe Rust (assuming that's practical), presumably at least 70% of its high-severity security bugs would have been avoided.
You're right that they still wouldn't have perfection. The only shot at perfect security is through formal methods, following SeL4, but as things stand, formal methods cannot scale to a problem as large as a modern web browser. Sadly, even TLS is beyond what we can implement with formal methods.
This quote simply means there are 70% high severity security bugs and it doesn't implies anything about sandbox.
You can have a use-after-free exploit but it's worthless without a sandbox escape.
Sandboxing is very effective at memory bugs but certainly bad at logical bugs.
That's not true. The language is full of footguns, from its basic expressions (signed overflow, integer division-by-zero, and read-before-write are all undefined behaviour) up to concurrency (data-races are undefined behaviour) and everything in between (unsafe arrays, unsafe memory management, alignment, the list goes on). The high-level constructs in modern C++ help, but you're never further than one misstep from undefined behaviour.
Even high-profile codebases by highly skilled teams using all the modern tools, still run into trouble. Even Chromium has security issues arising from the dangers of C++. The push for Rust is out of pragmatism, not trend-chasing or theoretical purity.
Of course, C codebases like the Linux kernel also continue to have these issues.
"I am bracing for the inevitable comments about C++ being the most terrible thing since forever :("
> Even though network libraries like NFS and SMB let you treat files on remote machines “as if” they were local, sometimes the connection becomes very slow or goes down, and the file stops acting like it was local, and as a programmer you have to write code to deal with this. The abstraction of “remote file is the same as local file” leaks.
Worst contender is Java with
Appendable append(CharSequence csq) throws IOException
where Appendable is also used for a StringBuffer.