C++ is a language, while iostream is one of the many C++ tools available to write to the standard output. It’s available by default, it’s part of the standard library, but you don’t have to use it. printf is also available, and will be as fast as using it from C.
Note that in most cases, the speed of writing to standard output is irrelevant unless you are implementing cat or some command line text processing tool.
I find iostream very convenient and type safety is a big plus compared to printf. It’s also very easy to implement your own handlers for your own types.
C++ adds a layer of functional indirection above the level of the C structure. This is what causes the bloat. The implementation of the C++ object vs. the C struct.
Iostream is implemented as an object.
C++ is definitely more convenient and that was the intent of its creators. It does not replace C but provides a more convenient alternative to using C structs and fn pointers at the cost of a small performance penalty (maybe slow and bloated is over-expressing the penalty).
This "bloat" of C objects you mention is only there if you use virtual functions. Same as using function pointers in a C struct. But in some cases C++ can avoid the dynamic indirection thanks to templates like in std::sort while C's qsort gets passed a function pointer, called for each comparison.
Iostream do use virtual functions but it’s not the reason of its suboptimal performance. The main reason is that some strings will be copied, allocated and freed during the construction of the whole expression while printf can do its processing with minimal dynamic allocations.
No virtual fns means no inheritance.
Also, you can blame the implementation and not the language.
But real-world it is slower. Can't say by how much. Why bother?
I don't believe anyone tries to hide it. Except a couple of well-known cases, C++ is (relatively slightly) slower than C, but it's far more powerful. You can argue about compile times - yes, there is a huge difference here; also some specific features like virtual methods are slow - but overall, C++ is one of the fastest languages available.
Of course, same code compiles into same asm, but in original article author is comparing different source code for C++, which includes iostream and stdlib.h.
Strictly speaking yes (or the "C++ version" could replace std::endl with \n), but the observable behavior isn't going to change much. The runtime is going to flush the output buffer at program exit anyway.
That's not really what I mean (I meant that it's not going to result in any difference in which system calls are made etc). If you flush manually, the runtime-provided routine that closes all streams (see e.g. https://www.gnu.org/software/libc/manual/html_node/Normal-Te...) will just do nothing, so you get a little bit of overhead from that
> What we are testing by running these programs is the overhead due to the choice of programming language.
That's just wrong. The first example is a perfectly fine hello world in C++. This is overhead due to the choice of API used to print to stdout
It's well known that a typical iostream implementation is slower than a typical printf implementation, which is why printf is still used in C++ and one of the reasons we got std::format in C++20
People get to be professors because they spend their time investigating things they don't understand (that is, that they're ignorant about), not because they spend all their time doing things they already understand well. You just aren't used to seeing that process in action (it usually isn't blogged), so you project your own embarrassment onto Lemire, who doesn't deserve it.
Well, I would expect a professor to not waste their time experimenting to learn something that's already been documented. So yes, there are things a professor should be embarrassed for not knowing.
Because you have experience enough to know what is and isn’t documented. If you don’t have enough experience, you probably shouldn’t be publishing your opinions.
If you don't publish your wrong opinions, nobody will tell you they're wrong, and you'll stay ignorant. Ignorance is nothing to be ashamed of, but it's also not something you should protect like a precious jewel!
There are times when a month in the lab will save you an afternoon in the library, yes. But there are also times when a month in the library will save you an afternoon in the lab.
I categorically deny that anyone should ever be embarrassed for not knowing anything. For attempting plans without bothering to find out, maybe. Or for claiming to know things they don't. But not for the ignorance itself.
It seems you're implying that no knowledge is required to be a professor, only a particular methodology. I guess we have different expectations of what a professor is.
"Investigating" means taking apart and systematically analyzing, exponentially cutting down the problem space. Making hypotheses, validating them, discarding the unconfirmed ones, and navigating further down the causal tree until you find the root cause. What this blog post is all about is called "bitching", pardon my French.
And it kinda makes sense given the tremendous pay gap between the academia and industry in Computer Science. So if instead of joining a startup or a corporate research team, you decide to sit in a University job, you are either lazy, or don't like accountability, or both. Sorry.
There are a lot of perqs to academic jobs that aren't provided by industry, some of which can be bought with money (like sabbatical years) and some of which can't (like the freedom to speak your mind, or job security). Some people value them more than money, sometimes correctly. Also, there are a very small number of industry jobs whose metric of fitness is academic; most are looking for people who can get profitable things done, not find novel things out.
I would describe this blog post as an initial note on an investigation in progress, rather than a report of a completed investigation. You can see that he did take apart the program and systematically analyze its performance; he just hasn't gotten far enough yet to see if the problem is the C++ compiler or, say, iostream.
Since you've put yourself in the position of measuring your dick against Lemire's, what's your most influential research publication?
>like the freedom to speak your mind, or job security
Nope, it only works as long as you play along with the political games the University administration is playing. Speak out too much and you will find problems getting funding/equipment/people and will be stuck in a basement office working on a 10-year-old laptop.
It gets worse, if you refuse to kiss ass to a particular boss in industry, you can always shop around and find another job. As long as you are competent and know how to sell yourself, you will be fine. In academia, one ruined relation can permanently screw up your entire career. I wouldn't call it freedom. I would dare say, a fairly recognizable blog in a relevant area, and a professional network of ~50 people will give you orders of magnitude more freedom than any kind of academic track record.
>Since you've put yourself in the position of measuring your dick against Lemire's, what's your most influential research publication?
Well, I didn't start it, but since you want to engage in penisometry, I managed to create a product and build a business around it, that pays more than a salaried job would. And I would dare say, comparing academic citations to paying customers is like being paid in monopoly money vs. the real thing. But I realize that the risk/reward ratio of this approach is not for everyone, so I normally prefer to keep my dick in my pants.
Huh? It measures 2 idiomatic ways to print a string. Printf is faster, it's expected. This is just a note checking common sense. Totally worth doing it, especially for a scientist!
Which makes his first example valid C++ code down to the same set of machine code as a C compiler would generate, thus a pointless article from a person that should know better.
I agree completely. Lemire's blog is usually a gold mine when it comes to detailed low level algorithms and encoding problems but this is the second time I've seen a post just come off so amateurish.
And I've been a huge fan of his.
He just comes off as really removed from whole program and language wide thinking — and maybe he's just a bit too distanced from it given his specialty.
That surprises me. If you had asked me, I'd have said that with normal optimizations switched on, both programs would boil down to exactly the same single function call that copies a byte-array to the stdout stream. I would have to wonder if there any optimisations someone has missed if there's even a tiny difference.
In the C world, I could image there's a loop at runtime scanning the printf string for % characters, but I would equally imagine that the compiler people have made a special case for printf that silently replaces printf calls with a single string literal parameter with no %s are replaced with a puts call. (Which itself gets optimised to to a byte-array copy.)
Iostream's performance problems have been well known for decades. This post is striking shallow and uninformed for something coming from a professor "focused on software performance".
If we're talking about the majority of C++ code I see in commercial systems, then `std::endl` is definitely used more than `'\n'` to emit a line-break.
(std::endl also has the benefit of doing the right thing across different operating systems)
I wouldn't have expected that. Is it compiler-specific? Because it sounds totally broken, but maybe the language spec covers this and I didn't realize it.
Similar to FTP, iostreams have a concept of text mode and binary mode for streams. In text mode, newlines are translated and in binary mode they are not. Which mode std::cout is in by default is implementation-defined just to be obnoxious. VC++ defaults to text mode.
In the code I write and review I care about the buffer-flush semantics and will revise, or ask for a revision, if excessive flushing will result from too many std::endl uses when generating large volumes of output. It isn't just an academic point; you might want to look at that code to see if std::endl is justified. It is appropriate if you want to guarantee that a given record reaches the output file quickly, and if not much is written it's a don't-care.
That's my policy too. I was pointing out that what we consider best practice isn't necessarily common practice. So the original story might benefit from additional nuance / distinctions.
Besides the std::endl gotcha others already mentioned: IIRC, just including the <iostream> header is enough to make it run some extra initialization code before main().
The author did not bother to check whether the delay comes from the library loading time vs. the actual string formatting.
He is complaining about a single millisecond difference (that is near the measurement error) and haven't bothered to run the entire thing in a loop and get the difference to something meaningful (where the terminal performance can eat away any noticeable differences).
He picked a completely unrealistic metric (console output is usually meant to be read by humans, so it was never optimized for +- millisecond performance).
He doesn't distinguish between the language (C++) and the I/O API (printf() vs iostream).
The main advantage of C++ is that if you bother to understand how it works, you can let the compiler/debugger do a lot of mind-numbing and error-prone work that C developers proudly do by hand (collections, RAII, inline templates with concepts) at exactly 0.0 runtime overhead. The key is "if you bother to understand how it works". If you randomly pick an arbitrary API and complain how it's worse than a different arbitrary API from C for an arbitrary use case, it only shows your own incompetence.
> I do not believe that printing ‘hello world’ itself should be slower or faster in C++, at least not significantly. What we are testing by running these programs is the overhead due to the choice of programming language.
> The post is "Hello world" is slower in C++, not "C++ is slower" or anything else.
The post pretty clearly blames "C++" if you read more than the title - note the "due to C++" here:
>> Yet if these numbers are to be believed, there is a significant penalty due to C++ for tiny program executions, under Linux.
> The speed of the Hello world application is the metric
And it's a poor metric, mostly testing process setup/teardown.
> and they used hyperfine - a benchmarking tool.
And benchmarking tools are only as good as their usage and application.
A more apples-to-apples comparison would be to use `\n` instead of `std::endl`, and `std::ios_sync_with_stdio(false)` to avoid excessive syncronization with C I/O. Admittedly, neither particularly helps here: I observe most of the overhead the article does merely by switching out clang/gcc for clang++/g++ on the same C source code, with ~6ms-9ms total runtime under wsl, or ~9ms (no measurable difference between the C++ or C code) when compiled with cl 19.15.26732.1 and run on windows. This is mostly measuring process setup/teardown and OS I/O overhead: redirecting stdout to null, I don't see a significant perf impact until I put the printing in some rather large loops:
#include <stdio.h>
#include <stdlib.h>
int main() {
for (int i=0; i<1000000; ++i) printf("hello world\n");
return EXIT_SUCCESS;
}
#include <iostream>
#include <stdlib.h>
int main() {
std::ios::sync_with_stdio(false);
for (int i=0; i<1000000; ++i) std::cout << "hello world\n";
return EXIT_SUCCESS;
}
Guess which is faster? The C++ version, oddly enough, both on wsl ubuntu linux:
/mnt/c/local/ben$ clang++ -Os hello_world.cpp && cargo run --bin bench --quiet --release
average runtime: 36.78434ms
/mnt/c/local/ben$ clang -Os hello_world.c && cargo run --bin bench --quiet --release
average runtime: 86.33667ms
/mnt/c/local/ben$ clang++ -O3 hello_world.cpp && cargo run --bin bench --quiet --release
average runtime: 36.34467ms
/mnt/c/local/ben$ clang -O3 hello_world.c && cargo run --bin bench --quiet --release
average runtime: 86.75787ms
And on windows:
C:\local\ben>cl /nologo /O2 hello_world.cpp && cargo run --bin bench --quiet --release
hello_world.cpp
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.15.26726\include\xlocale(319): warning C4530: C++ exception handler used, but unwind semantics are not enabled. Specify /EHsc
average runtime: 1.44849203s
C:\local\ben>cl /nologo /O2 hello_world.c && cargo run --bin bench --quiet --release
hello_world.c
average runtime: 1.75088561s
Take these numbers with a massive grain of salt: Launching the first subprocess is extra expensive (on the order of 90-100ms) and included in the above averages, distoring them, presumably from delayed loading or initialization of a library in the benchmarking process as subsequent runs of the benchmarking process show the same overhead for the first subprocess[1]. Meanwhile, the author of the original article is calling overhead that has to be rounded up to 1ms a "huge penalty".
(EDIT[1]: to rule out first-access overhead being from windows defender or similar, I confirmed that launching a sacrificial, unrelated, first subprocess such as "cmd /C ver" eliminates the overhead for the first "./a.out" execution)
> However, you're testing the speed to print something, not the application.
I'm testing what impact modifications to the application have, to better understand what is and isn't contributing to the original application's performance and bottlenecks. I've shown how far I have to modify the original to actually start to hit I/O bottlenecks, which by extension, also shows how little they contribute to the bottlenecks of the original application, despite people looking to blame it for the "C" vs "C++" differences elsewhere in the discussion tree.
> I'm assuming they wanted to include the process start and teardown.
And there are interesting discussions to have about that in terms of breaking down OS overhead, API overhead, first launch vs subsequent launches, etc. - none of which the original post bothered with. We could peek at a more realistic workflow involving, say, an xargs-spammed executable processing a piped stream, if we really wanted to simulate a startup/teardown heavy workload. We could fire up `perf` to prepare to profile - I at least installed the frontend before realizing actually using it on wsl requires compiling stuff.
The post didn't bother with any of that. And to be fair, I didn't bother too much either. Low-effort on my part as well ;)
The author knows C++ (https://github.com/simdjson/simdjson) and writes a lot about his performance experiments. I don't see why he - or anybody else - shouldn't raise such questions and arguments without having people (like you) getting angry about it. Is it offensive?
Anyway, he has a point, `cout` is used extensively as a logging mechanism. If you don't see that "single millisecond" making any difference, you certainly haven't work on a relevant system.
66 comments
[ 3.0 ms ] story [ 154 ms ] threadNote that in most cases, the speed of writing to standard output is irrelevant unless you are implementing cat or some command line text processing tool.
I find iostream very convenient and type safety is a big plus compared to printf. It’s also very easy to implement your own handlers for your own types.
Iostream do use virtual functions but it’s not the reason of its suboptimal performance. The main reason is that some strings will be copied, allocated and freed during the construction of the whole expression while printf can do its processing with minimal dynamic allocations.
https://godbolt.org/z/fTPM35Erq
As the "C code" is also valid C++.
> A simple C program might print ‘hello world’ on screen:
> ....
> You can write the equivalent in C++:
He could keep using his original C example and simply change from gcc into g++.
But then he wouldn't be able to use "C is better than C++", when using stdio for both languages.
That's just wrong. The first example is a perfectly fine hello world in C++. This is overhead due to the choice of API used to print to stdout
It's well known that a typical iostream implementation is slower than a typical printf implementation, which is why printf is still used in C++ and one of the reasons we got std::format in C++20
In general, the ability to search human knowledge (especially in deeply technical fields) has vastly outstripped the generation of such knowledge.
I categorically deny that anyone should ever be embarrassed for not knowing anything. For attempting plans without bothering to find out, maybe. Or for claiming to know things they don't. But not for the ignorance itself.
A serious person would have at least drilled down into the assembly, this is just an incredibly childish experiment with no follow-up.
And it kinda makes sense given the tremendous pay gap between the academia and industry in Computer Science. So if instead of joining a startup or a corporate research team, you decide to sit in a University job, you are either lazy, or don't like accountability, or both. Sorry.
I would describe this blog post as an initial note on an investigation in progress, rather than a report of a completed investigation. You can see that he did take apart the program and systematically analyze its performance; he just hasn't gotten far enough yet to see if the problem is the C++ compiler or, say, iostream.
Since you've put yourself in the position of measuring your dick against Lemire's, what's your most influential research publication?
Nope, it only works as long as you play along with the political games the University administration is playing. Speak out too much and you will find problems getting funding/equipment/people and will be stuck in a basement office working on a 10-year-old laptop.
It gets worse, if you refuse to kiss ass to a particular boss in industry, you can always shop around and find another job. As long as you are competent and know how to sell yourself, you will be fine. In academia, one ruined relation can permanently screw up your entire career. I wouldn't call it freedom. I would dare say, a fairly recognizable blog in a relevant area, and a professional network of ~50 people will give you orders of magnitude more freedom than any kind of academic track record.
>Since you've put yourself in the position of measuring your dick against Lemire's, what's your most influential research publication?
Well, I didn't start it, but since you want to engage in penisometry, I managed to create a product and build a business around it, that pays more than a salaried job would. And I would dare say, comparing academic citations to paying customers is like being paid in monopoly money vs. the real thing. But I realize that the risk/reward ratio of this approach is not for everyone, so I normally prefer to keep my dick in my pants.
PS The author is... not just any professor.
He could have avoided writing all that nonsense by using a C++ compiler on his "C" code with 100% the same machine code being generated.
And I've been a huge fan of his.
He just comes off as really removed from whole program and language wide thinking — and maybe he's just a bit too distanced from it given his specialty.
In the C world, I could image there's a loop at runtime scanning the printf string for % characters, but I would equally imagine that the compiler people have made a special case for printf that silently replaces printf calls with a single string literal parameter with no %s are replaced with a puts call. (Which itself gets optimised to to a byte-array copy.)
std::cout << "hello world" << std::endl;
but is, rather,
std::cout << "hello world\n";
The std::endl doesn't just print an '\n' character, it also does a buffer flush.
If we're talking about the majority of C++ code I see in commercial systems, then `std::endl` is definitely used more than `'\n'` to emit a line-break.
(std::endl also has the benefit of doing the right thing across different operating systems)
One isn't forced to use iostreams, format, or now the new kid in town, print.
The author did not bother to check whether the delay comes from the library loading time vs. the actual string formatting.
He is complaining about a single millisecond difference (that is near the measurement error) and haven't bothered to run the entire thing in a loop and get the difference to something meaningful (where the terminal performance can eat away any noticeable differences).
He picked a completely unrealistic metric (console output is usually meant to be read by humans, so it was never optimized for +- millisecond performance).
He doesn't distinguish between the language (C++) and the I/O API (printf() vs iostream).
The main advantage of C++ is that if you bother to understand how it works, you can let the compiler/debugger do a lot of mind-numbing and error-prone work that C developers proudly do by hand (collections, RAII, inline templates with concepts) at exactly 0.0 runtime overhead. The key is "if you bother to understand how it works". If you randomly pick an arbitrary API and complain how it's worse than a different arbitrary API from C for an arbitrary use case, it only shows your own incompetence.
The speed of the Hello world application is the metric and they used hyperfine - a benchmarking tool.
> I do not believe that printing ‘hello world’ itself should be slower or faster in C++, at least not significantly. What we are testing by running these programs is the overhead due to the choice of programming language.
The post pretty clearly blames "C++" if you read more than the title - note the "due to C++" here:
>> Yet if these numbers are to be believed, there is a significant penalty due to C++ for tiny program executions, under Linux.
> The speed of the Hello world application is the metric
And it's a poor metric, mostly testing process setup/teardown.
> and they used hyperfine - a benchmarking tool.
And benchmarking tools are only as good as their usage and application.
A more apples-to-apples comparison would be to use `\n` instead of `std::endl`, and `std::ios_sync_with_stdio(false)` to avoid excessive syncronization with C I/O. Admittedly, neither particularly helps here: I observe most of the overhead the article does merely by switching out clang/gcc for clang++/g++ on the same C source code, with ~6ms-9ms total runtime under wsl, or ~9ms (no measurable difference between the C++ or C code) when compiled with cl 19.15.26732.1 and run on windows. This is mostly measuring process setup/teardown and OS I/O overhead: redirecting stdout to null, I don't see a significant perf impact until I put the printing in some rather large loops:
Guess which is faster? The C++ version, oddly enough, both on wsl ubuntu linux: And on windows: Take these numbers with a massive grain of salt: Launching the first subprocess is extra expensive (on the order of 90-100ms) and included in the above averages, distoring them, presumably from delayed loading or initialization of a library in the benchmarking process as subsequent runs of the benchmarking process show the same overhead for the first subprocess[1]. Meanwhile, the author of the original article is calling overhead that has to be rounded up to 1ms a "huge penalty".(EDIT[1]: to rule out first-access overhead being from windows defender or similar, I confirmed that launching a sacrificial, unrelated, first subprocess such as "cmd /C ver" eliminates the overhead for the first "./a.out" execution)
However, you're testing the speed to print something, not the application. I mean, I'm assuming they wanted to include the process start and teardown.
But sure, the test isn't fair.
I'm testing what impact modifications to the application have, to better understand what is and isn't contributing to the original application's performance and bottlenecks. I've shown how far I have to modify the original to actually start to hit I/O bottlenecks, which by extension, also shows how little they contribute to the bottlenecks of the original application, despite people looking to blame it for the "C" vs "C++" differences elsewhere in the discussion tree.
> I'm assuming they wanted to include the process start and teardown.
And there are interesting discussions to have about that in terms of breaking down OS overhead, API overhead, first launch vs subsequent launches, etc. - none of which the original post bothered with. We could peek at a more realistic workflow involving, say, an xargs-spammed executable processing a piped stream, if we really wanted to simulate a startup/teardown heavy workload. We could fire up `perf` to prepare to profile - I at least installed the frontend before realizing actually using it on wsl requires compiling stuff.
The post didn't bother with any of that. And to be fair, I didn't bother too much either. Low-effort on my part as well ;)
Anyway, he has a point, `cout` is used extensively as a logging mechanism. If you don't see that "single millisecond" making any difference, you certainly haven't work on a relevant system.
Even doing this:
* gcc max performance compilation option (-O3) https://godbolt.org/z/18ejPnzrj
* gcc min compilied size compilation options: https://godbolt.org/z/EK1Tzznes
without any written work will give much more insights into differences between C++ and C implementations
https://godbolt.org/z/8jYh79e7e
Second, everyone ones knows that iostream flexibility and type safety comes with a price.
Third, the modern way is to use fmt, or std::format.