$ ./demo
Smooth |=====================================================| ETA: 0h00m06s
Three Second Task with a long label |========================| ETA: 0h00m03s
Fast |=======================================================| ETA: 0h00m02s
Custom <.....................................................> ETA: 0h00m06s
Indeterminate: 0:00:03
Status bar with a really long label: 0:00:01
Custom: 0:00:03
The API doesn't let you set the progress, you must know in advance how many steps there are and then call the increment function that exact number of steps. This is not conducive to, for example, a network transfer where you have some total number of bytes, and then you receive a packet of some arbitrary size and you want to set the progress to a percent.
As mentioned earlier, pv have been around for ages and does this "the right way" (tm). Also, as an exercise, spot the potential buffer overflow in this source in less than 10 seconds for extra credits!
The correct solution would be a) check the length of format, and if it less than or equal to 4 but greater than 0, then copy the bytes, else, don't... I guess? Why is 4 such an arbitrary chosen number?
// new->format is allocated to 4 bytes
size_t len = strlen(format);
if(len > 0 && len <= 4) {
memcpy(new->format, format, len); //i to < 4
new->format[3] = '\0';
}
Or it just be better to allocate the new->length to the return value of strlen(format) + 1.
Yes it was. Good catch. The size of the buffer is 4, so the last byte would be located at 4-1 = index 3. Because we're 0 based...Wow can't believe I did that. it's 12:30 AM here and I've had a few beers tonight.
It's common to allocate your size needed plus 1 for the NULL byte. But this programmer only allocates 4 bytes.. with no thought of the NULL byte at the end.
Please be. Seemingly random bits of (honestly - actually very useful) information like yours are why I'm reading HN in the first place instead of, say, random Youtube comments.
Someone may argue about this being overly pedantic, or about such comment being more suitable for sites like Stack Overflow, or about the whole thing being completely clear from the standards documentation itself... etc. But that's really not the point of it when this isn't the kind of thing you would be even intentionally looking at, or for, in the first place.
Every time I open a HN thread, I can be sure that I will find at least a good few nice and memorable comments that seemingly don't add anything revolutionary to the topic at hand, but instead will make you pause, slap your forehead and exclaim "holy moly, that's in essence actually perfectly obvious, yet somehow I never even thought about it from this point of view before!" Which in turn suddenly makes you realize something completely unrelated in your own world, yet affected or plagued by a similar type of issue, that would otherwise end up completely unnoticed if not for your comment.
In short, one man's pedantic can still be other man's sudden epiphany.
This sort of pedantry is exactly why I submitted this library to Show HN in the first place -- I'm not a C programmer by any particular training other than self-taught, and I've never worked at such a low level in industry. Knowing that C programmers consider something like sizeof(char) to be distractingly weird (where I would just consider it consistent/defensive) is good to know!
How often is knowing malloc has failed useful? You going to segfault either way.
I always use sizeof with char just to stay consistent - if you don’t you will get caught out one day when you change something and forget to add in the needed new sizeof.
It's _very_ important to know if it failed if you don't want to introduce undefined behavior. With undefined behavior, _anything_ can happen, (e.g. arbitrary code execution). But let me make it clear: malloc failing doesn't mean that undefined behavior will happen. However, pretending like malloc didn't fail when it really did will most likely get you undefined behavior. You're not going to segfault either way if you handle things correctly. Consider a web server running out of memory to allocate a request structure, do you think it is a good idea to just let it SIGSEV?
> I was under the impression that compiler optimisation often removed any NULL checks anyway
Goodness.
No, you are misunderstanding the example. The optimizer did not introduce a bug by removing a NULL check. The code had a bug because it dereferenced the pointer before checking for NULL.
> thinking you are checking for a null pointer may not always hold especially once inlining happens
Incorrect. The example code did not check for a null pointer. It dereferenced a pointer, which implies that the pointer will never be null, and then it pointlessly checked for null on something that could not have been null. The optimizer in fact made the code more clear and easier to read, while maintaining correctness.
"While this is intentionally a simple and contrived example, this sort of thing happens all the time with inlining: inlining a function often exposes a number of secondary optimization opportunities. This means that if the optimizer decides to inline a function, a variety of local optimizations can kick in, which change the behavior of the code. This is both perfectly valid according to the standard, and important for performance in practice.”
In practice derefencing a NULL pointer on any x86 platform will always cause a crash (defined undefined behaviour I guess), but you can’t assume this on other platforms (ARM seems to be the big one).
I have to say before today I had assumed dereferncing a NULL pointer would always cause a segfault. It is always good to learn something new.
You can't be further from the truth. A vast magntitude of bugs are due to the system running as if nothing has happened operating on garbage data, and giving you garbage results. If your system is robust to garbage data, say neural network training, good luck finding the bug then.
Another is to pre-allocate enough reserve memory to be able to save and exit cleanly. When malloc fails, switch to the reserve memory and either shut down nicely, or post a dire warning to the user that the end is neigh and to "SAVE AND EXIT IMMEDIATELY".
Have you experienced malloc failing a lot? How do you test to ensure error handling code paths work as expected?
Have you read 'notes' section of Linux malloc manpage? It says:
By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. In case it turns out that the system is out of memory, one or more processes will be killed by the OOM killer.
I have never knowingly experienced malloc failing in any of my code which is why I asked if it is something to worry about in practice. Once you run out of memory things start dying or the system locks up hard.
Edit. It is impossible for malloc to fail (at least on linux) [1]. Seems rather pointless checking for something that can’t happen.
That link shows that malloc failed, just not due to exceeding available RAM. At the end it says "malloc finally failed because of address space exhaustion."
On virtual memory systems, it is a safe bet that malloc will "never" fail to return a value. It is just good form to check the return value of system calls. If you program for embedded systems (e.g. Arduino) and bring over the bad habit of never checking malloc, you are going to be in for a surprise. I'm not sure, but I believe on iOS, malloc could fail on the device.
If malloc failure testing is important to you, then you'll need some way to replace your use of the system malloc/free with special versions of your own. (Eg, through function pointers, or LD_PRELOAD.) Your own versions can be crafted to insert faults in just the right spots.
You're not going to segfault either way, especially if you're a library. If you're a library then when the API user creates your context, if possible, you should preallocate all the memory you're going to use, and if that fails, report the error. If preallocation is not possible then, as a library, it is your duty to report memory allocation errors just as you would any other error, ideally as a return code.
No no no. You can't predict where that library is going to be used. Check the return value of malloc, communicate the failures to the caller and let them handle it in an application-appropriate way. They may want to crash, and they may not.
Just in case anyone else finds this fascinating: Correct, sizeof(char) is always 1 byte. A byte contains CHAR_BIT bits (page 37). The implementation-defined value of CHAR_BIT is at least 8 (page 21). There are some implementations [1] that chose other values like 16 or 32. So while sizeof(char) is always 1 byte it's not necessarily always 1 octet.
Everybody, please look at this Readme file. It starts with a section called "What is this thing?"
That is a fantastic example of something everyone should have in their OSS projects. A simple section at the top of the main doc that states what the thing is. Seems obvious, but I only mention it because it's so rarely done.
Beware weaving progress bar code into computationally heavy routines. As soon as you want to start multithreading, it gets tricky. The worst is a ProgressBar singleton in a big C++ GUI program. If I'm going to bother using a library for progress bars, I want it to deal with all those work of combining multiple threads' progress, i.e. it has a queue, knows each thread's estimated share of the total work, and all the threads push progress messages on the queue.
43 comments
[ 3.3 ms ] story [ 95.7 ms ] threadDates back to 2002.
The correct solution would be a) check the length of format, and if it less than or equal to 4 but greater than 0, then copy the bytes, else, don't... I guess? Why is 4 such an arbitrary chosen number?
Or it just be better to allocate the new->length to the return value of strlen(format) + 1.It's common to allocate your size needed plus 1 for the NULL byte. But this programmer only allocates 4 bytes.. with no thought of the NULL byte at the end.
The standard [0] states that the size of `char` will be 1 byte. So there is no point in doing the sizeof operator on char. (e.g: https://github.com/doches/progressbar/blob/master/lib/progre...)
6.5.3.4 - Paragraph 3
> The sizeof operator... (page 80)
[0] http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf
Please be. Seemingly random bits of (honestly - actually very useful) information like yours are why I'm reading HN in the first place instead of, say, random Youtube comments.
Someone may argue about this being overly pedantic, or about such comment being more suitable for sites like Stack Overflow, or about the whole thing being completely clear from the standards documentation itself... etc. But that's really not the point of it when this isn't the kind of thing you would be even intentionally looking at, or for, in the first place.
Every time I open a HN thread, I can be sure that I will find at least a good few nice and memorable comments that seemingly don't add anything revolutionary to the topic at hand, but instead will make you pause, slap your forehead and exclaim "holy moly, that's in essence actually perfectly obvious, yet somehow I never even thought about it from this point of view before!" Which in turn suddenly makes you realize something completely unrelated in your own world, yet affected or plagued by a similar type of issue, that would otherwise end up completely unnoticed if not for your comment.
In short, one man's pedantic can still be other man's sudden epiphany.
I always use sizeof with char just to stay consistent - if you don’t you will get caught out one day when you change something and forget to add in the needed new sizeof.
I was under the impression that compiler optimisation often removed any NULL checks anyway [1].
1. http://blog.llvm.org/2011/05/what-every-c-programmer-should-...
Goodness.
No, you are misunderstanding the example. The optimizer did not introduce a bug by removing a NULL check. The code had a bug because it dereferenced the pointer before checking for NULL.
Are there any modern platforms where dereferncing a NULL pointer will not cause a crash?
Edit. Answering my own question it looks like ARM is one [1].
1. https://www.securecoding.cert.org/confluence/display/c/EXP34...
Incorrect. The example code did not check for a null pointer. It dereferenced a pointer, which implies that the pointer will never be null, and then it pointlessly checked for null on something that could not have been null. The optimizer in fact made the code more clear and easier to read, while maintaining correctness.
"While this is intentionally a simple and contrived example, this sort of thing happens all the time with inlining: inlining a function often exposes a number of secondary optimization opportunities. This means that if the optimizer decides to inline a function, a variety of local optimizations can kick in, which change the behavior of the code. This is both perfectly valid according to the standard, and important for performance in practice.”
In practice derefencing a NULL pointer on any x86 platform will always cause a crash (defined undefined behaviour I guess), but you can’t assume this on other platforms (ARM seems to be the big one).
I have to say before today I had assumed dereferncing a NULL pointer would always cause a segfault. It is always good to learn something new.
e.g. https://github.com/git/git/blob/master/wrapper.c#L63
Have you experienced malloc failing a lot? How do you test to ensure error handling code paths work as expected?
Have you read 'notes' section of Linux malloc manpage? It says:
By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. In case it turns out that the system is out of memory, one or more processes will be killed by the OOM killer.
Edit. It is impossible for malloc to fail (at least on linux) [1]. Seems rather pointless checking for something that can’t happen.
1. https://scvalex.net/posts/6/
Actually the real issue here is that you should be aware that on some systems malloc might fail and not segfault.
It's tedious work.
[1] http://bytes.com/topic/c/answers/223565-implementations-char...
That is a fantastic example of something everyone should have in their OSS projects. A simple section at the top of the main doc that states what the thing is. Seems obvious, but I only mention it because it's so rarely done.