That also works, but strtok is of so limited use that it's probably better to roll your own anyway. For example you can't handle arbitrary whitespace with strtok.
with all manual pages installed[1] (POSIX, Linux, BSD), you get a pretty good idea as to how to use it properly, or reading the SEI C Coding Standard helps. OpenBSD deprecated it in favor of strsep. It is not painful to use if you know (read all the manual pages at the very least).
Been happily using it for a long time (unless other functions are not more ideal for the specific use-case).
There are some things to pay attention to, but the manual pages pretty much tell you everything you need to know.
[1] https://archlinux.org/packages/core/any/man-pages/ on Arch Linux, perhaps it is "man-pages" for other distros, too. As per Arch wiki: "man-pages provides both the Linux and the POSIX.1 man pages".
You may also do:
curl -sL 'https://man.archlinux.org/man/strtok.raw' | man -l -
or
curl -sL 'https://man.archlinux.org/man/getaddrinfo_a.raw' | man -l -
for "strtok", although not sure it provides ALL manual pages.
If your use case is covered by strtok(), why not? Like all functions, you have to use it properly, and the use case for strtok() is narrow, but sometimes, it is just what you need.
I rarely use it myself, but emulating it is silly. Also strstr() is probably not the most appropriate, maybe you mean strpbrk()?
As far as I know, the only function from the libc that you should never use is gets(). It is even written in the manual: "Never use this function". As expected, it is removed from recent standards.
The worst problem with strtok() isn't that it modifies its input; it's that it's stateful. It retains a pointer to the input string internally, and returns subsequent words from it when called with an input of NULL. strsep() fixes this by using a char * argument instead.
I'm amazed it has taken so long to get rid of strtok(). This is one of the worst designed and most misused functions I have ever seen - just about every use of it I have ever come across has been wrong. Write your own mini-parser rather than use it.
I'm amazed people who are skilled in C find it difficult to use, considering everything you have to know about it is available in man-pages (and pretty much everywhere online).
If it is misused, that is a skill issue (yeah, sorry). I could misuse virtually anything in other languages, too...
So, all in all, I don't understand what you are trying to say. There are cases where I would rather use strtok than to roll my own. Sometimes all you need is already available, e.g. strtok, or strsep, or strstr, or what have you.
Consider that I misuse this and that in your favourite language. Is it a reflection of the programming language, or my limited "skills"?
> If it is misused, that is a skill issue (yeah, sorry).
No it isn't. It's very easy to misuse so even skilled people misuse it.
> Consider that I misuse this and that in your favourite language. Is it a reflection of the programming language, or my limited "skills"?
Depends on what this and that are. If they are things that are easy to misuse and many people do, then it's a reflection of the programming language. If they're well-designed and it's only you getting it wrong, then it's your limited skills.
If what you need is strtok, use it as documented. If you do not use it as documented, that is a skill issue. Do you even need strtok? If you do, RTFM.
> If they are things that are easy to misuse and many people do, then it's a reflection of the programming language. If they're well-designed and it's only you getting it wrong, then it's your limited skills.
Whatever "well-designed" means. At any rate, wonder where you got the "many people" from, who actually have read the manual.
For some of my use-cases, strtok is designed well enough. If I need something else, then I will use something else, and so should you. No need to re-implement strtok (or any other functions) readily available.
Plus, do you really think people who cannot use strtok properly can roll their own?
So you are resorting to ad hominems instead of answering me.
I get it, strtok is difficult for you to use even after a hundreds of correct examples of its usage and having read the manual pages. Then do not use it. Stick to whatever you may for all I care.
I just don't know of a good way to help you understand the insane levels of hubris you have. Tell me this. Do you ever make mistakes? Do you always read the manual exhaustively and remember it perfectly?
When I'm coding, I have at least 2 workspaces out of 10 dedicated to manual pages and documentation, so I don't need to always remember (sometimes incorrectly). I do make mistakes, but usage of strtok is not one of them.
So you do make mistakes. And despite having the documentation open you don't always read all of it, correct? Or are you trying to say you always read all of the documentation?
You see where I'm going with this right? Either you never make mistakes and always read all the documentation (which is impossible), OR it's beneficial to avoid functions where mistakes and missed documentation are highly likely to result in misuse.
That's just basic logic. Hopefully there are no skill issues understanding that?
I do read most documentations of most functions that I am not entirely sure how to use.
I am not going to reimplement strtok if it already exist and is the right tool for the job AND is perfectly documented.
> OR it's beneficial to avoid functions where mistakes and missed documentation are highly likely to result in misuse.
That is where you got it wrong. If you do not know how to use strtok, check the documentation (which exists) or examples of proper usage (also exist), or do not use it. How can you even implement strtok if you know jack shit about what it does, anyway? Don't you need the documentation for that to begin with? If not, then you are reimplementing something that already exists. I'm not going to reimplement strlcpy. Sometimes I copy the implementation of asprintf into the codebase because I may want to avoid GNU extensions, that's about it.
I get it, you (and many people who refuse to read the manual pages) can't use strtok properly, that is fine, don't use it then.
The basic logic is: don't use functions you are not sure how to use, look up the readily available documentation. Hell, ask an LLM. It applies to the whole field. FWIW, you do know you can read manual pages from the IDE these days, too, right?
I'm sorry, but you sound just like my friend who wanted to learn how to use Linux without actually wanting to read Wiki pages about it (installation guide, for one). "Make Linux easier to use [because I cba to read]", well, install an Ubuntu then, but you will still not be a power user or learn anything. In this case it does not matter then if you use Windows or Linux if you are just sticking to using a browser.
I don't know where or how you work, but why would anyone spend time writing documentation if you are not even willing to read it through? At least not everyone avoids reading like a plague.
BTW, what issues do you have with strtok() exactly? It is straightforward to use after having read the documentation. I do not buy the "many people misuse" argument. Who? Wannabe script kiddies? People who are reluctant to read the documentation? From where do you get "many people"?
I don't know why you are going on about "skills" - I never mentioned such things in my comment - something you have introduced, I think. I do agree though that however "skilled" you are strtok() (and other C strXXX functions) is difficult to use correctly.
Because of another commenter who did, BUT I do not think that this is true, not after having read the manual pages. Do you think people who misuse strtok can implement their own alternative? The Internet is full of proper uses of strtok, and I bet LLMs know it too, and would help you through the process if you so want. This applies to most functions e.g. strcpy vs. strncpy vs. strlcpy, just to name one.
Honestly, if someone cannot use strtok (if they should really use it) after having read all the manual pages there is, I doubt they would be able to come up with their own, better version of it.
> While the old API expected that the shared object is not used directly, the API made it possible which led to undefined behaviour, typically producing a data race.
- C++ is insecure, don't use it
[C++ fixes a hole]
- C++ always deprecates the easy APIs
[Stanley face]
Luckily most C++ developers don't immediately jump on and begin using backwards incompatible features. The C++ community is still only recently generally requiring C++11. And that's great. Introduce new features but don't immediately use them in practice. Some other languages and their communities could really learn from this.
> The C++ community is still only recently generally requiring C++11.
this doesn't match what I see. The Jetbrains 2023 survey reported that 80% of C++ in use was >= C++17. Qt, one of the most common C++ UI framework, has been requiring C++17 for five years and support C++20 types in its API.
Ah, Jetbrains must be touching a different part of the elephant. My related experience is with compiling C++ programs meant for human people to run on their own home computers. I imagine internal company dev projects like Jetbrains would cover don't care nearly so much about compatibility.
Also I note that 25%+ of the Jetbrains 2023 survey use C++11 or older. And 50% of Jetbrains survey devs say "No, I don't plan to move to another C++ standard" from C++11. So it's likely that 25% will stay high.
C++ has many ugly parts to it, but I wouldn't call sticking with a 14 year old language "great". There's plenty of things that are really damn good, for example
- std::span
- std::string_view
- std::source_location
- if constexpr
...
I feel exactly the opposite. If a language adds a new feature that's useful to me, and it helps me write better code, I'm going to use it. Perhaps not on the first day it comes out, but definitely the next time I need its functionality. And why not? That's why the feature was added in the first place.
There's the chance that its newness means it's not thoroughly tested yet. In any language with a reasonable development process, I'd open a bug report and then figure out a workaround until it's fixed. But I'd expect the interval between "feature is released" and "feature works as documented, modulo the usual long tail of edge cases" to be on the order of weeks or months, not decades.
Like, I can't imagine only now upgrading to Python 2.7 (which was current in 2011). I wouldn't expect every shop to be on 3.13 already, but neither would I scowl at a dev who wanted to use something new from a 2-year-old version.
I agree with you from the developer point of view. But you should consider the point of view that other people might want to compile your software.
Python is a great example of a software community that has enormous trouble due to always changing the languae and using new features. It's pretty much infeasible to run a python program on your system python these days. Every single python program requires it's entire own custom python in a container.
I'm only sympathetic to a point. If I distributed C++ code to paying customers who wanted to compile it locally, I'd surely take their needs into account. For anything I was releasing as FOSS, deal with it. For instance, my nearby server is running Debian 12/Bookworm with GCC 12. According to https://gcc.gnu.org/projects/cxx-status.html, that supports nearly all of C++20. If you can't be bothered to upgrade to your system every few years, I'm not going to make more than 0 effort to cater to your ancient compilers.
In Python terms, today I wouldn't break a sweat to manage unsupported Python versions (https://devguide.python.org/versions/). If my stuff runs on an old Python 3.8 system, right on! I wouldn't deliberately break it or anything. But if something came along in 3.9 that makes my life more pleasant, I wouldn't hesitate to use it.
I also vehemently disagree with your Python characterization. There are any number of common, well-supported ways to easily install arbitrary versions on your system, including pyenv and uv. I have multiple versions installed on the laptop I'm typing this on. I do zero development inside containers: all my work runs directly on the host. And yet, none of them conflict with each other at all.
The idea that you have to have multiple pythons is weird, new, and despite what you say about not breaking a sweat, vastly more complex and time consuming than how python used to work before it became popular. It's broken now and people have grown used to the brokenness, even integrated it into their workflow and come to believe that this is good. To me it sounds like spacebar heating: https://xkcd.com/1172/
> System python shouldn't be a thing in the first place.
Sour gapes. The idea that "system python shouldn't be a thing" is really radical and massive change from the status quo of languages. Especially interpreted languages.
There are plenty of languages that work just fine on system interpreter/compiler with no need of an entire language for each program. Perl is one example that is very comparable to python. The important difference is that Perl hasn't been split into hundreds of different languages like python due to popularity.
> Every platform provides a very easy way to install the latest version of clang.
Once again sour grapes. Abandoning the idea of your system compiler working for C++ is wild.
> Once again sour grapes. Abandoning the idea of your system compiler working for C++ is wild.
why? most operating systems (Windows, macOS, Android, iOS etc) don't even have a system compiler, it's a separate app that you install (and for which you can thankfully install multiple versions in parallel, unlike the stupid linux idea that every app should only ever have one version installed at any given time).
In general C++17 is the common standard most C++ projects refer to.
Additionally between C++11 and C++14, it is a mistake to use C++11 instead of C++14, as the later is basically the set of features that missed C++11, especially relevant in smart pointers machinery.
52 comments
[ 3.0 ms ] story [ 115 ms ] threadBeen happily using it for a long time (unless other functions are not more ideal for the specific use-case).
There are some things to pay attention to, but the manual pages pretty much tell you everything you need to know.
[1] https://archlinux.org/packages/core/any/man-pages/ on Arch Linux, perhaps it is "man-pages" for other distros, too. As per Arch wiki: "man-pages provides both the Linux and the POSIX.1 man pages".
You may also do:
or for "strtok", although not sure it provides ALL manual pages.I rarely use it myself, but emulating it is silly. Also strstr() is probably not the most appropriate, maybe you mean strpbrk()?
As far as I know, the only function from the libc that you should never use is gets(). It is even written in the manual: "Never use this function". As expected, it is removed from recent standards.
[1] https://man.openbsd.org/strsep.3
If it is misused, that is a skill issue (yeah, sorry). I could misuse virtually anything in other languages, too...
So, all in all, I don't understand what you are trying to say. There are cases where I would rather use strtok than to roll my own. Sometimes all you need is already available, e.g. strtok, or strsep, or strstr, or what have you.
Consider that I misuse this and that in your favourite language. Is it a reflection of the programming language, or my limited "skills"?
https://wiki.sei.cmu.edu/confluence/display/c/STR06-C.+Do+no... is worth a read, too.
No it isn't. It's very easy to misuse so even skilled people misuse it.
> Consider that I misuse this and that in your favourite language. Is it a reflection of the programming language, or my limited "skills"?
Depends on what this and that are. If they are things that are easy to misuse and many people do, then it's a reflection of the programming language. If they're well-designed and it's only you getting it wrong, then it's your limited skills.
> If they are things that are easy to misuse and many people do, then it's a reflection of the programming language. If they're well-designed and it's only you getting it wrong, then it's your limited skills.
Whatever "well-designed" means. At any rate, wonder where you got the "many people" from, who actually have read the manual.
For some of my use-cases, strtok is designed well enough. If I need something else, then I will use something else, and so should you. No need to re-implement strtok (or any other functions) readily available.
Plus, do you really think people who cannot use strtok properly can roll their own?
I get it, strtok is difficult for you to use even after a hundreds of correct examples of its usage and having read the manual pages. Then do not use it. Stick to whatever you may for all I care.
sighs
You see where I'm going with this right? Either you never make mistakes and always read all the documentation (which is impossible), OR it's beneficial to avoid functions where mistakes and missed documentation are highly likely to result in misuse.
That's just basic logic. Hopefully there are no skill issues understanding that?
I am not going to reimplement strtok if it already exist and is the right tool for the job AND is perfectly documented.
> OR it's beneficial to avoid functions where mistakes and missed documentation are highly likely to result in misuse.
That is where you got it wrong. If you do not know how to use strtok, check the documentation (which exists) or examples of proper usage (also exist), or do not use it. How can you even implement strtok if you know jack shit about what it does, anyway? Don't you need the documentation for that to begin with? If not, then you are reimplementing something that already exists. I'm not going to reimplement strlcpy. Sometimes I copy the implementation of asprintf into the codebase because I may want to avoid GNU extensions, that's about it.
I get it, you (and many people who refuse to read the manual pages) can't use strtok properly, that is fine, don't use it then.
The basic logic is: don't use functions you are not sure how to use, look up the readily available documentation. Hell, ask an LLM. It applies to the whole field. FWIW, you do know you can read manual pages from the IDE these days, too, right?
I'm sorry, but you sound just like my friend who wanted to learn how to use Linux without actually wanting to read Wiki pages about it (installation guide, for one). "Make Linux easier to use [because I cba to read]", well, install an Ubuntu then, but you will still not be a power user or learn anything. In this case it does not matter then if you use Windows or Linux if you are just sticking to using a browser.
I don't know where or how you work, but why would anyone spend time writing documentation if you are not even willing to read it through? At least not everyone avoids reading like a plague.
Honestly, if someone cannot use strtok (if they should really use it) after having read all the manual pages there is, I doubt they would be able to come up with their own, better version of it.
That tracks with my experiences with C++.
A lot of C (not so much C++) programmers thought it was easy to use correctly. Mostly, they were wrong.
Is this humor?
nit: the contrast between the font and the background is too low
this doesn't match what I see. The Jetbrains 2023 survey reported that 80% of C++ in use was >= C++17. Qt, one of the most common C++ UI framework, has been requiring C++17 for five years and support C++20 types in its API.
Also I note that 25%+ of the Jetbrains 2023 survey use C++11 or older. And 50% of Jetbrains survey devs say "No, I don't plan to move to another C++ standard" from C++11. So it's likely that 25% will stay high.
There's the chance that its newness means it's not thoroughly tested yet. In any language with a reasonable development process, I'd open a bug report and then figure out a workaround until it's fixed. But I'd expect the interval between "feature is released" and "feature works as documented, modulo the usual long tail of edge cases" to be on the order of weeks or months, not decades.
Like, I can't imagine only now upgrading to Python 2.7 (which was current in 2011). I wouldn't expect every shop to be on 3.13 already, but neither would I scowl at a dev who wanted to use something new from a 2-year-old version.
Python is a great example of a software community that has enormous trouble due to always changing the languae and using new features. It's pretty much infeasible to run a python program on your system python these days. Every single python program requires it's entire own custom python in a container.
In Python terms, today I wouldn't break a sweat to manage unsupported Python versions (https://devguide.python.org/versions/). If my stuff runs on an old Python 3.8 system, right on! I wouldn't deliberately break it or anything. But if something came along in 3.9 that makes my life more pleasant, I wouldn't hesitate to use it.
I also vehemently disagree with your Python characterization. There are any number of common, well-supported ways to easily install arbitrary versions on your system, including pyenv and uv. I have multiple versions installed on the laptop I'm typing this on. I do zero development inside containers: all my work runs directly on the host. And yet, none of them conflict with each other at all.
Every platform provides a very easy way to install the latest version of clang.
> It's pretty much infeasible to run a python program on your system python these days.
System python shouldn't be a thing in the first place. If you want to run some python program in 2025 you just
Sour gapes. The idea that "system python shouldn't be a thing" is really radical and massive change from the status quo of languages. Especially interpreted languages.
There are plenty of languages that work just fine on system interpreter/compiler with no need of an entire language for each program. Perl is one example that is very comparable to python. The important difference is that Perl hasn't been split into hundreds of different languages like python due to popularity.
> Every platform provides a very easy way to install the latest version of clang.
Once again sour grapes. Abandoning the idea of your system compiler working for C++ is wild.
why? most operating systems (Windows, macOS, Android, iOS etc) don't even have a system compiler, it's a separate app that you install (and for which you can thankfully install multiple versions in parallel, unlike the stupid linux idea that every app should only ever have one version installed at any given time).
Additionally between C++11 and C++14, it is a mistake to use C++11 instead of C++14, as the later is basically the set of features that missed C++11, especially relevant in smart pointers machinery.