>I fell in love with C, but strings stand between us. So I made this lib.
You and me both!
You'd like Pascal's strings, most notably those of Delphi/FreePascal -- the standard string type in those languages contains the length of the string before the pointer to the string (out-of-band signalling as opposed to in-band signalling as C's standard strings are, with a zero marking their end) -- much like your string implementation does.
Also, Delphi/FreePascal's strings -- contain a reference count, which is automatically updated when a whole string is copied from one string variable to another, and not freed until the reference count goes to zero.
The result is that Delphi/FreePascal's string copies, e.g.,
StringA := StringB;
Are lightning fast -- because only the pointer is being copied! (well, that and the ref count increased by one...)
If code later changes any value of StringB for any reason, then and only then is StringB copied and the value changed!
But anyway, getting back to strings in C, yes, I agree completely, there needs to be a better standard string type, so your idea is conceptually brilliant!
Now, as far as your code goes, I don't do that much C, so I can't really comment.
On the plus side, I've seen libraries in C that used far more code -- to accomplish far less!
On the "I'm not too sure about this" side (but then again, I don't do that much C, and other people can and will have other opinions) -- I'm not too sure about the use of macros which use other macros...
In other words, while this practice is certainly convenient, it can (and frequently does) lead to newer programmers not really understanding what the code does in those places...
Now, that's your call.
All I know is that if I was writing this for the maximum understanding by the programmer community, including new programmers, then my "rule of thumb" for that purpose would be "less macros" and "no nested macros"...
Things might be a little bit more difficult that way, but conceptually, what you're doing, at the highest level is simply this:
1) You're taking a series of bytes in memory.
2) You're wrapping this series of bytes in memory with various string handling functions, similar to, but not exactly the same as (because length is signaled before the string, etc.) the standard C ones, including functions to allocate/reallocate(resize)/copy and free that memory.
3) Net result of these modifications should be that C strings handled with these wrapper methods are more robust than regular C strings.
That, at its highest conceptual level, is all that's going on (or did I miss something?)
I know I could do all of that in one relatively terse source code file without most of the macros, and definitely without nested macros...
Of course, those are just my personal tastes as a programmer; but (as Harry and Paul so eloquently said!) "I am not the market" of all computer programmers, so it's quite possible that your coding style would both surprise and delight and make happy legions of other programmers who prefer that type of coding style!
In other words, take all that I say with the proverbial "grain of salt".
But, better C strings are sorely needed -- so a brilliant choice of area to work on!
> reference count ... copies ... only then is StringB copied
You're right I should - and will - use this. But note it won't cover cases where the user modifies/copies the string from outside the API. The lib won't be aware.
> your idea is conceptually brilliant!
Oh I wish the neg offset header was my invention ! Alas it was at least used in Microsoft' BSTR and later by the famous antirez/sds (https://github.com/antirez/sds)
> no nested macros
I hear you. I thought it made them more readable.
> is all that's going on
I also (maybe naively) pretend to make mem faults occur less/at all.
>> reference count ... copies ... only then is StringB copied
>You're right I should - and will - use this. But note it won't cover cases where the user modifies/copies the string from outside the API. The lib won't be aware.
But that's perfectly OK! Just explain any and all caveats/gotchas/potential issues -- to the end-user in the docs, and they'll understand and work around them! Most programmers are smart people, and they'll be totally OK with it if you explain those things to them in the docs!
>> no nested macros
>I hear you. I thought it made them more readable.
The macros DO make the code more readable(!) -- at least the non-nested ones!
The nested ones are kind of OK, and yes, they kind of make the code more readable -- but in exchange for this readability, you're getting farther and farther away from the underlying generated C code!
That's OK to a point -- but I've seen codebases where they've went N levels (not 1, not 2, but multiple, multiple) levels deep on the macros -- and what was gained in early macro readability -- gets lost in massively recursive macro hierarchies!
For your future sanity, try to avoid going in that direction!
Like, maybe set a limit of no more than two, two being the absolute maximum, and only if you really really need to do it -- levels of macros...
Because the temptation is going to be (at least in the future!) -- to use even more levels of macros!
That'll work great for a while (it's a double-edged sword!)... but at some point, a deeply nested structure can really, and I mean really bite you in the butt -- with hard-to-find bugs and anomalies, when you least expect it, and when you most need it to work!
But again, all of this being said, "I am not the market".
You might find that 1,000,000 plus C programmers -- absolutely love what you've done with the nested macros, and if so, I say "go with it!" (at least, if appealing to a broad base is your goal, and the broad base wants things done that way!)
But, all of this being said, you choose an excellent problem to solve, and you put in an excellent effort, and I applaud your work!
Hey, nice project, overall looks pretty useful.
I have a question, and also some feedback (from the point of view of a mostly C++ programmer, who has some knowledge of C by osmosis).
Why did you choose to use a char* typedef with a magic "header before" in the alloc, instead of just padding around the header struct? You get some marginal convenience when passing it into an api that expects a char, but it doesn't seem worth it when you consider that you won't get type warnings if you try to use a char where you expected an stx_t. (My understanding is that unlike C++, C compilers will generally allow wrong-pointer-types, but you can turn on warnings for it).
I think the stx_load function is a bit out of place, this doesn't seem like it should be a string library's responsibility. There is also the added issue that passing a char* in as a filepath is not suitable for windows. You (generally, there are other ways) need to use a wide char string on windows for file I/O, to handle non-ascii characters.
I'm also not sold on the usefulness of having an append function that can't reallocate. I have never wanted this, and if I did, I could emulate it with a reallocating version by just manually truncating the copy to the current capacity.
As for licensing commercially, I don't think that's likely to work out really. It is a useful project for sure, but I think it's just a bit too small to justify buying it. Most C programmers would probably prefer to write their own instead.
> marginal convenience(...) you won't get type warnings
The aliasing of char* permits the user to directly use all his string-read tools. I'm not super convinced myself it's the right design choice though but it sure is convenient.
The alternative would force him using Header->str every time he wants access to the real string.
Yes, there is no real type safety. The user has to be conscious. But he'll be warned by stx_t being const if he ever uses out-of-API writers.
> stx_load function is a bit out of place
I agree. It's a bit hard to know where to stop...
> not sold on the usefulness of having an append function that can't reallocate
For constrained/embedded environments ? Or just a socket buffer.
> I could emulate it with a reallocating version by just manually truncating the copy to the current capacity.
That would be much slower I think.
> Most C programmers would probably prefer to write their own instead
Capacity is required only if you plan to grow a string in place (so you know how much you can add before reallocating).
Reallocating is possible here because the string is one object, including the header. Reallocating would change the identity of the object, which is a non-starter when multiple places in the program are sharing the pointer via reference counting.
Therefore, all the strings are immutable here, and so all operations return a new string object.
There is a key difference in the 'long way' and 'stricky way' example in the readme - the 'long way' doesn't do any allocations and thus has a performance advantage out of the box. Also, where in the 'stricky way' does page get freed? This also needs to be shown in order to be equivalent to the 'long way'.
If you look later on at the readme, you'll see that the stx_catf function does not reallocate, it truncates, there is another function for reallocating append.
That said, I don't really see why one would need such a no-realloc function. In c++, I would just use std::string::reserve(some_big_num).
14 comments
[ 2.8 ms ] story [ 39.2 ms ] threadI fell in love with C, but strings stand between us. So I made this lib.
It's certainly more experimental than 'industrial strength', but I learned a great deal and intend to make it a solid useful tool.
In the mean time, quite thorough unit tests ensure we don't go astray.
Also, I tried to produce the cleanest code I could, to please the reader.
A few questions :
0- Is my header layout foolproof and portable ?
1- Would it be a sound idea to return a static zero-valued item than NULL ?
2- Once completely tooled and tested, could I pretend to double-license the lib with a commercial part ?
You and me both!
You'd like Pascal's strings, most notably those of Delphi/FreePascal -- the standard string type in those languages contains the length of the string before the pointer to the string (out-of-band signalling as opposed to in-band signalling as C's standard strings are, with a zero marking their end) -- much like your string implementation does.
Also, Delphi/FreePascal's strings -- contain a reference count, which is automatically updated when a whole string is copied from one string variable to another, and not freed until the reference count goes to zero.
The result is that Delphi/FreePascal's string copies, e.g.,
StringA := StringB;
Are lightning fast -- because only the pointer is being copied! (well, that and the ref count increased by one...)
If code later changes any value of StringB for any reason, then and only then is StringB copied and the value changed!
But anyway, getting back to strings in C, yes, I agree completely, there needs to be a better standard string type, so your idea is conceptually brilliant!
Now, as far as your code goes, I don't do that much C, so I can't really comment.
On the plus side, I've seen libraries in C that used far more code -- to accomplish far less!
On the "I'm not too sure about this" side (but then again, I don't do that much C, and other people can and will have other opinions) -- I'm not too sure about the use of macros which use other macros...
In other words, while this practice is certainly convenient, it can (and frequently does) lead to newer programmers not really understanding what the code does in those places...
Now, that's your call.
All I know is that if I was writing this for the maximum understanding by the programmer community, including new programmers, then my "rule of thumb" for that purpose would be "less macros" and "no nested macros"...
Things might be a little bit more difficult that way, but conceptually, what you're doing, at the highest level is simply this:
1) You're taking a series of bytes in memory.
2) You're wrapping this series of bytes in memory with various string handling functions, similar to, but not exactly the same as (because length is signaled before the string, etc.) the standard C ones, including functions to allocate/reallocate(resize)/copy and free that memory.
3) Net result of these modifications should be that C strings handled with these wrapper methods are more robust than regular C strings.
That, at its highest conceptual level, is all that's going on (or did I miss something?)
I know I could do all of that in one relatively terse source code file without most of the macros, and definitely without nested macros...
Of course, those are just my personal tastes as a programmer; but (as Harry and Paul so eloquently said!) "I am not the market" of all computer programmers, so it's quite possible that your coding style would both surprise and delight and make happy legions of other programmers who prefer that type of coding style!
In other words, take all that I say with the proverbial "grain of salt".
But, better C strings are sorely needed -- so a brilliant choice of area to work on!
Neat analogy !
> reference count ... copies ... only then is StringB copied
You're right I should - and will - use this. But note it won't cover cases where the user modifies/copies the string from outside the API. The lib won't be aware.
> your idea is conceptually brilliant!
Oh I wish the neg offset header was my invention ! Alas it was at least used in Microsoft' BSTR and later by the famous antirez/sds (https://github.com/antirez/sds)
> no nested macros
I hear you. I thought it made them more readable.
> is all that's going on
I also (maybe naively) pretend to make mem faults occur less/at all.
>You're right I should - and will - use this. But note it won't cover cases where the user modifies/copies the string from outside the API. The lib won't be aware.
But that's perfectly OK! Just explain any and all caveats/gotchas/potential issues -- to the end-user in the docs, and they'll understand and work around them! Most programmers are smart people, and they'll be totally OK with it if you explain those things to them in the docs!
>> no nested macros
>I hear you. I thought it made them more readable.
The macros DO make the code more readable(!) -- at least the non-nested ones!
The nested ones are kind of OK, and yes, they kind of make the code more readable -- but in exchange for this readability, you're getting farther and farther away from the underlying generated C code!
That's OK to a point -- but I've seen codebases where they've went N levels (not 1, not 2, but multiple, multiple) levels deep on the macros -- and what was gained in early macro readability -- gets lost in massively recursive macro hierarchies!
For your future sanity, try to avoid going in that direction!
Like, maybe set a limit of no more than two, two being the absolute maximum, and only if you really really need to do it -- levels of macros...
Because the temptation is going to be (at least in the future!) -- to use even more levels of macros!
That'll work great for a while (it's a double-edged sword!)... but at some point, a deeply nested structure can really, and I mean really bite you in the butt -- with hard-to-find bugs and anomalies, when you least expect it, and when you most need it to work!
But again, all of this being said, "I am not the market".
You might find that 1,000,000 plus C programmers -- absolutely love what you've done with the nested macros, and if so, I say "go with it!" (at least, if appealing to a broad base is your goal, and the broad base wants things done that way!)
But, all of this being said, you choose an excellent problem to solve, and you put in an excellent effort, and I applaud your work!
Why did you choose to use a char* typedef with a magic "header before" in the alloc, instead of just padding around the header struct? You get some marginal convenience when passing it into an api that expects a char, but it doesn't seem worth it when you consider that you won't get type warnings if you try to use a char where you expected an stx_t. (My understanding is that unlike C++, C compilers will generally allow wrong-pointer-types, but you can turn on warnings for it).
I think the stx_load function is a bit out of place, this doesn't seem like it should be a string library's responsibility. There is also the added issue that passing a char* in as a filepath is not suitable for windows. You (generally, there are other ways) need to use a wide char string on windows for file I/O, to handle non-ascii characters.
I'm also not sold on the usefulness of having an append function that can't reallocate. I have never wanted this, and if I did, I could emulate it with a reallocating version by just manually truncating the copy to the current capacity.
As for licensing commercially, I don't think that's likely to work out really. It is a useful project for sure, but I think it's just a bit too small to justify buying it. Most C programmers would probably prefer to write their own instead.
> marginal convenience(...) you won't get type warnings
The aliasing of char* permits the user to directly use all his string-read tools. I'm not super convinced myself it's the right design choice though but it sure is convenient.
The alternative would force him using Header->str every time he wants access to the real string.
Yes, there is no real type safety. The user has to be conscious. But he'll be warned by stx_t being const if he ever uses out-of-API writers.
> stx_load function is a bit out of place
I agree. It's a bit hard to know where to stop...
> not sold on the usefulness of having an append function that can't reallocate
For constrained/embedded environments ? Or just a socket buffer.
> I could emulate it with a reallocating version by just manually truncating the copy to the current capacity.
That would be much slower I think.
> Most C programmers would probably prefer to write their own instead
That's not good news haha
Reallocating is possible here because the string is one object, including the header. Reallocating would change the identity of the object, which is a non-starter when multiple places in the program are sharing the pointer via reference counting.
Therefore, all the strings are immutable here, and so all operations return a new string object.
That said, I don't really see why one would need such a no-realloc function. In c++, I would just use std::string::reserve(some_big_num).
My point is that the readme example is misleading, as the two code samples are not equivalent.
This 'long way' thing is definitly not super useful..