The trouble I have with this approach (which, conceptually, I agree with) is that it's damned hard to do anything with the parse results. Want to print that email_t? Then you're right back to char*, unless you somehow write your own I/O system that knows about your opaque conventions.
So you say, okay, I'll make an `email_to_string` function. Does it return a copy or a reference? Who frees it? etc, etc, and you're back to square one again. The idea is to keep char* and friends at "the edge", but I've never found a way to really achieve that.
Could just be my limitations as a C programmer, in which case I'd be thrilled to learn better.
`_t` should not be used for custom types since it's reserved for future standard types (and/or types declared in a header you might include someday). This does cause real-world problems (`key_t` anyone?).
Gratuitous allocations are gratuitous.
The whole "prevent double free" claim is completely bogus. Setting a variable to `NULL` only works for cases where there is one, obvious, owner, which is not the circumstance under which double free is prone to happening in the first place. Actually preventing double free requires determining ownership of every object, and C sucks at that.
> The whole "prevent double free" claim is completely bogus.
The way I interpreted the author's intent was that, the logic of error handling (something C sucks even more at) can be greatly simplified if your cleanup routine can freely be called multiple times. At the moment an error happens you no longer have to keep track of where you are in the lifecycle of each local variable, you can just call cleanup() on everything. I actually like the idea from that standpoint.
> Here, I’ll build on that by showing how this technique can be used outside of niche academic languages by demonstrating it in a language that is as practical as it is dangerous - C.
The "practical" part really bugged me because the entire post is trying to explain exactly why it is not.
The only way to make C reasonably safe is to encode information via newtype pattern. Wrap `char *` inside a struct that have proper names and include the size in there as well.
Basically, there should be ZERO pointers except at creation and consumption by outside libraries (open, write, etc)
I've been shouting this from "the rooftops" ever since I eventually independently had come to the same conclusion through sufficient volume of experience and empiry. I consider it a small tragedy, however, that my words tend to fall on deaf ears as I try to explain to e.g. co-workers why they should, in fact, parse their input and use the language's type system to their advantage, vs. all the `validateThis` and `validateThat` they seem to be copying from e.g. Stack Overflow. I see them stumbling and breaking their nose time and again passing URIs around as strings -- even in languages other than C (so this isn't obviously a problem C created and needs to solve alone) -- where it's never enforced that the URI is e.g. "absolute" or "relative" (nevermind the fact these people often don't seem to care for the difference). They do the same error with date and time (how many decades of advice do we need to accumulate and impart on our younger peers for them to irrevocably understand why e.g. date and time should never be passed around as numbers or strings, with very clearly documented and justified exceptions?). But the tragedy is never more evident when the person looks at you like you're suddenly talking Klingon to them and how bored they are with the notion and how it cannot be so important as to disturb their very important "coding project" (we don't write Knuth's "literal" programs any more -- we _code_).
Such a recognisable name, greetz after many years :D You might remember me from uni as lycium, a few lifetimes ago. Going to check the rest of your blog now...
> Such a recognisable name, greetz after many years :D You might remember me from uni as lycium, a few lifetimes ago. Going to check the rest of your blog now...
I do remember you? You had/joined a sartup in NZ, correct? Warm regards Lycium :-)
One flaw I've seen in "Parse, Don't Validate" as it pertains to real codebases is that you end up with a combinatorial prolieration of types.
E.g., requiring that a string be base64, have a certain fixed length, and be provided by the user.
E.g., requiring that a file have the correct MIME type, not be too large, and contain no EXIF metadata.
If you really always need all n of those things then life isn't terrible (you can parse your data into some type representing the composition of all of them), but you often only need 1, 2, or 3 and simultaneously don't want to duplicate too much code or runtime work, leading to a combinatorial explosion of intermediate types and parsing code.
As one possible solution, I put together a POC in Zig [0] with one idea, where you abuse comptime to add arbitrary tagging to types, treating a type as valid if it has the subset of tags you care about. I'm very curious what other people do to appropriately model that sort of thing though.
You might be interested in Lean's way of doing things. They have normal types (e.g. numeric types) and subtypes (e.g. numbers less than zero). An element of the subtype "numbers less than zero" can be understood as a tuple containing the actual number (which has a normal numeric type) and a proof that this specific number is indeed less than zero.
Structs are representations of combinatorial types! In your file case, you could parse the input into a struct, and then accept or reject further processing based on that contents of that struct.
Of course, it would be reasonable to claim that the accept/reject step is validation, but I believe “Parse, don’t validate” is about handling input, not an admonition to never perform validation.
> E.g., requiring that a file have the correct MIME type, not be too large, and contain no EXIF metadata.
"Parse, don't validate" doesn't mean that you must encode everything in the type system -- in fact I'd argue you should usually only create new types for data (or pieces of data) that make sense for your business logic.
Here the type your business logic cares about is maybe "file valid for upload", and it is perfectly fine to have a function that takes a file, perform a bunch of checks on it, and returns a "file valid for upload" new type if it passes the checks.
From experience, parsing input into data structures that fit the problem domain once at the "edge" is a good idea. The code becomes a lot more maintainable without a bunch of validation checks scattered all over the place, picking a data structure for the problem at hand usually leads to cleaner solutions, and errors usually show up much earlier and are easier to debug.
From experience though I've found that wrapping all data in newtypes adds too much ceremony and boilerplate. If the data can reasonably be expressed as a primitive type, then you might as well express that way. I can't think of a time where newtype wrapping would have saved me from accidentally not validating or accidentally inputting the wrong data as a parameter. Especially the email example is quite weak, with ~30 lines of code just being ceremony due to wrapping a string, and most likely it's just going to be fed as is to various crud operations that will cast the data to a string immediately.
Interacting with Haskell/elm libraries that have pervasive use of newtypes everywhere can be painful, especially if they don't give you a way to access the internal data. If a use-case comes up that the library developer didn't account for, then you might have no way of modifying the data and you end up needing to patch the library upstream.
I think it can be useful to think of the parsing and logic parts both as modules, with the parsing part interfacing with the outside world via unstructured data, and the parsing and logic parts interfacing via structured data, i.e.: the validated types.
From that perspective, there is a clear trade-off on the size of the parsing–logic interface. Introducing more granular, safer validated types may give you better functionality, but it forces you to expand that interface and create coupling.
I think there is a middle ground, which is that these safe types should be chunked into larger structures that enforce a range of related invariants and hopefully have some kind of domain meaning. That way, you shrink the conceptual surface area of the interface so that working with it is less painful.
This pattern seems to be shown almost as a comprehensive security solution when it's really just one layer of defense. This parse dont validate has to be combined with resource limits, and other protective measures during the parsing phase itself.
While this is the style I do in all of my (web) applications, it took a while to build my framework/tooling that overcomes the boilerplate to parse all incoming data into value objects. In a typical application of mine, there is usually well over dozens of entities and hundreds to thousands of fields. Creating value objects manually for all stringly types and numeric fields is a lot of work. Most value objects are some simple validation like limit to 255 characters or ints of some range.
I solve for this by using reflection and auto generating all value objects (inheriting by default from base types) and auto generating all accessor/controller classes or methods into the domain model. Therefore I model in base types, override the generated value object constructors for validation (if required), and all of the boundaries are using value objects. The internal code generally works with the underlying base types, because boxing/unboxing the value objects can be non-negligible performance impact when serializing a lot of data (which tends to be common in web applications... SQL > JSON > HTML).
I'm a huge fan, but I think ymmv. Web applications tend to have a wide interface (much of the domain model is user accessible). I think it's ideal for this case because of the number of fields a user can ultimately set and reused across many places.
24 comments
[ 4.5 ms ] story [ 63.4 ms ] threadSo you say, okay, I'll make an `email_to_string` function. Does it return a copy or a reference? Who frees it? etc, etc, and you're back to square one again. The idea is to keep char* and friends at "the edge", but I've never found a way to really achieve that.
Could just be my limitations as a C programmer, in which case I'd be thrilled to learn better.
Gratuitous allocations are gratuitous.
The whole "prevent double free" claim is completely bogus. Setting a variable to `NULL` only works for cases where there is one, obvious, owner, which is not the circumstance under which double free is prone to happening in the first place. Actually preventing double free requires determining ownership of every object, and C sucks at that.
The way I interpreted the author's intent was that, the logic of error handling (something C sucks even more at) can be greatly simplified if your cleanup routine can freely be called multiple times. At the moment an error happens you no longer have to keep track of where you are in the lifecycle of each local variable, you can just call cleanup() on everything. I actually like the idea from that standpoint.
The "practical" part really bugged me because the entire post is trying to explain exactly why it is not.
The only way to make C reasonably safe is to encode information via newtype pattern. Wrap `char *` inside a struct that have proper names and include the size in there as well.
Basically, there should be ZERO pointers except at creation and consumption by outside libraries (open, write, etc)
it is against the rules to call someone dumb on this server.
;-)
I do remember you? You had/joined a sartup in NZ, correct? Warm regards Lycium :-)
I saw you're doing consultancy last years, cool :) Lemme know if you wanna chat on Discord or so, cheers!
E.g., requiring that a string be base64, have a certain fixed length, and be provided by the user.
E.g., requiring that a file have the correct MIME type, not be too large, and contain no EXIF metadata.
If you really always need all n of those things then life isn't terrible (you can parse your data into some type representing the composition of all of them), but you often only need 1, 2, or 3 and simultaneously don't want to duplicate too much code or runtime work, leading to a combinatorial explosion of intermediate types and parsing code.
As one possible solution, I put together a POC in Zig [0] with one idea, where you abuse comptime to add arbitrary tagging to types, treating a type as valid if it has the subset of tags you care about. I'm very curious what other people do to appropriately model that sort of thing though.
[0] https://github.com/hmusgrave/pdv
https://lean-lang.org/doc/reference/latest/Basic-Types/Subty...
Of course, it would be reasonable to claim that the accept/reject step is validation, but I believe “Parse, don’t validate” is about handling input, not an admonition to never perform validation.
"Parse, don't validate" doesn't mean that you must encode everything in the type system -- in fact I'd argue you should usually only create new types for data (or pieces of data) that make sense for your business logic.
Here the type your business logic cares about is maybe "file valid for upload", and it is perfectly fine to have a function that takes a file, perform a bunch of checks on it, and returns a "file valid for upload" new type if it passes the checks.
From experience though I've found that wrapping all data in newtypes adds too much ceremony and boilerplate. If the data can reasonably be expressed as a primitive type, then you might as well express that way. I can't think of a time where newtype wrapping would have saved me from accidentally not validating or accidentally inputting the wrong data as a parameter. Especially the email example is quite weak, with ~30 lines of code just being ceremony due to wrapping a string, and most likely it's just going to be fed as is to various crud operations that will cast the data to a string immediately.
Interacting with Haskell/elm libraries that have pervasive use of newtypes everywhere can be painful, especially if they don't give you a way to access the internal data. If a use-case comes up that the library developer didn't account for, then you might have no way of modifying the data and you end up needing to patch the library upstream.
From that perspective, there is a clear trade-off on the size of the parsing–logic interface. Introducing more granular, safer validated types may give you better functionality, but it forces you to expand that interface and create coupling.
I think there is a middle ground, which is that these safe types should be chunked into larger structures that enforce a range of related invariants and hopefully have some kind of domain meaning. That way, you shrink the conceptual surface area of the interface so that working with it is less painful.
I solve for this by using reflection and auto generating all value objects (inheriting by default from base types) and auto generating all accessor/controller classes or methods into the domain model. Therefore I model in base types, override the generated value object constructors for validation (if required), and all of the boundaries are using value objects. The internal code generally works with the underlying base types, because boxing/unboxing the value objects can be non-negligible performance impact when serializing a lot of data (which tends to be common in web applications... SQL > JSON > HTML).
I'm a huge fan, but I think ymmv. Web applications tend to have a wide interface (much of the domain model is user accessible). I think it's ideal for this case because of the number of fields a user can ultimately set and reused across many places.
Examples where I've used it in the past: ValidatedEmail, which is a special form of Email, one that has been validated by the user.
We can have actions that require a `PriviligedUser`, which can be created from a `User`. That creation validates ONCE whether your user is privileged.
This saves you from a whole bunch of .is_priviliged() calls in your admin panel.
The post 'Boolean Blindness' [0] talks about much of the same issues.
[0]: https://existentialtype.wordpress.com/2011/03/15/boolean-bli...