58 comments

[ 2.5 ms ] story [ 118 ms ] thread
I use assert, but I feel like effort writing assertions in production code could be spent writing any and/or more thorough automated tests. Even if the tests are slow, you won't affect production speed. You usually can test far more in depths than verifying things inline in functions. You also wouldn't have the problem the OP mentions about not realizing that Intellij was by default not running with assertions enabled which could easily be dangerous. Just write real tests!
I don't see this as either or. Good quality asserts can be very effective when managing objects with complex state transitions

For me assertions are for invariants. The code has no expectation of successful execution

I also like (borrowing from testing) the idea of expects, where we can hope that the object is in a specific state, but have an explicit compensation path for when it isn't, perhaps objects that need to be migrated to a new schema etc.

Inline assertions do have one big advantage over tests: context. They're right there in the very place where the condition must hold. For that reason, I have long thought of assertions as 'comments with teeth'. Also, inline assertions can be enabled any time, not just when running unit tests.

Of course in theory that last bit shouldn't make any difference, but then I'm sure you've already heard the old joke about the difference between theory and practice..

> I'm sure you've already heard the old joke about the difference between theory and practice..

It's not a joke, it's a fact of life that we learn soon after graduating, and then relearn again, and again, and ...

Asserts are essentially checked comments and an automated test is not a replacement for comments.

I've done things like the following many times:

    if (x != 5) {
        ... 40 lines here ...
    } else {
        assert(x == 5);
        ... more lines ...
    }
The main purpose of the assert is to remind the reader that x == 5. There was a time where I would use a regular comment for a reminder like this, but since it's something the computer can check, I mine as well turn my regular comment into a checked comment.
The danger is change. What if someone comes and then due to a new business requirement changes the code to

    if (x != 5 && x != 6) {
        ... 40 lines here ...
    } else {
        assert(x == 5);
        ... more lines ...
    }
Because of the 40 intervening lines the person might never see your assertion!

If you work in a project with automated tests that do not have sufficient coverage, this is just a spectacular way of blowing up production.

Assertions are bad because they are checked at runtime which is too late. The ideal checked comment should have the checks happen at compile time. Something like Liquid Haskell.

> blowing up production.

I prefer what Java does: Assertions are disabled at runtime by default, however you can turn them on without requiring a different code-release and you can even do it for specific packages/classes.

So you might even use them in production: If feature X is already failing in some bizarrely uninformative way, turning on assertions in relevant code may provide you a better hint to what's going awry.

Then the assert() triggers and you fix the assert() (or the bug). What's the issue here?
In this example the assert is less explanatory than a comment. Why do

    if (condition):
        ...
    else:
        assert(!condition)
        ...
When you could instead if (condition): ... else: #if (condition) ...

Or otherwise explicitly comment the condition that was checked earlier. At least then if the comment is unchanged, whoever is touching your code will assume the comment is outdated, rather than assuming that you're asserting that x should be some undocumented magic number.

> If you work in a project with automated tests that do not have sufficient coverage, this is just a spectacular way of blowing up production.

Great, I'd rather have this error cause a loud and obvious failure that can be easily hotfixed, than have it lead to subtly incorrect behavior that goes unnoticed for days and takes another day to debug.

The entire point of failing fast is to fail fast.
Assertions do not fail fast enough. They need to fail at compile time as much as possible not at runtime.
> They need to fail at compile time as much as possible not at runtime.

Wouldn't that be nice! Seems like you're conflating two different classes of failures, though. There's no way to bypass runtime constraints entirely, though—all you can do is move it out of the production code and into tests, which takes a great deal of effort compared to an assert (which work just as well under tests, btw).

I'm not talking about tests. I'm talking about the compiler.
That's not possible. If I'm about to write some code that assumes the gyroscope is spinning, I might assert that the gyroscope is spinning. That cannot be checked at compile time.

Also, until almost all the world's software has been rewritten in Idris, compile time checks just cannot do very much. This is a reality of our century.

Keep in mind that my original example isn't really about checking that x == 5. The focus is not on checking integer values, it's just an example. Although, again, even something as simple as checking integer values is beyond the compile time abilities of our compilers. Halting problem sends its regards!

But how do you know that the gyroscope is or isn't spinning when that code is written? You don't. You might as well do a regular check for if the gyroscope is spinning, and if not return an error.

You do not need to have the entire world's software rewritten in Idris (which I don't even like that much). This is purely local reasoning that can be accomplished by minor changes to existing languages if we want to. The compiler doesn't need access to the world. It is you who are unnecessarily raising the bar for what you expect compilers to do and then throwing in the towel just because the compiler can't actually do it.

Any argument that invokes the halting problem has a standard response: you don't need to handle every case; perfect is the enemy of good. Furthermore it is often better to have the programmer rewrite the code to help the compiler deduce these facts, because helping the compiler also helps future readers of the code.

> You might as well do a regular check for if the gyroscope is spinning, and if not return an error.

I mean yea, it'd be great if programs propagated every possible error state or violated invariant possible up the stack. I don't see that happening any time soon, nor that being worth the effort. When was the last time you checked and propagated allocation failure? I'm guessing not in a long time, unless you're working with a runtime where allocation failure is explicitly handled as part of normal program progression (like the `alloc` libc family) and where you expect to run into memory limits.

Granted, exceptions are a good deal easier to handle "invisibly", but those have notably been absent from this discussion except in the case where they are unhandled as an assert might trigger.

This [1] might be a better example. C isn't Pascal, you can't just define a type of integer with a fixed range, so here, I check to ensure that the code above has done its job. I don't want an if statement here for this, for if the code above works, the if will never be triggered, and thus, as far as I'm concerned, is dead code.

[1] https://github.com/spc476/mod_blog/blob/master/src/backend.c...

Well use static_assert then. Not all checks can be done at runtime.

Also isn't failing the assertion the whole point here? Without the assertion you would have a possibly hard-to-find bug rather than an assertion failure.

What if `... more lines ...` depends on `x == 5`? Then you'd want the compiler to catch the assertion error, rather than whatever obscure error is thrown later on. And if it breaks production, you are breaking production without the assertion; runtime assertions aren't ideal but they're better than nothing.

If you don't rely on `x == 5` in the else branch it's a bad idea, and too many of these spurious assertions will create a habit of just removing assertions whenever they fail.

> this is just a spectacular way of blowing up production.

If your devs and QA team (you do have QA and not just rely on automated tests I hope) don't trigger this assert *before* deployed to production then there's something seriously wrong with your development workflow.

After all, the code after that assert also assumes that the (now invalid) assumption that was checked by the assert is true. Without the assert telling you exactly what is wrong and where, you'd have a much bigger problem: some piece of code that now runs under wrong assumptions, and which may or may not cause much harder to diagnose problems further down the road.

Think of asserts as trip-wires, they are most useful at the entry of a function and prevent an error from propagating (because the earlier an error condition is caught, the easier it is to diagnose).

Do you have QA running debug builds? In "release" builds, the asserts are compiled out. (Not sure what language and platform you're using)

Or does QA switch to release builds at some point?

When asserts fail for QA, what do they see/collect?

> In "release" builds, the asserts are compiled out.

That's just a convention, and IMHO not a good one. All the C++ games I helped shipping had asserts enabled in the release exe. We had our own custom assert macros, and different levels of assert checks (e.g. special "hot path asserts" were not included in release builds, but that was an exception and should be avoided - specifically, range-check asserts for array accesses were never removed since this was the most common source of problems). In general, the performance difference between keeping asserts in or removing them was mostly negligible (IIRC less than 2% in a typical frame), the only downside is that the executable becomes about 25% larger (and easier to reverse engineer) because of the embedded assert condition strings. This would have been solvable by replacing the assert strings with comptime hashes, but we didn't deem this important enough to implement).

In addition, the custom asserts also generated a stack trace and a mini dump for easier post mortem debugging.

> When asserts fail for QA, what do they see/collect?

A MessageBox with:

- the assert condition string

- filename and line number of the assert

- an optional printf-style programmer message (depending on the assert macro type)

- the pretty-printed function/method name which contains the assert (very useful for generic code, e.g. you don't just see `Array<T>::push()` but also the resolved template parameters)

- a stack trace (we generated a PDB file also for release builds to make the 'inhouse' stack traces human readable, but didn't ship the PDB to users - so stack traces for the shipped game would just display raw addresses, but those could be resolved on our side with the PDB we were storing for each release version - we wrote an extra inhouse tool for that)

- plus a mini dump is written for post-mortem debugging, but with all the information above that was hardly ever needed for figuring out what the problem was

The content of Windows MessageBoxes can be Copy-Pasted, and that's what QA was supposed to do when writing a ticket.

PS: also important to note that we didn't use the C++ stdlib (with very few exceptions, like std::sort), e.g. most importantly we wrote our own container and string types).

Thanks for the details, sounds like a great setup. Any GitHub repos you can recommend?

Except for writing your own stdlib. How long did that take? Was that to enhance bounds checking?

It's all inhouse code unfortunately, and from a previous life :)

What I still do in my open source libraries is to allow overriding the assert macro so that people can integrate their own assert implementation (along with allowing to override memory allocation).

> Except for writing your own stdlib. How long did that take? Was that to enhance bounds checking?

TBH the main reason was that I think the C++ stdlib APIs have terrible ergonomics. We modelled our container classes after C# containers and some common sense. Full control over memory allocation and asserts was just a side effect.

Depends on your goals for production. The code above indicates a bug. In some domains it really is preferable to crash with righteous fury rather than to continue executing with broken invariants and doing who knows what.

The assertion didn't cause you to have limited test coverage that let you edit the branch condition without detecting that it introduced a bug in the false branch.

Yes, if you've got a language that supports compile time checking of complex invariants then that's awesome! But few languages support this today.

The problem with crashing is that it is almost always a bad idea. You almost always want a limited scope crash that affects whatever currently is being processed. Imagine that in some languages an assertion raises an AssertionError exception which is caught by some faraway handler that causes the server the respond with a 500 error. That's strictly better than crashing.

I also used to think it might be preferable to crash in some domains. With stories like https://news.ycombinator.com/item?id=37461695 it makes me think perhaps there are none.

In any case my main point is that assertions would be perfect if only they are checked at compile time.

Unit tests are complementary to assertions, they don't supersede them.

I feel that assertions are the first line of defense against programming judgement errors/omissions and are much easier to write in the given context than unit tests.

> effort writing assertions in production code

Presumably the assert is used because it doesn't consume resources (ie effort) to write. Testing takes a lot of effort.

Seems to me that asserts are a force multiplier on tests. If you have a bunch of asserts, every test case you write is extra valuable -- you aren't just verifying correct input/output behavior, you're also verifying that all asserts held.

In the extreme case, I can imagine ditching all your small-scale unit tests for a combination of asserts and end-to-end integration tests.

In C and C++ at least, asserts are also hints for the static analyzer (for instance that a pointer isn't expected to be null).

It's better to do both, asserts and tests are not mutually exclusive, they complement each other.

This has always been a debate in my head whether to make the function return error codes or to throw those errors.

Irrespective of throwing an error or returning an error you need to handle it somewhere. If you are returning an error then the type system can handle it and the person calling your function can get an idea.. while if you are throwing an error in most of the languages your caller function will not get an idea that this function can throw an error.

>if you are throwing an error in most of the languages your caller function will not get an idea that this function can throw an error

Is this true any more? I feel like most languages, relative to popularity, have compiler-enforced thrown error handling.

C# is one of the offenders. And while Java has checked exceptions, many considered Thema an annoyance and wrap everything in RuntimeException.
Very true. Checked exceptions in Java are a form of function coloring problem for modern libraries.

Especially painful when using any of the functional style stream operations, like `map`, `filter`, etc, or any other higher order function library. Most take in the standard `Function` or `BiFunction` interface arguments, which will not support method referencing for anything which includes a checked exception as part of its signature.

But a better language would allow generalizing over exceptions. In pseudo C++:

  auto map(collection<T> C collection, function<T> F f) -> invoke_result<F, T> throws(invoke_exception<F, T>);
> Checked exceptions in Java are a form of function coloring problem for modern libraries.

So is returning result or error via Either/Expect/Result sum types. Exceptions or Expected, code gets messed up either way.

Not quite of the same caliber. A checked exception has the form `R func(T t) throws Err`. A concrete result type (like Rust has) would look like `Result<R, Err> func(T t)`.

The later fits nicely into the `Function<,>` interface, `Function<T, Result<R, Err>>` vs the former requires a different interface entirely, something like `FuncE<T, R, Err>` where `Err` breaks out into an argument for the throws value of the signature.

Because most of the functional libraries in Java work with Function, BiFunction, etc, we end up with incompatible arguments for common patterns such as `map`, `filter`, etc.

Typescript, Python all are exception
That’s the thinking behind Either (in functional programming, but not only).

TBH, I don’t understand how the absence of checked exceptions or Either in most languages is tolerated!

> the absence of checked exceptions or Either

It seems an unpopular opinion, but I like Checked Exceptions and wish more languages had them. Most of the "problems" people mention about CEs boil down to the damage done by lazy developers who don't want to think of separation-of-concerns/decoupling/layering in their own code, and that can be curbed by better syntactic sugar.

It's very controversial, but I like Go's approach of treating errors as return values - because that is ultimately what they are.

You can have your language hide exceptions from the control flow if you wish, but they are still things that result from function calls that you should deal with. Why not make them front and center?

You should use asserts or throw errors to prevent misuse of a procedure by the caller. Ideally, you could model these constraints at the type level. You can think of pre-conditions as a sort of pseudo-dependent typing.

Consider the following pseudocode:

    // Idris
    binarySearch : SortedList a -> a -> Maybe Nat
    binarySearch = ?implementation

    // Java
    int binarySearch(List<T> elems, T elem) {
        assert isSorted(elems);
        throw new NotImplemented();
    }
(The generic type parameter should also implement Comparable or similar, but I digress).

In this case, it would be nonsensical to pass an unsorted list to either procedure, so we prevent it - in Idris' case by using dependent typing; in Java's case by using assertions.

To contrast, consider the following:

    // Idris
    readLines : Path -> IO (Either FileError (List String))
    readLines = ?implementation
Here, the FileError represents a number of things that could go wrong: no file existing at the given path, lack of permissions, running out of memory during read and so on. However, this is not misuse of the procedure - these are all expected deviations from the happy path, and hence we model that in the type definition.

(I've omitted a corresponding Java example, as following a similar style would be terribly non-idiomatic. Java prefers throwing and catching exceptions to manage non-happy paths)

Note that we can't constrain the Path parameter to only resolve to files that exist. Doing so would require knowledge of the current filesystem state, e.g. the type signature would be something like the following:

    readLines : (fs : FileSystem) -> (p : Path) -> fileExists fs p -> IO (Either FileError (List String))
    readLines = ?implementation
To summarize, use preconditions, assertions or type-level constraints to prevent misuse of procedures. Return errors (using sum types, for example) when it is known and expected that the result of a procedure can deviate from the happy path. You may refrain from modeling conditions at the type level, even if possible in principle, as it may be prohibitively complex.
>I wouldn’t blame you for invoking Objects.requireNotNull on your arguments, but think about whether that’s better than an assertion for the end user. My position is that it’s not, since the user would be seeing precisely the same exception, manifesting in exactly the same way. Note that it’s not wrong to use both, as assertions can be far more powerful.

Is this true if the function forwards the value to another function, which may do the same thing?

> The pronunciation of assert

This section I like very much. I wish Haskell people had the balls to do this for the X -> x -> x sentences and a myriad of other Haskellisms.

I still pronounce assert(x) as "If you see the message, then the condition it asserts has FAILED"

(comment deleted)
Honestly, my first question was "why not use a combination of user-created types and exceptions to ensure program validity." But I get the impression that asserts are used for alerting programmers to their own syntactical and logical errors; unlike exceptions, which can occur in logically proper code.
Exactly.

Assertions are for programmers and their internal work. Exceptions for the rest, mainly the dynamic, user controlled inputs and their derivatives.

i dont understand why i as a user would want a program to assert (i.e. crash) in an interactive program. i would lose all my in-progress work and have to start the program over from scratch. and chances are very low i will be able to report the bug to the programmer, for various reasons. so the programmer will never know about it. i will just keep having to restart the program over and over and workaround it.

i get that its better than memory corruption. but ... is it better than just muddling through and reporting you cannot do operation x?

This would probably be used in places where, if the “x” in assert(x) doesn’t hold, then the program could do more harm than good. You’ll generally try to handle situations that are recoverable without blowing up stuff like this, but in some software and some situations you want it to blow up fast rather than destroy data or whatever. I program in Python more, which has slightly different assert behavior so it’s considered more of a code smell in that language, but it seems like Java programmers use it more because it can be turned off easily when not wanted.
Asserts are not an alternative to validating user input or input data in general (e.g. data loaded from files or received over the network).

If wrong input triggers an assert, that means that the input validation is incomplete (and in that case the assert may prevent further much worse damage, and also provide important information to the programmer who's going to fix that bug).

Basically, asserts should never trigger if the user "does something wrong".

The user obviously doesn't want the program to crash, but they don't want to encounter any bugs. If an assertion fails, it means they already encountered a bug and the program is about to wander through a minefield of unspecified behavior, which could be much worse than simply stopping.
>Assertions MUST NOT be the only thing between misbehavior and input data, including data returned from other APIs (e.g. fopen calls, i.e. auto x = fopen ("foo"); assert (foo != NULL); is invalid). Anything dealing with foreign data must not assume anything about the state or contents of the data, and as such, cannot ever assert, as it doesn’t have an assertion to make. If you take away only one thing from this article, let it be this: assertions do not verify arbitrary inputs.

The author seems overconfident in this point. Like, what if the "input data" is from another software system I control?

Triggering an assert failure on bad user input is a little sloppy, but if there's no way to recover at that point anyways, I don't see why it matters.

Can someone enlighten me?

Re: data from another of your own systems. If there’s an interface between them that suggests you probably want those systems decoupled. Maybe in the future you replace one system with another you don’t control - ideally it would be nice not to have to make code changes because of that.

Re: asserting on user input. If you assert only then what happens on bad inputs when asserts are switched off in prod? Seems like an outcome you’d like to control at the time rather than allow to propagate (by writing bad data to a db and only being discovered months later, for example).

>what happens on bad inputs when asserts are switched off in prod

Good point

It'd be nice if the author mentioned the real purpose of assertion statements: *it's meant to catch a programmers' errors (your own, or other people working with your code). Not a users' errors.* From that stems all assert use-cases and all requirements to it.
and from that - all the reasons to use it. A properly placed assertion statement is the only thing that stays between you and a misery of weeks-long debugging sessions.
"Assertions MUST NOT be the only thing between misbehavior and input data"

I don't agree with this. I'm quite happy to assert that an external function call worked.