Wait a second--is this really using exceptions for control flow?
One of the Pragmatic Programmer's tips[1] is "Use Exceptions for Exceptional Problems". Network delays and problems aren't exceptional--they should be expected. It might make sense to throw an exception if you aren't going to handle the network issue, or if you have already tried to handle it and failed. But if you're trying to handle the issue then it shouldn't throw an exception.
Even if you don't buy this from a theoretical standpoint, you should look at your performance using this method. Exception performance is terrible in almost every language. I'm not sure how Clojure exception performance is, but guessing from Java, it's probably not an exception (har har) to the rule.
I'm not a Clojure expert, but I would be really surprised if Clojure didn't offer a non-exception way to do retries.
The most expensive part of Java exceptions is building the stacktrace. You can avoid this by overriding the fillInStacktrace method with an empty implementation.
> Wait a second--is this really using exceptions for control flow?
No.
If it used exceptions for control-flow, the defined function would actually throw an exception. Here, we catch "Exceptional Problems" and ignore them at most 3 times.
A common example of using non-local exit is to return from deeply nested recursive calls.
(edit: of course, you can argue that network delays are not exceptional (surley you love Go), but the code shown by OP is supposed to handle any kind of exceptional situations. Of course, if there was an error value, you could define another function that checks that error value)
That being said, "Exceptions for exceptional situations" is exactly like "never use goto". Non-local exits are definitely useful and not necessarly costly.
> Performance is terrible in almost every language
Even in cases where this is true, it can make sense to use them for control flow according to your requirements and actual measures.
The problem with "Use Exceptions for Exceptional Problems" is that it avoids ever defining "exceptional". Usually, it is then interpreted to mean "couldn't be predicted". But if there is a try-catch anywhere in the code, then obviously it could be predicted. Therefore, a corollary to "use exceptions for exceptional problems" is "never use exceptions", which is ridiculous.
My rule of thumb is to use an exception whenever the function cannot complete its contract. After that, it is up to the calling function to fix the problem, or however high up the stack until some function knows what is going on. If I have a function LogIn(username, password), I expect to be logged in after the function completes. Anything else, and I should have a exception thrown, whether it be due to lost connection, incorrect username/password, or anything else. On the other hand, if I have a function AttemptLogIn(username, password), then it would be sufficient for the function to return a boolean for success/failure.
I would modify the statement to "Use Exceptions for Infrequent Problems". Interacting with a user? Sure. Anything that trickles back up to the user and is displayed as an error message will have lag based on the user's reading speed. During heavy numerical calculations? No.
> The problem with "Use Exceptions for Exceptional Problems" is that it avoids ever defining "exceptional". Usually, it is then interpreted to mean "couldn't be predicted".
Huh? No. "Exceptional problems" are ones where the only thing that the code does is report the error. Whether a problem is exceptional is dependent on whether or not it's part of the normal workflow.
This is why there are, for example, 2 integer parsing methods in many languages, one that returns false for strings that aren't integers and one that throws an exception. It's up to the user of the function whether they have a way to handle the exception or whether they just want to report it to the user and exit.
> My rule of thumb is to use an exception whenever the function cannot complete its contract.
Exceptions that a function might throw are part of its contract.
> If I have a function LogIn(username, password), I expect to be logged in after the function completes.
That's a terrible design. You've violated a ton of different principles: single responsibility principle being the biggest.
For this I would have something like `authorize(username, password)` that figures out whether the username/password is valid, and then `login()` that actually creates the session. I might have another function `tryLogin(username, password)` that tries to login and returns some error info on failure.
In your system, what happens when a login happens and slips past your exception handler because it's used in a place that you didn't expect it to be used, and there's no type checking on exceptions? You just let the whole system go down. What happens when you want to log in with an auth token instead of a username/password? Have fun separating out your session logic. What happens when you want to write stress tests against your login system and now have to piece together a test suite that covers all cases for a monolith because you don't have any units to unit test? You rewrite the whole thing is what happens.
> Anything that trickles back up to the user and is displayed as an error message will have lag based on the user's reading speed.
Or it will cause lag for your entire server because wrong logins are fucking your stack and therefore your garbage collection.
> Exception performance is terrible in almost every language.
Duck-typing using exceptions is very idiomatic in Python. Also, raising an exception instead of returning error values is a popular API design. Hell, the iterator API - the foundation of loops in Python - raises an exception (StopIteration).
I guess this prescription about exceptions was made with some specific language in mind and it is dangerous to generalize.
Yes, I do about 25% of my daily dev in Python and have done so for years, and I'm well aware that this is idiomatic in Python. Python was the specific language I was thinking of when I said "almost".
> I guess this prescription about exceptions was made with some specific language in mind and it is dangerous to generalize.
I guess this post about generalizing was made ignoring the word "almost" and it is dangerous to ignore qualifiers.
I still think "almost any language" is a stretch, but I'm not in the mood of profiling several languages in order to assert this claim - without numbers my experience is as anecdotal as yours.
I agree that using exceptions as flow control is b-a-d. This is an easy fix in this case, set `:throw-exceptions false` in your http call and check the status code, rather than catch exceptions.
> Network delays and problems aren't exceptional--they should be expected. ... might make sense to throw an exception if you aren't going to handle the network issue, or if you have already tried to handle it and failed....But if you're trying to handle the issue then it shouldn't throw an exception.
It sounds like you think exceptions should be used for things you've never thought of, and have no code to handle. But if you don't think there's a way to handle it, what good are exceptions for you - you might as well just barf.
I think in practice people do use exceptions for things they both expect to happen, and know how to handle at that point.
For that to work, you need to enumerate either the acceptable failures (network timeouts) or the unacceptable failures (logic error).
However, for idempotent actions, why does it matter what caused the exception? The worst that happens is you waste 3x the resources, and the benefit is you avoid false negatives from overspecifying your unacceptable failures.
In java there's this distinction between Error (sth that's usualy non-recoverable), checked exception, and runtime exception. You would usually only catch runtime exceptions here if this was java (checked are checked staticaly, so they need to be caught anyway).
It's running on the Java VM, so I guess catching the exception created by the user pressing CTRL-C is very bad. It's also bad to catch the one created by Runnable.interrupt.
The advantage of a macro is that one doesn't have to wrap one's own code in a closure. It's cleaner to write `(try-n-times (let ((foo (bar 1))) (baz foo 2)) 3)` than `(try-n-times (lambda () (let ((foo (bar 1))) (baz foo 2))) 3)`.
Also: man, writing Lisp without paren-matching bites.
That's nice, but what about adding backoff? I just came across this blog post[1] over at AWS about exponential backoff - timely because I'd just been working on some retry logic myself and wondered if I'd made good decisions about how to implement jitter. Turns out, I hadn't.
29 comments
[ 3.7 ms ] story [ 64.5 ms ] threadOne of the Pragmatic Programmer's tips[1] is "Use Exceptions for Exceptional Problems". Network delays and problems aren't exceptional--they should be expected. It might make sense to throw an exception if you aren't going to handle the network issue, or if you have already tried to handle it and failed. But if you're trying to handle the issue then it shouldn't throw an exception.
Even if you don't buy this from a theoretical standpoint, you should look at your performance using this method. Exception performance is terrible in almost every language. I'm not sure how Clojure exception performance is, but guessing from Java, it's probably not an exception (har har) to the rule.
I'm not a Clojure expert, but I would be really surprised if Clojure didn't offer a non-exception way to do retries.
[1] https://pragprog.com/the-pragmatic-programmer/extracts/tips
No.
If it used exceptions for control-flow, the defined function would actually throw an exception. Here, we catch "Exceptional Problems" and ignore them at most 3 times. A common example of using non-local exit is to return from deeply nested recursive calls.
(edit: of course, you can argue that network delays are not exceptional (surley you love Go), but the code shown by OP is supposed to handle any kind of exceptional situations. Of course, if there was an error value, you could define another function that checks that error value)
That being said, "Exceptions for exceptional situations" is exactly like "never use goto". Non-local exits are definitely useful and not necessarly costly.
> Performance is terrible in almost every language
Even in cases where this is true, it can make sense to use them for control flow according to your requirements and actual measures.
You realize that catching and ignoring an exception is using exceptions for control flow, right?
My rule of thumb is to use an exception whenever the function cannot complete its contract. After that, it is up to the calling function to fix the problem, or however high up the stack until some function knows what is going on. If I have a function LogIn(username, password), I expect to be logged in after the function completes. Anything else, and I should have a exception thrown, whether it be due to lost connection, incorrect username/password, or anything else. On the other hand, if I have a function AttemptLogIn(username, password), then it would be sufficient for the function to return a boolean for success/failure.
I would modify the statement to "Use Exceptions for Infrequent Problems". Interacting with a user? Sure. Anything that trickles back up to the user and is displayed as an error message will have lag based on the user's reading speed. During heavy numerical calculations? No.
Huh? No. "Exceptional problems" are ones where the only thing that the code does is report the error. Whether a problem is exceptional is dependent on whether or not it's part of the normal workflow.
This is why there are, for example, 2 integer parsing methods in many languages, one that returns false for strings that aren't integers and one that throws an exception. It's up to the user of the function whether they have a way to handle the exception or whether they just want to report it to the user and exit.
> My rule of thumb is to use an exception whenever the function cannot complete its contract.
Exceptions that a function might throw are part of its contract.
> If I have a function LogIn(username, password), I expect to be logged in after the function completes.
That's a terrible design. You've violated a ton of different principles: single responsibility principle being the biggest.
For this I would have something like `authorize(username, password)` that figures out whether the username/password is valid, and then `login()` that actually creates the session. I might have another function `tryLogin(username, password)` that tries to login and returns some error info on failure.
In your system, what happens when a login happens and slips past your exception handler because it's used in a place that you didn't expect it to be used, and there's no type checking on exceptions? You just let the whole system go down. What happens when you want to log in with an auth token instead of a username/password? Have fun separating out your session logic. What happens when you want to write stress tests against your login system and now have to piece together a test suite that covers all cases for a monolith because you don't have any units to unit test? You rewrite the whole thing is what happens.
> Anything that trickles back up to the user and is displayed as an error message will have lag based on the user's reading speed.
Or it will cause lag for your entire server because wrong logins are fucking your stack and therefore your garbage collection.
Duck-typing using exceptions is very idiomatic in Python. Also, raising an exception instead of returning error values is a popular API design. Hell, the iterator API - the foundation of loops in Python - raises an exception (StopIteration).
I guess this prescription about exceptions was made with some specific language in mind and it is dangerous to generalize.
> I guess this prescription about exceptions was made with some specific language in mind and it is dangerous to generalize.
I guess this post about generalizing was made ignoring the word "almost" and it is dangerous to ignore qualifiers.
It sounds like you think exceptions should be used for things you've never thought of, and have no code to handle. But if you don't think there's a way to handle it, what good are exceptions for you - you might as well just barf.
I think in practice people do use exceptions for things they both expect to happen, and know how to handle at that point.
They provide debugging information to you, as well as the ability to report to the user what went wrong rather than just that something went wrong.
> I think in practice people do use exceptions for things they both expect to happen, and know how to handle at that point.
This is true, and also causes terrible performance. Especially for situations like this where retries are likely to happen quite frequently.
However, for idempotent actions, why does it matter what caused the exception? The worst that happens is you waste 3x the resources, and the benefit is you avoid false negatives from overspecifying your unacceptable failures.
So yeah, catching throwable is wrong usually.
A normal function would have done pretty much the same job...
Also: man, writing Lisp without paren-matching bites.
(map (lambda (x) (try-3 x)) collection-of-try-threeables)
...that's why a lightweight anonymous function notation is useful (Clojure's #() reader macro, etc.).
[1] http://www.awsarchitectureblog.com/2015/03/backoff.html
So, TRY-N-TIMES should either be named RETRY-N-TIMES or the base case should be 1; as it is, `(try-n-times #(foobar) 1)` will potentially try twice.