We use async code in two modes which have different very different consequences for concurrency issues.
We have an imperative code flow where we perform a series of tasks that involve IO, and apply the effects sequentially. Here the biggest problem is holding a lock for a long transaction and starving the rest of the system. So we break it up into a finite state machine where the lock is held mostly during the synchronous parts.
The other is asking a lot of questions and then making a decision based on the sum of the answers. These actually happen in parallel, and we often have to relax the effective isolation levels to make this work. But it always seems to work better if the parallel task can be treated as a pure function. Purity removes side effects, which removes the need for write locks, which if applied consistently removes the Dining Philosopher’s problem. “Applied consistently” is the hard part. Because it requires not just personal discipline but team and organizational discipline.
> There is usually not much of a point in writing a finalizer that touches only the object being finalized, since such object updates wouldn’t normally be observable. Thus useful finalizers must touch global shared state.
That seems like an “Abandon hope, all ye who enter here.”
Unless I'm missing something, this has nothing to do with asynchronous code. The delete is just synchronous code running, same as if we called a function/closure right there.
This is just about syntax sugar hiding function calls.
While I think the problem highlighted in the article is a longstanding problem for Rust[1], I don't think the example, or finalizers was the problem with Futurelock as described by Oxide.
I'm not sure you can write a simple example in Python, because Rust's future's architecture and Python's is different. `futurelock` is an issue of cancellation safety which is a stranger concept (related to finalizers, but not in the way OP has described).
Personally, I think `tokio::select!` is dangerous and I don't use it my code - it's very easy to deadlock yourself or create weird performance issues. I think the interface is too close to Go and if you don't understand what is going on, you can create deadlocks. That said, even if you avoid `tokio::select!`, I think cancellation safety is one of those dragons that exist in async rust.
A __del__ that does any kind of real work is asking for trouble. Use it to print a diagnostic reminding you to call .close() or .join() or use a with statement, and nothing else. For example:
def close(self):
self._closed = True
self.do_interesting_finalisation_stuff()
def __del__(self):
if not self._closed:
print("Programming error! Forgot to .close()", self)
If you do anything the slightest bit more interesting than that in your __del__, then you are likely to regret it.
Every time I've written a __del__ that did more, it has been trouble and I've ended up whittling it down to a simple diagnostic. With one notable exception: A __del__ that put a termination notification into a queue.Queue which a different thread was listening to. That one worked great: If the other thread was still alive and listening, then it would get the message. If not, then the message would just get garbage-collected with the Queue, but message would be redundant anyway, so that would be fine.
The article missunderstands the futurelock problem. It assumes that the mutex is a "normal" blocking mutex and that the future blocks. This is not the case: The future does cooperate and returns the control back to the `select!` call. The problem is that the `select!` call does not have access to the future that holds the lock, so it cannot make progress.
10 comments
[ 3.5 ms ] story [ 23.8 ms ] thread[0]: https://news.ycombinator.com/item?id=45898923
We have an imperative code flow where we perform a series of tasks that involve IO, and apply the effects sequentially. Here the biggest problem is holding a lock for a long transaction and starving the rest of the system. So we break it up into a finite state machine where the lock is held mostly during the synchronous parts.
The other is asking a lot of questions and then making a decision based on the sum of the answers. These actually happen in parallel, and we often have to relax the effective isolation levels to make this work. But it always seems to work better if the parallel task can be treated as a pure function. Purity removes side effects, which removes the need for write locks, which if applied consistently removes the Dining Philosopher’s problem. “Applied consistently” is the hard part. Because it requires not just personal discipline but team and organizational discipline.
> There is usually not much of a point in writing a finalizer that touches only the object being finalized, since such object updates wouldn’t normally be observable. Thus useful finalizers must touch global shared state.
That seems like an “Abandon hope, all ye who enter here.”
This is just about syntax sugar hiding function calls.
I'm not sure you can write a simple example in Python, because Rust's future's architecture and Python's is different. `futurelock` is an issue of cancellation safety which is a stranger concept (related to finalizers, but not in the way OP has described).
Personally, I think `tokio::select!` is dangerous and I don't use it my code - it's very easy to deadlock yourself or create weird performance issues. I think the interface is too close to Go and if you don't understand what is going on, you can create deadlocks. That said, even if you avoid `tokio::select!`, I think cancellation safety is one of those dragons that exist in async rust.
[1] https://without.boats/blog/poll-drop/
Every time I've written a __del__ that did more, it has been trouble and I've ended up whittling it down to a simple diagnostic. With one notable exception: A __del__ that put a termination notification into a queue.Queue which a different thread was listening to. That one worked great: If the other thread was still alive and listening, then it would get the message. If not, then the message would just get garbage-collected with the Queue, but message would be redundant anyway, so that would be fine.
Yeah, at some point, I was working on a prototype of finalization for JavaScript, and that was also my conclusion.
Did anyone assure that this code is using a recursive mutex???
Because, well, lol, a second lock on a non-recursive mutex would look like what is reported here.
KISS and gn8