Just want to note and curio and trio, two non-stdlib python async libraries, have file io as batteries included.
I would also like to point out that asyncio made a huge mistake here. IO is not limited to interactions with the network buffer. I don't know how something called asynio can be released in to stdlib, and not include disk io.
The fact that there is no actual async file io on the OS doesn't matter. Dealing with files is a core concept in almost every application, and so pushing this to stdlib and hoping the python ecosystem takes up the slack, or people handbake threaded access in themselves, is a joke.
Some of this is a little dated, but a friend of mine wrote this and included my complaints about file io in there somewhere too:
I disagree that dealing with files is a core concept in almost every asyncio application. I would say that the most common instances of file IO is at the start of an asyncio process (parsing config files) rather than in the “hot” asynchronous part.
Some problems might require extensive file IO, sure. But most don’t, and something like reading from a database or making a HTTP request is orders of magnitude more common.
The way that the linked thread passionately presents asyncio as very obviously wrong, like it was written by idiots, is a bit of a turn off. The author seems especially offended that asyncio's interface isn't as good as curio (of which trio is a fork):
> Then I found curio. And I realised how badly designed asyncio was.
That quotation, the rest of the text at that link, and the comment I'm replying to, all hinge on the fact that asyncio was the first async library available. There are two consequences of this:
1. asyncio was used to drive library design forward. Yes it's easy to look at curio (or trio) and say that the design is obviously better than asyncio, but even the author admits they only realised it after they saw curio. This shows how legitimately difficult it is to design an async API. The best way to come up with any design is really to start off with a first attempt and use that (especially by analysing its weaknesses) to come up with a new one. That's what happened here; I don't think curio could ever have been created without asyncio coming first to inspire them, even if it was partly the mistakes that inspired them. Indeed the asyncio API has changed a lot since that post was written, partly learning from its own mistakes directly (e.g. there's almost no reference to the event loop in the API now) and partly learning from the API in curio/trio (e.g. asyncio.run()).
2. asyncio was used to drive async language features forward, many of which are relied upon by curio and trio for the designs referred to in that post. These were ultimately used in asyncio too (as you said, many arguments in that link are now out of date - the main objection is surely "Ignoring the generator API" which is no longer true). I don't think an external library could have pushed language evolution as quickly as a part of the standard library did. For language stability reasons, the standardised version of asyncio always lagged the language features by one Python version.
In many ways, the primary job of asyncio was to get something working to allow these two things to be iteratively improved. Missing out file IO is reasonable because it still allows the library to accomplish those goals, and people can still call out to a thread to do it (which needs to have a good API anyway because there's no avoiding it for CPU-bound tasks). In many platforms, that's all that would happen under the hood anyway.
------
A couple of specific responses to the objections in the link (many are simply "that's no longer true" and I'm not going to try and list all those):
> Additionally, writing is not async. Is all your data sent when you write? Nope! ... The work around, of course, is to call writer.drain(). Why it doesn't do this automatically, who knows?
Sometimes you want to queue up data to be written to a socket and then do something else (in that coroutine), and sometimes to you want to queue up that data and wait until it actually gets written. The only time I've used asyncio in a real-world project, I've wanted the former; waiting would've been a disaster! The obvious way to allow this flexibility is to split up those two operations - and they really are separate operations - into two separate functions. This isn't a "workaround", it's clean design.
The alternative is to make it all in one function. That still technically allows both use cases: if you want to just queue up data then you can spawn a new coroutine for doing that with a queue/stream to send it data from other coroutines. But that's causing one of the two API uses to have way more code than the other, whereas having to choose between calling one or two functions is really not a big deal. It also means there's the runtime overhead of two queues, since the internal queue of data to write still really exists, and it's harder to arrange for smaller packets to be coallesced for writing... and so o...
> This is likely in part because operating systems themselves also lack these facilities.
This is a surprising statement. My impression was that OS file operations are generally async, with sync built on top.
Node.js uses libuv for async file operations -- And libuv has its own internal thread pool to make file ops appear fully async to a node program.
From the libuv docs [0]:
Unlike network I/O, there are no platform-specific file I/O primitives libuv could rely on,
so the current approach is to run blocking file I/O operations in a thread pool.
For a thorough explanation of the cross-platform file I/O landscape, checkout this post [1].
This has actually changed with io_uring in the latest sets of linux kernels. It's all still very new and not well supported in most standard libraries or languages yet though.
For cases like that discussion about opening lots of files over NFS this should allow it to be done completely async. The actual time to do it won't be better on a per file basis but you'd be able to parallelize them and do other work while waiting on the opens to complete.
well yeah io_uring would fix this, but as far as I know there is no support for opening files with io_uring, yet? maybe I'm wrong here tough, but the initial interface was without.
> The other threads need to continue running while the thread waiting on open(2) is paused, but ptrace pauses the whole process.
This definitely doesn't sound right. Running strace, you need to specify -f in order to trace threads, otherwise only the main thread is traced. gdb does stop all threads on interrupts, but only by default for convenience, and I don't think it's atomic.
Because of this, LD_PRELOAD is not the easiest way to delay system calls, strace -e inject:delay is.
$ cat > a.c << EOF
#include <stdio.h>
#include <sys/types.h>
#include <fcntl.h>
#include <pthread.h>
#include <unistd.h>
void *start_thread(void *arg) {
while (1) {
puts("thread");
sleep(1);
}
}
int main() {
pthread_t thr;
pthread_create(&thr, NULL, start_thread, NULL);
while (1) {
puts("main");
close(open("/dev/null", O_RDONLY));
sleep(1);
}
}
EOF
$ gcc a.c
$ strace -o /dev/null -P /dev/null -e inject=openat:delay_enter=3s ./a.out
main
thread
thread
thread
thread
thread
main
thread
thread
thread
thread
thread
[ and so on. ]
using threads for file IO is a hammer approach imho.
Often times the price you pay to send the job to the queue, yield for another coroutine, wait for IO task to finish, receive a notification back from the thread pool and resume is higher than doing the blocking call. Under high load it might have some small benefits but my hunch is that it's unlikely to buy you much.
Luckily, io_uring officially made it to the linux kernel few months ago and can solve these problems in a much more efficient manner. *BSD solved this problem long time ago, in 2002, when kqueue was introduced.
Note this is specifically about opening and closing files, which may be on network mounts. kqueue has no support for this, io_uring just very recently got it. So there's no alternative to threads today.
kqueue does support this via EVFILT_VNODE. io_uring is in linux as of kernel 5.1(march 2020). alternatives are available but might not be viable for everyone.
kqueue is not responsible for initiation. It's responsible to notify you about events. when you open(2) a file with O_NONBLOCK flag, you can register it with kqueue to let you know when it's readable/writable.
I've been solving similar problem to decouple python daemon logging IO from event loop in order to make logging completely non-blocking. I'm pretty sure this particular case must be most popular file IO case for async applications.
In my case it wasn't important to open log asynchronously, but it was critical to have async writes.
It turns out, depths of Python stdlib hide logging.handlers.QueueHandler and logging.handlers.QueueListener which allow to split log message handler into separate thread from log message emitters, making them communicate via synchronized queue.
It is convenient that you can still use logging as usual, without any
async obligations. You just have logging function which works in a
fire-and-forget fashion.
> Asynchronous reads and writes would require all new APIs with different coloring. You’d need an aprint() to complement print(), and so on, each returning an awaitable to be awaited.
> This is one of the unfortunate downsides of async/await. I strongly prefer conventional, preemptive concurrency, but we don’t always have that luxury.
I was sceptical about Golang for a long time, but recently I've started practicing it. It's designed with concurrency in mind from very beginning and goroutines have advantages of both cooperative and preemptive multitasking. And also it doesn't have GIL, so your goroutines will use all available CPUs without any additional effort from your side.
From programmers point of view, coding with goroutines looks exactly like coding with use of pthreads (even simpler, in fact). So, there are no function "coloring", no need in sync/async library flavors and so on. But these "threads" also have cost close to cost of python coroutines. It sounds too good to be true, but I've rewritten bunch of my Python/asyncio code in Go and I can say it's true. For this reason (and many others I discovered later) I quit using Python and asyncio for network applications in favor of Go.
People seem to complain and I do agree that we are not yet approaching an elegant solution.
I have used Asyncio for an API I built for https://www.abstractapi.com/ in order to be able to manage high load and avoid being locked by network latency.
It has been painful and I almost thought about migrating to Amazon Lambda to distribute the load but it feels like a bad use case for Lambda when you're actually only want to perform async stuff and not lock a full thread.
SCan't believe we still don't have a proper async lib for Python in 2020. Still have to test Curio but the whole thing makes me seriously reconsider which language I'll use for my next project. Async is something I need to handle more and more.
27 comments
[ 4.4 ms ] story [ 21.7 ms ] threadI would also like to point out that asyncio made a huge mistake here. IO is not limited to interactions with the network buffer. I don't know how something called asynio can be released in to stdlib, and not include disk io.
The fact that there is no actual async file io on the OS doesn't matter. Dealing with files is a core concept in almost every application, and so pushing this to stdlib and hoping the python ecosystem takes up the slack, or people handbake threaded access in themselves, is a joke.
Some of this is a little dated, but a friend of mine wrote this and included my complaints about file io in there somewhere too:
https://web.archive.org/web/20171206105117/https://veriny.tf...
[1] http://www.dabeaz.com/ https://dabeaz.com/talks.html
Some problems might require extensive file IO, sure. But most don’t, and something like reading from a database or making a HTTP request is orders of magnitude more common.
From personal experience, even for serving files, I find myself relying a lot more on web services rather than a local disk.
> Then I found curio. And I realised how badly designed asyncio was.
That quotation, the rest of the text at that link, and the comment I'm replying to, all hinge on the fact that asyncio was the first async library available. There are two consequences of this:
1. asyncio was used to drive library design forward. Yes it's easy to look at curio (or trio) and say that the design is obviously better than asyncio, but even the author admits they only realised it after they saw curio. This shows how legitimately difficult it is to design an async API. The best way to come up with any design is really to start off with a first attempt and use that (especially by analysing its weaknesses) to come up with a new one. That's what happened here; I don't think curio could ever have been created without asyncio coming first to inspire them, even if it was partly the mistakes that inspired them. Indeed the asyncio API has changed a lot since that post was written, partly learning from its own mistakes directly (e.g. there's almost no reference to the event loop in the API now) and partly learning from the API in curio/trio (e.g. asyncio.run()).
2. asyncio was used to drive async language features forward, many of which are relied upon by curio and trio for the designs referred to in that post. These were ultimately used in asyncio too (as you said, many arguments in that link are now out of date - the main objection is surely "Ignoring the generator API" which is no longer true). I don't think an external library could have pushed language evolution as quickly as a part of the standard library did. For language stability reasons, the standardised version of asyncio always lagged the language features by one Python version.
In many ways, the primary job of asyncio was to get something working to allow these two things to be iteratively improved. Missing out file IO is reasonable because it still allows the library to accomplish those goals, and people can still call out to a thread to do it (which needs to have a good API anyway because there's no avoiding it for CPU-bound tasks). In many platforms, that's all that would happen under the hood anyway.
------
A couple of specific responses to the objections in the link (many are simply "that's no longer true" and I'm not going to try and list all those):
> Additionally, writing is not async. Is all your data sent when you write? Nope! ... The work around, of course, is to call writer.drain(). Why it doesn't do this automatically, who knows?
Sometimes you want to queue up data to be written to a socket and then do something else (in that coroutine), and sometimes to you want to queue up that data and wait until it actually gets written. The only time I've used asyncio in a real-world project, I've wanted the former; waiting would've been a disaster! The obvious way to allow this flexibility is to split up those two operations - and they really are separate operations - into two separate functions. This isn't a "workaround", it's clean design.
The alternative is to make it all in one function. That still technically allows both use cases: if you want to just queue up data then you can spawn a new coroutine for doing that with a queue/stream to send it data from other coroutines. But that's causing one of the two API uses to have way more code than the other, whereas having to choose between calling one or two functions is really not a big deal. It also means there's the runtime overhead of two queues, since the internal queue of data to write still really exists, and it's harder to arrange for smaller packets to be coallesced for writing... and so o...
Ordinary Linux file i/o for example cannot be made non-blocking and ignores O_NONBLOCK flag. See open(2).
Lookup the libuv threadpool for more details.
From the libuv docs [0]:
[0] http://docs.libuv.org/en/v1.x/design.html#file-i-o[1] https://blog.libtorrent.org/2012/10/asynchronous-disk-io/
well reading and writing can, opening can't. there was also a good conversation about that on HN: https://news.ycombinator.com/item?id=17654550
https://lwn.net/Articles/810414/
For cases like that discussion about opening lots of files over NFS this should allow it to be done completely async. The actual time to do it won't be better on a per file basis but you'd be able to parallelize them and do other work while waiting on the opens to complete.
This definitely doesn't sound right. Running strace, you need to specify -f in order to trace threads, otherwise only the main thread is traced. gdb does stop all threads on interrupts, but only by default for convenience, and I don't think it's atomic.
Because of this, LD_PRELOAD is not the easiest way to delay system calls, strace -e inject:delay is.
Often times the price you pay to send the job to the queue, yield for another coroutine, wait for IO task to finish, receive a notification back from the thread pool and resume is higher than doing the blocking call. Under high load it might have some small benefits but my hunch is that it's unlikely to buy you much.
Luckily, io_uring officially made it to the linux kernel few months ago and can solve these problems in a much more efficient manner. *BSD solved this problem long time ago, in 2002, when kqueue was introduced.
In my case it wasn't important to open log asynchronously, but it was critical to have async writes.
It turns out, depths of Python stdlib hide logging.handlers.QueueHandler and logging.handlers.QueueListener which allow to split log message handler into separate thread from log message emitters, making them communicate via synchronized queue.
Final solution is quite compact: https://github.com/Snawoot/postfix-mta-sts-resolver/blob/589...
Usage example: https://github.com/Snawoot/postfix-mta-sts-resolver/blob/589...
It is convenient that you can still use logging as usual, without any async obligations. You just have logging function which works in a fire-and-forget fashion.
> Asynchronous reads and writes would require all new APIs with different coloring. You’d need an aprint() to complement print(), and so on, each returning an awaitable to be awaited.
> This is one of the unfortunate downsides of async/await. I strongly prefer conventional, preemptive concurrency, but we don’t always have that luxury.
I was sceptical about Golang for a long time, but recently I've started practicing it. It's designed with concurrency in mind from very beginning and goroutines have advantages of both cooperative and preemptive multitasking. And also it doesn't have GIL, so your goroutines will use all available CPUs without any additional effort from your side.
From programmers point of view, coding with goroutines looks exactly like coding with use of pthreads (even simpler, in fact). So, there are no function "coloring", no need in sync/async library flavors and so on. But these "threads" also have cost close to cost of python coroutines. It sounds too good to be true, but I've rewritten bunch of my Python/asyncio code in Go and I can say it's true. For this reason (and many others I discovered later) I quit using Python and asyncio for network applications in favor of Go.
https://github.com/gabime/spdlog
I have used Asyncio for an API I built for https://www.abstractapi.com/ in order to be able to manage high load and avoid being locked by network latency.
It has been painful and I almost thought about migrating to Amazon Lambda to distribute the load but it feels like a bad use case for Lambda when you're actually only want to perform async stuff and not lock a full thread.
SCan't believe we still don't have a proper async lib for Python in 2020. Still have to test Curio but the whole thing makes me seriously reconsider which language I'll use for my next project. Async is something I need to handle more and more.