Author seems to miss the the primary reason people build network apps with async: resource usage.
If you don't have co-routines, you need to have something else. That "something else" is most commonly threads, and each thread has non-trivial overhead, in scheduling and memory. A coroutine is much more memory efficient.
Removing GIL has nothing to do with that, coroutines are even worse re GIL problems. And of course, there are plenty of thread-based frameworks for Python, already - Flask and Django are most well known examples, but there are tons of others.
So my first question for author would be: "how is this different from Flask"?
Agreed. But I am not proposing a thread-per request model. The model will most likely be a fixed pool of workers, potentially combined with multiple processes, an M x N model.
I also agree that co-routines are cheaper than OS threads if the application itself is mostly waiting for network I/O. My question is what happens the moment the workload isn't purely I/O bound.
Right now, the solution seems to be an escape hatch where CPU-bound work is executed behind a thread-pool - see FastAPI or Starlette for example.
With free-threaded Python, I am interested in whether a framework can instead own that complexity: application code remains synchronous or async-partially (at the main thread).
> Removing GIL has nothing to do with that
It doesn't make network I/O faster, agreed. But, the more interesting consequence of removing the GIL is that with threads we can actually parallelise for CPU-bound Python code which earlier required multi-processor.
> How is it different from Flask or Django
This is probably the most important question. It's still in beta, so a lot of things can change. But simply put that Flask/Django lets one write sync code but do not provide the execution model I am describing. Concurrency is supplied at the server layer - gunicorn workers, etc.
The thesis that I am proposing is a framework runtime where concurrency and parallelism are first-class primitives. Now, that free-threaded makes threads capable of CPU parallelism can the framework own the concurrency and parallelism while application code stays async.
3 comments
[ 3.0 ms ] story [ 22.4 ms ] threadIf you don't have co-routines, you need to have something else. That "something else" is most commonly threads, and each thread has non-trivial overhead, in scheduling and memory. A coroutine is much more memory efficient.
Removing GIL has nothing to do with that, coroutines are even worse re GIL problems. And of course, there are plenty of thread-based frameworks for Python, already - Flask and Django are most well known examples, but there are tons of others.
So my first question for author would be: "how is this different from Flask"?
Agreed. But I am not proposing a thread-per request model. The model will most likely be a fixed pool of workers, potentially combined with multiple processes, an M x N model.
I also agree that co-routines are cheaper than OS threads if the application itself is mostly waiting for network I/O. My question is what happens the moment the workload isn't purely I/O bound.
Right now, the solution seems to be an escape hatch where CPU-bound work is executed behind a thread-pool - see FastAPI or Starlette for example.
With free-threaded Python, I am interested in whether a framework can instead own that complexity: application code remains synchronous or async-partially (at the main thread).
> Removing GIL has nothing to do with that
It doesn't make network I/O faster, agreed. But, the more interesting consequence of removing the GIL is that with threads we can actually parallelise for CPU-bound Python code which earlier required multi-processor.
> How is it different from Flask or Django
This is probably the most important question. It's still in beta, so a lot of things can change. But simply put that Flask/Django lets one write sync code but do not provide the execution model I am describing. Concurrency is supplied at the server layer - gunicorn workers, etc.
The thesis that I am proposing is a framework runtime where concurrency and parallelism are first-class primitives. Now, that free-threaded makes threads capable of CPU parallelism can the framework own the concurrency and parallelism while application code stays async.