Coerce is often misused to imply encouragement or persuasion - but it specifically means to engage the efforts of others through violence or threat of violence. It's an ugly word.
There has been some friction between the two groups (tokio and async-std) unfortunately.
I'm not sure what the original source of friction was, but bad feelings seem to seep out when the projects are discussed in active Rust communities. I wonder how long the drama will go on. But at least both parties seem to be trying to be civil about it (and succeed).
There's nothing "standard" about async-std as of yet. The name is because it tries to build something that can be used just like std library facilities, except for the async support.
I'm not sure having multiple runtimes is such a problem. The Rust async IO model was explicitly designed to accommodate different runtimes. There are a lot of tradeoffs around how a runtime does multi-threading, for example, and embedded systems may also do their own thing. Tokio vs async-std might or might not make sense as a long term split, but I think it's too early to say.
This is a great thread! Lots of different perspectives on a difficult technical problem, plus appearances from many of my favourite Rust characters: we've got Ralf telling everyone that only abstract machines exist, boats turning up halfway through to contradict everyone, and Douman dropping by to offer the unhelpful C++ perspective.
The position of the async-std project is that people should use _futures-rs_ as the integration point and avoid integrating into tokio or async-std if possible. (both are IO optimised)
Especially for things like an actor system, I would recommend to not use either, but instead spawn a dedicated number of threads handling the actors. This allows you to be specifically optimised to the problem. bastion is a great example for that.
async-std ships a second library called "async-task", which implements just the task/future allocation component and allows you to quickly implement your own runtime.
Full disclosure: I'm one of async-std's maintainers.
I understand this, but it still doesn't answer the main point: are you working together with tokio maintainers to move to futures-rs as soon as possible and even extend the interface of futures-rs more if needed?
In my view the separate interfaces increase the long term damage that new libraries are creating every day, and it should be higher priority to resolve it together before implementing new features in either of the two libraries (async-std and tokio).
We don't maintain futures-rs. But for the reasons you outline above (stability), our interface is also effectively done.
The choice of not using all of futures for their interface is outspoken the strategy of tokio. We have encouraged them to prefer the futures layers, but we don't see the work happening.
So, to answer your question: we are not working with them, the place of collaboration is futures-rs.
I have heard that using the async-std APIs (but not the reactor and executor) causes background futures to be spawned to the async-std specific executor. I’m not sure of the veracity of this claim, can you comment?
Sooo... in standard profile, yes. `async_std::spawn` uses async-stds runtime. Also, our io interfaces (TcpListener, etc.) are effectively bound to our runtime (they need our mio instance in the back).
But this cannot made work generically in the current ecosystem, as there are no common abstractions here. We can't come up with those unless we have buy-in from other players.
For applications, that doesn't really matter: you pick one Listener type and you're done. Applications aren't very generic.
For libraries, that's harder, but there's strategies around that make it feasible to work with this situation (mainly: injecting the exact type to use into your library).
Finally: there's a flag in async-std called "runtime". If you turn it off, you can use async-std without that behaviour, at the cost of having to bring your own runtime.
Why should it be resolved ASAP? This mentality that there should be no more than one viable solution to address a need is what needs resolving. Rust would not even exist if enough people agreed with that sentiment.
I’m always excited to see people experimenting with different things.
I wonder if the author could comment on how the goals of this project might differ from those of Actix?
Many people are still new to async/await; did you consider add a runtime/executor in your readme example? I didn’t see the creation of a Runtime to drive the futures, that might be a nice thing to show.
The goal is to eventually have actors spread across multiple servers with automatic load balancing etc. I'm currently working on remote actors, with the goal for the API to be exactly the same whether you're talking to a local actor or a remote actor - it shouldn't matter to the user!
The runtime is just Tokio's runtime, actors are just tasks awaiting on messages (and handling them) from their Receiver!
Regarding data serialization, I'm currently using serde_json, however that's just because it's easy to work with for development/testing purposes though I do love the idea of using protobufs!
Node compatibility and I guess distribution overall are problems I'm extremely looking forward to tackling! :)
Cap'n Proto has good a good Rust library, but it's overall language support versus alternative formats is somewhat lacking, unfortunately.
That said, Cap'n Proto has a substantially better design (both in theory and implementation) than any of the alternatives IMO, so it's worth using anyway if your language supports it well. And especially so if you're using it "internally", so other language clients aren't as big of a deal.
Have you heard of Citybound? The author has implemented something similar for the simulation of the units in a multiplayer scenario. Cars/houses/people are all actors and can be simulated on any computer in the game and it all just magically works.
They have a talk somewhere where they explain it and do a live demo, pretty neat stuff :)
This sort of reminds me of an old agent based Java middleware called Voyager I've encountered sometime in the 90s. The rough idea of this middleware was that you could write Java class derived from a special prototype and the instance of said class can travel around various Voyager servers all over the world to perform tasks on local context and then come back with the answer. Could be used for the whole bunch of scenarios from map reduce to accessing some content that only available at particular place and not accessible by public API. It was neat. Not sure what did happen to this middleware later on.
Worth to mention that Bastion has been developed a lot and it has its own design(in simple terms SMP based executor) to leverage asynchronous programming (with fault-tolerant manner). It isn't dependent on any runtime. You can mix and use any runtime with it and supplies hierarchical reduction based concurrency.
We proudly take Erlang's runtime model and implemented it in Rust. Moreover, we are currently working on distributed carrier protocol to make Bastion form a cluster, exchange data and recover from partial failures in distributed workloads too.
Local failures are recovered and we have built-in lifecycle management.
As landing page states it is a runtime supplies actor-model like concurrency. If you are familiar with Erlang. Bastion is the reflection of the Erlang runtime in Rust. Far beyond an actor system implementation. It is a complete system with full asynchronous communication.
"...It supplies actor-model-like concurrency with a lightweight process implementation and utilizes all of the system resources efficiently guaranteeing of at-most-once message delivery."
It might be a single process, pile of workers, a bunch of http servers dispatching requests, various middleware layers connected with asynchronous boundaries. With all these built on top of crash and fault recovery.
What are your thoughts about the project that you sidelined-- Coerce? That is what brought people here in the first place and the project deserves recognition.
This is pretty cool. We ended up building our own version of something like this internally last year, but I could see us tossing our code in favour of this.
The big advantage to writing code this way is that you just don't need to write these actors thread-safe. They will only ever handle a single message at a time. For an async system it's really fast to get something up and running.
Read through the code just now.Thanks for the small but powerful project,I have tried to make something similar, but failed.
The lib is very close to what I want to do actor-based programming in Rust. It would be great if actor can send message to itself.
54 comments
[ 1.8 ms ] story [ 112 ms ] threadIt seems that there's still a huge split inside the community between async-std and tokio interfaces that should be resolved ASAP.
Is there some working group to help standardizing the interfaces and upgrade tokio based apps to the standard?
There has been some friction between the two groups (tokio and async-std) unfortunately.
I'm not sure what the original source of friction was, but bad feelings seem to seep out when the projects are discussed in active Rust communities. I wonder how long the drama will go on. But at least both parties seem to be trying to be civil about it (and succeed).
I'm not sure having multiple runtimes is such a problem. The Rust async IO model was explicitly designed to accommodate different runtimes. There are a lot of tradeoffs around how a runtime does multi-threading, for example, and embedded systems may also do their own thing. Tokio vs async-std might or might not make sense as a long term split, but I think it's too early to say.
There's a bug for it in Tokio, but it's low priority as far as I know.
https://github.com/tokio-rs/tokio/issues/1297
(See the last section in particular)
I understand that there are lots of hard technical issues to be resolved, it's just the visibility of this pull request wasn't that good.
Thanks for the answer!
Very interesting thread.
There is a huge amount of complaints just fundamentally about async-std's existence, though.
Especially for things like an actor system, I would recommend to not use either, but instead spawn a dedicated number of threads handling the actors. This allows you to be specifically optimised to the problem. bastion is a great example for that.
async-std ships a second library called "async-task", which implements just the task/future allocation component and allows you to quickly implement your own runtime.
Full disclosure: I'm one of async-std's maintainers.
In my view the separate interfaces increase the long term damage that new libraries are creating every day, and it should be higher priority to resolve it together before implementing new features in either of the two libraries (async-std and tokio).
The choice of not using all of futures for their interface is outspoken the strategy of tokio. We have encouraged them to prefer the futures layers, but we don't see the work happening.
So, to answer your question: we are not working with them, the place of collaboration is futures-rs.
But this cannot made work generically in the current ecosystem, as there are no common abstractions here. We can't come up with those unless we have buy-in from other players.
For applications, that doesn't really matter: you pick one Listener type and you're done. Applications aren't very generic.
For libraries, that's harder, but there's strategies around that make it feasible to work with this situation (mainly: injecting the exact type to use into your library).
Finally: there's a flag in async-std called "runtime". If you turn it off, you can use async-std without that behaviour, at the cost of having to bring your own runtime.
I wonder if the author could comment on how the goals of this project might differ from those of Actix?
Many people are still new to async/await; did you consider add a runtime/executor in your readme example? I didn’t see the creation of a Runtime to drive the futures, that might be a nice thing to show.
The goal is to eventually have actors spread across multiple servers with automatic load balancing etc. I'm currently working on remote actors, with the goal for the API to be exactly the same whether you're talking to a local actor or a remote actor - it shouldn't matter to the user!
The runtime is just Tokio's runtime, actors are just tasks awaiting on messages (and handling them) from their Receiver!
Cheers!
And thanks for the answer!
Node compatibility and I guess distribution overall are problems I'm extremely looking forward to tackling! :)
or FlatBuffers!
That said, Cap'n Proto has a substantially better design (both in theory and implementation) than any of the alternatives IMO, so it's worth using anyway if your language supports it well. And especially so if you're using it "internally", so other language clients aren't as big of a deal.
They have a talk somewhere where they explain it and do a live demo, pretty neat stuff :)
https://aeplay.org/citybound
Networks are much less reliable or predictable than a PCI or SATA bus. You have to honesty handle timeouts, unavailability, retries at the very least.
Hiding this from the user is usually a bad idea. An intelligent handling layer does help, but it cannot e.g. entirely mask unavailability.
So the common interface would have to be the handling-it-all interface, likely as a number if separate traits.
We proudly take Erlang's runtime model and implemented it in Rust. Moreover, we are currently working on distributed carrier protocol to make Bastion form a cluster, exchange data and recover from partial failures in distributed workloads too.
Local failures are recovered and we have built-in lifecycle management.
Maybe after these words you might want to take a look: https://github.com/bastion-rs/bastion
You can write a fully fault-tolerant program with our macro mechanisms too: https://github.com/bastion-rs/fort
Here our distributed protocol repository: https://github.com/bastion-rs/artillery
And our landing page: https://bastion.rs/
But.. what is this?
I read the landing page and browsed a couple examples and I have no clue what I would use it for.
Would it be a pile of distributed workers handling jobs? Sort of like pushing a job into a job queue and having a bunch of workers consume jobs?
If that's the case, what are some reasons I might choose this Bastion model over a more traditional JobQueue <-> Worker model?
Thanks to any replies. I feel I should be interested, but I'm confused :)
"...It supplies actor-model-like concurrency with a lightweight process implementation and utilizes all of the system resources efficiently guaranteeing of at-most-once message delivery."
It might be a single process, pile of workers, a bunch of http servers dispatching requests, various middleware layers connected with asynchronous boundaries. With all these built on top of crash and fault recovery.
For more information what it provides please take a look to my latest presentation at Rust Berlin: https://slides.com/vertexclique/crash-course-with-bastion#/
Maybe I'll just write a proc macro over this lib instead.
* https://github.com/insanitybit/derive_aktor
edit: Is this not on crates-io?
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3418003
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3459566
The big advantage to writing code this way is that you just don't need to write these actors thread-safe. They will only ever handle a single message at a time. For an async system it's really fast to get something up and running.