I feel like it's the best ATProto architecture explanation on the web.
I hope it will make people who argue that "there are no instances in ATProto" is wrong, understand that it is actually true and not just a naming debate.
Is ATProto federated, P2P, or a protocol flexible enough to build all kinds of things?
Is it opinionated about which type of system to build?
Skimming this page, it looks like ATProto is a schema for the ontology, and despite saying "P2P", it looks built around federation and concerned about scaling infrastructure (rather than scaling message broadcast).
I'm interested in a p2p social network with federated supernodes (p2p first class, federation second class, relays for devices that don't want to join the swarm) for people and agents. Could you easily build that with ATProto, or is it the wrong domain?
I'm interested in true, first-class P2P. Not a bolt-on afterthought. That's the problem with all of these federated services like Bluesky and Mastodon.
A true social protocol should work like email. Any machine on the network should be able to talk it to any other machine. Of course centralization nodes will appear, but they should be optional. Used as aids for user friendliness, anonymity, and firewalling yourself off.
Mastodon works like email. Users have their data hosted on instances and most big instances can communicate with most other big instances, and all instances blacklist a bunch of other instances for spam or political reasons. Just like e-mail.
ATProto is definitely not P2P and probably not federated. Every user has their own personal storage and gives apps like Bluesky permission to write to them.
They solve different problems and, in theory, you can use ATProto and ActivityPub (Mastodon/Threads/Peertube/etc) together
The article did not make me understand that at all.
> Since we're making a new app, we're going to want two things: an app server (which hosts our API & frontend) and a view server (which collects data from the network for us).
Yes, the user data is decoupled from the apps, but aren't both stored on some kind of instances? Also:
> Why are we listening to the event stream if we're the one making the write? Because we're not the only ones making writes! There are lots of user repos generating events, and lots of apps writing to them!
Who is hosting this event stream? Aren't we making some kind of federated network by deciding on our app server which event streams or which users/apps pushing updates to listen to?
>Yes, the user data is decoupled from the apps, but aren't both stored on some kind of instances?
There are two main kinds of "nodes" in atproto:
- Hosting aka "personal data servers" (PDS). This is dumb JSON hosting that you can query by HTTP or watch by Websocket. Super cheap to run. They don't talk to each other. You can have one per user, or one per many thousands.
- Apps. These are your normal webapps. (But they ingest data from everyone's hostings.) They also don't talk to each other.
So yes, there are "instances" in the sense of "boxes which run software" but the topology is completely different from Mastodon or such. Data flows from hosting into apps (and then apps write to hosting). There is no hosting-to-hosting or app-to-app connection. Hosting is app-agnostic, and apps are hosting-agnostic.
To make all of this practical, there are things in the middle that make the situation easier for app developers — either relays (which combine event stream from many hostings), or caches like Hubble[1] and Constellation[2] (which let you query the entire network in one request).
>Aren't we making some kind of federated network by deciding on our app server which event streams or which users/apps pushing updates to listen to?
Ideally you would listen to every relevant event from the entire network (and filter out every irrelevant one). It isn't hard today — you can either use an existing relay or run your own for ~$30/month or pool with someone. The discovery mechanism is that (1) a hosting can request any relay to crawl it, and (2) a relay can discover more hostings it hasn't crawled yet by following links — similar to how Google crawls the web.
Lol reminds me of "ITS SERVERLESS!" Which always cracks me up, as someone who does find serverless useful for very strategic scenarios, naming things as though they don't require a computer somewhere to function has been one of the weirdest trends in tech.
The article seems to forget about content addressable storage and signing key cryptography and how these tools might make their software more distributed.
Shameless plug, we built Chronon to solve a very similar problem. Chronon is now open source (apache 2) and used at Netflix, OpenAI, Airbnb, Stripe etc.
It is used to serve features/context to ML/AI models and rule engines, along with app views.
We are more scale and perf oriented as project given the use-cases and less decentralization oriented.
We also ship with a dataframe / prql like declarative api.
12 comments
[ 0.21 ms ] story [ 29.2 ms ] threadI hope it will make people who argue that "there are no instances in ATProto" is wrong, understand that it is actually true and not just a naming debate.
Is it opinionated about which type of system to build?
Skimming this page, it looks like ATProto is a schema for the ontology, and despite saying "P2P", it looks built around federation and concerned about scaling infrastructure (rather than scaling message broadcast).
I'm interested in a p2p social network with federated supernodes (p2p first class, federation second class, relays for devices that don't want to join the swarm) for people and agents. Could you easily build that with ATProto, or is it the wrong domain?
I'm interested in true, first-class P2P. Not a bolt-on afterthought. That's the problem with all of these federated services like Bluesky and Mastodon.
A true social protocol should work like email. Any machine on the network should be able to talk it to any other machine. Of course centralization nodes will appear, but they should be optional. Used as aids for user friendliness, anonymity, and firewalling yourself off.
They solve different problems and, in theory, you can use ATProto and ActivityPub (Mastodon/Threads/Peertube/etc) together
> Since we're making a new app, we're going to want two things: an app server (which hosts our API & frontend) and a view server (which collects data from the network for us).
Yes, the user data is decoupled from the apps, but aren't both stored on some kind of instances? Also:
> Why are we listening to the event stream if we're the one making the write? Because we're not the only ones making writes! There are lots of user repos generating events, and lots of apps writing to them!
Who is hosting this event stream? Aren't we making some kind of federated network by deciding on our app server which event streams or which users/apps pushing updates to listen to?
There are two main kinds of "nodes" in atproto:
- Hosting aka "personal data servers" (PDS). This is dumb JSON hosting that you can query by HTTP or watch by Websocket. Super cheap to run. They don't talk to each other. You can have one per user, or one per many thousands.
- Apps. These are your normal webapps. (But they ingest data from everyone's hostings.) They also don't talk to each other.
So yes, there are "instances" in the sense of "boxes which run software" but the topology is completely different from Mastodon or such. Data flows from hosting into apps (and then apps write to hosting). There is no hosting-to-hosting or app-to-app connection. Hosting is app-agnostic, and apps are hosting-agnostic.
To make all of this practical, there are things in the middle that make the situation easier for app developers — either relays (which combine event stream from many hostings), or caches like Hubble[1] and Constellation[2] (which let you query the entire network in one request).
>Aren't we making some kind of federated network by deciding on our app server which event streams or which users/apps pushing updates to listen to?
Ideally you would listen to every relevant event from the entire network (and filter out every irrelevant one). It isn't hard today — you can either use an existing relay or run your own for ~$30/month or pool with someone. The discovery mechanism is that (1) a hosting can request any relay to crawl it, and (2) a relay can discover more hostings it hasn't crawled yet by following links — similar to how Google crawls the web.
[1]: https://atproto.com/blog/introducing-hubble-a-public-mirror-...
[2]: https://constellation.microcosm.blue/
- https://overreacted.io/a-social-filesystem/
- https://overreacted.io/there-are-no-instances-in-atproto/
From the official site, I'd also recommend https://atproto.com/articles/atproto-ethos
It is used to serve features/context to ML/AI models and rule engines, along with app views.
We are more scale and perf oriented as project given the use-cases and less decentralization oriented.
We also ship with a dataframe / prql like declarative api.
[1] https://chronon.ai