There's ucan too, which fills some similar-ish roles. It's definitely not quite as comprehensive & featureful as ocapn, but worth mentioning. https://ucan.xyz/
That was my first question, too. From the fine article:
> We've drawn from and built upon the contributions of many others (the E programming language, Agoric, and Cap'n Proto to name a few) and is actively being shaped by input from many developers of those projects.
The feature list includes distributed garbage collection, from what I understand that's out of scope of capnproto rpc.
Ian "zenhack" Denhardt, a major Cap'n Proto contributor, was working closely with the ocapn effort to try to ensure some level of compatibility. Tragically, he passed away in July, leaving that effort (and so much else) in limbo. Unfortunately I have not had the bandwidth to keep up on ocapn, so I can't really comment much myself.
------------
I'll say a bit about "distributed garbage collection". I'm not caught up on what, exactly, the ocapn people mean here. E always advertised support for "distributed acyclic garbage collection", which in practice meant reference counting. Cap'n Proto also implements reference counting in exactly the same way. But I personally wouldn't call it "garbage collection" if it can't collect cycles. Moreover, Cap'n Proto applications commonly expect deterministic destruction when references are dropped -- they wouldn't play well with GCs that might arbitrarily delay collection.
My personal take is that distributed GC is simply not feasible. Garbage collection algorithms heavily lean on heuristics and amortization to achieve any kind of performance. In particular, most GCs will only actually execute when they detect "memory pressure", i.e., that the system is in need of freeing up some memory. If the system doesn't need memory, why waste time finding some? However, in a distributed system, it can easily be the case that the local system has plenty of memory, but is holding on to a reference to a remote object living on a system that does need memory. The reference may no longer be reachable. But, the local GC isn't going to run because it doesn't perceive a need. Meanwhile, the remote system has no way to know that the object it is hosting could be collected.
To solve problems like this, you cannot just have each machine running its own local GC, you must have a single distributed GC that can respond to memory pressure anywhere by triggering sweeps of the entire distributed object graph. This seems extremely difficult to design, far more so if the systems involved do not trust each other.
Maybe it's possible, but certainly there's quite a lot of research needed, and last I knew the ocapn people hadn't actually demonstrated anything that works. But my personal take is that GC in general is a terrible fit for systems programming, because it is so bad at handling any kind of external resource, which is what systems programming is all about. Cap'n Proto is unlikely to support anything more than reference counting.
Yeah I always thought of distributed GC as a failed research idea from the 80's and 90's, kind of like transparent remote objects (as opposed to messages)
I'd be interested to hear otherwise. And some details about a deployment
As an EC alum I can tell you that in the late 90s there was a lot of work on E to get both transparent remote objects and distributed garbage collection working. In fact, one of the first patents I remember us having approved was Arturo's garbage collector (https://patents.google.com/patent/US5991779A/en, yes same guy as the Meta whistleblower from a week or so ago) but it was still a bit wonky in practice from what I remember.
If you look at https://ocapn.org/ (linked within the original submission,
The group is comprised of:
* Agoric
* The Spritely Institute
* Cap'n Proto
* Many other independent experts and interested parties
from Cap'n Proto I thought that zenhack was the primary participant in ocapn, but see some other familiar names from Cap'n Proto community. Essentially it's trying to create a common protocol for these diverse implementations which all have similarities because they happen to share a common heritage. At least that is my understanding.
The expectation I think that everyone is currently operating under is that Spritely and Agoric will try to evolve into pretty directly speaking OCapN, and Cap'n Proto will ideally be able to communicate with OCapN via a bridge of some flavor, as Cap'n Proto is too widely established to shift its behavior much towards the direction Spritely and Agoric are going.
Unfortunately with Ian passing away, the Cap'n Proto side isn't seeing a ton of attention, though several involved parties are familiar with Cap'n Proto and reference it in discussions with regards to maintaining similarities and hopefully bridge-ability.
CapTP[1], which OCapN lists at the core of the project, has a presentation giving an overview here[2]. Think I'll need a bit more to fully grasp it but gave some insight.
18 comments
[ 2.5 ms ] story [ 56.5 ms ] thread> We've drawn from and built upon the contributions of many others (the E programming language, Agoric, and Cap'n Proto to name a few) and is actively being shaped by input from many developers of those projects.
The feature list includes distributed garbage collection, from what I understand that's out of scope of capnproto rpc.
Maybe Kenton is around and can add more details?
for example timeouts in ibc, client cannot timeout if server is dead for example. ibc goes hand in hand with identity and sagas.
------------
I'll say a bit about "distributed garbage collection". I'm not caught up on what, exactly, the ocapn people mean here. E always advertised support for "distributed acyclic garbage collection", which in practice meant reference counting. Cap'n Proto also implements reference counting in exactly the same way. But I personally wouldn't call it "garbage collection" if it can't collect cycles. Moreover, Cap'n Proto applications commonly expect deterministic destruction when references are dropped -- they wouldn't play well with GCs that might arbitrarily delay collection.
My personal take is that distributed GC is simply not feasible. Garbage collection algorithms heavily lean on heuristics and amortization to achieve any kind of performance. In particular, most GCs will only actually execute when they detect "memory pressure", i.e., that the system is in need of freeing up some memory. If the system doesn't need memory, why waste time finding some? However, in a distributed system, it can easily be the case that the local system has plenty of memory, but is holding on to a reference to a remote object living on a system that does need memory. The reference may no longer be reachable. But, the local GC isn't going to run because it doesn't perceive a need. Meanwhile, the remote system has no way to know that the object it is hosting could be collected.
To solve problems like this, you cannot just have each machine running its own local GC, you must have a single distributed GC that can respond to memory pressure anywhere by triggering sweeps of the entire distributed object graph. This seems extremely difficult to design, far more so if the systems involved do not trust each other.
Maybe it's possible, but certainly there's quite a lot of research needed, and last I knew the ocapn people hadn't actually demonstrated anything that works. But my personal take is that GC in general is a terrible fit for systems programming, because it is so bad at handling any kind of external resource, which is what systems programming is all about. Cap'n Proto is unlikely to support anything more than reference counting.
I'd be interested to hear otherwise. And some details about a deployment
But it feels like you're saying it was tried and didn't quite pan out? That's what my impression was
https://github.com/Agoric/agoric-sdk/issues/1827#issuecommen...
* Agoric
* The Spritely Institute
* Cap'n Proto
* Many other independent experts and interested parties
from Cap'n Proto I thought that zenhack was the primary participant in ocapn, but see some other familiar names from Cap'n Proto community. Essentially it's trying to create a common protocol for these diverse implementations which all have similarities because they happen to share a common heritage. At least that is my understanding.
Unfortunately with Ian passing away, the Cap'n Proto side isn't seeing a ton of attention, though several involved parties are familiar with Cap'n Proto and reference it in discussions with regards to maintaining similarities and hopefully bridge-ability.
[1]: http://erights.org/elib/distrib/captp/index.html
[2]: http://www.erights.org/talks/captp4omg/