Interesting. I didn’t know this body of work. I haven’t read the documentation other than the abstract.
If the protocol is about knowing what information to inject at what node in the network to achieve consensus, the protocol can (and will) be used to inject whatever truths the parties with knowledge on the protocol believe in. If there are enough parties with opposing beliefs, then the network cannot be gamed beyond the status quo.
How does this protocol “choose” the truth to propagate in the face of opposing truths?
What power does the protocol give to parties who know how to work the system?
Or rather what resilience does the protocol have against “inside traders”?
All good questions! We are discussing them regularly.
We take an adversarial approach to the protocol and constantly try to come up with ways it might be gamed. We want to design it in a way that achieves something like this: https://xkcd.com/810/
Some points regarding your questions:
The protocol will be fully open source so nobody can gain an advantage through information asymmetry.
We choose the best replies by measuring how much a reply "changes minds" (difference in voting behavior on a post given users have also voted on that reply). We adjust for biases using a technique called bridging-based ranking [1, 2]. The algorithm chooses the reply that had the highest impact on voting behavior after adjusting for biases.
We do realize that truth is a loaded term, so to elicit "truthful" posts and replies, we incentivize honesty with a technique called the Bayesian Truth Serum [3]. This will be linked with a reputation system that determines a user's impact.
If you spin up a bunch of bots, they will have low reputation in the beginning and therefore little impact. If you coordinate people (or bots) to vote manipulatively, their impact will be minimized by the bias correction because their votes are correlated. Our hope is that it would take considerable effort to build reputation on many accounts (by being honest and helpful over an extended period of time) and maintaining an attack over the lifecycle of a discussion would be infeasible because it works against the incentive structure and can at any point be exposed.
This is just some of our current thinking, if you can think of attack scenarios or weaknesses in our thinking, we'd be interested in your thoughts!
Very interesting! Seems like it needs a reasonably large number of events (votes?) for the statistics to be effective. How do you see that playing out in real systems?
9 comments
[ 6.1 ms ] story [ 43.8 ms ] threadWe have had some good discussions on HN on some of other posts about improving ranking algorithms in social platforms.
- Show HN: Quality News – Towards a fairer ranking algorithm for Hacker News: https://news.ycombinator.com/item?id=35183317
- Understanding Bridging-Based Ranking: https://news.ycombinator.com/item?id=38939660
- What Deserves our attention: https://news.ycombinator.com/item?id=36102608
We look forward to your feedback!
Is there any demo or illustration available of this system?
Thanks for the feedback!
We'll be launching something soon, if you're interested, you can keep up-to-date here: https://social-protocols.org/social-network/
The algorithm is explained here: https://social-protocols.org/global-brain/ This is a living document, we try to keep it up-to-date, but not all of it currently is.
We provide some demonstrations of how the algorithm evaluates certain scenarios here: https://social-protocols.org/GlobalBrain.jl
If the protocol is about knowing what information to inject at what node in the network to achieve consensus, the protocol can (and will) be used to inject whatever truths the parties with knowledge on the protocol believe in. If there are enough parties with opposing beliefs, then the network cannot be gamed beyond the status quo.
How does this protocol “choose” the truth to propagate in the face of opposing truths?
What power does the protocol give to parties who know how to work the system?
Or rather what resilience does the protocol have against “inside traders”?
We take an adversarial approach to the protocol and constantly try to come up with ways it might be gamed. We want to design it in a way that achieves something like this: https://xkcd.com/810/
Some points regarding your questions:
The protocol will be fully open source so nobody can gain an advantage through information asymmetry.
We choose the best replies by measuring how much a reply "changes minds" (difference in voting behavior on a post given users have also voted on that reply). We adjust for biases using a technique called bridging-based ranking [1, 2]. The algorithm chooses the reply that had the highest impact on voting behavior after adjusting for biases.
We do realize that truth is a loaded term, so to elicit "truthful" posts and replies, we incentivize honesty with a technique called the Bayesian Truth Serum [3]. This will be linked with a reputation system that determines a user's impact.
If you spin up a bunch of bots, they will have low reputation in the beginning and therefore little impact. If you coordinate people (or bots) to vote manipulatively, their impact will be minimized by the bias correction because their votes are correlated. Our hope is that it would take considerable effort to build reputation on many accounts (by being honest and helpful over an extended period of time) and maintaining an attack over the lifecycle of a discussion would be infeasible because it works against the incentive structure and can at any point be exposed.
This is just some of our current thinking, if you can think of attack scenarios or weaknesses in our thinking, we'd be interested in your thoughts!
[1] https://jonathanwarden.com/understanding-community-notes/
[2] https://vitalik.eth.limo/general/2023/08/16/communitynotes.h...
[3] https://nel.mit.edu/bayesian-truth-serum/