we cannot live in a society where I must demonstrate to another human that I'm human with a piece of paper just because the other human is a bureaucrat with a computer.
Their distinction follows dictionaries that I've looked at, but common usage is clearly diverging from that.
It's unclear to me at what point we should stop saying that lots of people are using words incorrectly, and start saying that lots of dictionaries are sticking with outmoded definitions, but looking at the past, transitions certainly occur sometimes.
There are lots of people that have this view that there is "objectively correct" language, and that you can find it in grammar books and dictionaries. Any linguist worth their salt knows that is a completely outdated and classist way of studying language. Dictionaries are supposed to be updated acording to how people use language, not people be "corrected" to try to follow the dictionary. While language learners speak a language "wrong", any group of native language speakers has rights of ownership of their language as any other group, and as long as they are communicating in a way that they can understand each other, they are never wrong.
As an avocational linguist I am sympathetic to all of that, but effective communication requires a large degree of mutual understanding of words, so it's not like anything goes, either; plus most people have not studied linguistics and do believe in "correct language" (prescriptive) and do not hesitate to correct people who use "incorrect language" -- so it's a bit complicated culturally, if not from the stance of descriptive linguistics.
In this particular case, there's no danger of misunderstanding, so I agree that this was just a nitpick about dictionary definitions versus a fairly common usage.
We consider many animals to be "sentient" -- we recognize that they are to some degree conscious, in this case meaning that they have the capability to sense, perceive their world, and have some kind of emotion about the things they perceive
"Sapience" on the other hand, is essentially "human-level intelligence, consciousness, and self-awareness" (from homo sapiens)
IMO, the meat of this paper is in section 4.3 and 4.4.
And I cannot say for sure, but the formal proof of 4.4 basically summarizes the same points pointed out in 4.3.
Most of these are not inherently mathematical problems but a social one.
> Verifying sentience is a fuzzy concept. While they can be bound together momentarily as we see in [66 ], the binding is very easily decoupled.The verified user might choose to sell off their uniqueness identifier at time period 𝑡 + 1 if the verification which binds sentience with uniqueness ends at 𝑡.
Basically, people can sell identities
----
What really concerns me though, is how much and how often this paper discusses DRM, or in their own words, a "trust anchor"
> With the assumed threat model in our case, the lack of inherent trust in the user only compounds the unreliability of the model without any trust anchor.
> Assuming a proof of location is for a mobile device, rather than a particular human being, then associating the proof of uniqueness obtained under such a condition, i.e., without the involvement of a trust anchor, is unreliable.
I know that the authors aren't directly calling for more centralized trust. But given recent development at Google, we all know how the readers would think
> > Verifying sentience is a fuzzy concept. While they can be bound together momentarily as we see in [66 ], the binding is very easily decoupled.The verified user might choose to sell off their uniqueness identifier at time period 𝑡 + 1 if the verification which binds sentience with uniqueness ends at 𝑡.
> Basically, people can sell identities
I can see why Sam Altman believes iris scans are the future, it's definitely much more cumbersome to 'sell off' your iris. Especially if it needs to be rescanned on a daily basis or sooner.
But Worldcoin isn't doing that. They scan your eyes once and issue you a private key that you can then sell. Maybe one day they can give away scanners CueCat-style but I haven't seen any discussion of that.
> Verifying sentience is a fuzzy concept. While they can be bound together momentarily as we see in [66 ], the binding is very easily decoupled.The verified user might choose to sell off their uniqueness identifier at time period 𝑡 + 1 if the verification which binds sentience with uniqueness ends at 𝑡.
I got half way (up to the proof). After that they seemed to be describing the regular biometric identity program any country would set up. (Aadhar for India is the easiest comparison).
I felt that this was suggesting that Meta/Google/Platforms, need to start creating Account Recovery Offices.
If your account is compromised, then recovery means you have to prove you are the legitimate owner of the account. You are the real person, not the adversary.
Right now, this must be happening online, with some mix of captcha/image/ ID / Device verification.
But one part of this trilemma is that you need location as well to determine uniqueness.
Maybe you can get it from IP, but the stronger form would be in person verification.
——-
This is so big brother. Prove to me you are who you say to be.
> Trolls, bots, and sybils distort online discourse and compromise the security of networked platforms.
In some sense I think the authors' hypothesis is a good thing, ie that you can never fully verify someone online. It prevents wholesale algorithmic management of people, which is really what governments and companies would like to do, and forced some level of human contact or at least intervention. I expect it's inevitable that they'll find a way to offload the problem onto the citizen, for the most part they already have, but I'm personally glad it's impossible to assign me some kind of infallible identifier that will let me be The Castle style abused remotely and without recourse.
I think a lot of people try to do identity things without understanding the fundamental nature of the problem they're attacking.
For instance: I'm really really worried that governments are going to default into an understanding of digital identity that involves ownership of an email address and mobile phone rather than ability to sign a document.
Or: I'm really annoyed that software services and web apps have clauses like "you can't use scripts or automation software to access our API" when a browser is just that. And they should really be enforcing rate limits and punishing abusive behavior whether the user clicked a button in a browser or a script did.
These type of things are not rooted in a fundamental understanding of identity, they're sloppy stop-gaps. Despite all its faults, this is one of the reasons I'm super excited about WebAuthN. At least is make common the idea that an identity is a cryptographic secret and not a "possession of an email and phone". We really really need to dig out of this "email address identifies you" hole.
Anyway it's exciting to see people discuss the topic more formally. It gives me hope that we can ultimately get to a better understanding of digital identity and not be trying to solve impossible problems by chasing an impossibly perfect solution that verifies all 3 tenets that actually doesn't exist and making a big mess of things because nobody stopped to ask or understand the scope of what we should be trying to do.
Ultimately identity should be empowering not oppressive. And right now it feels more like services oppress people into all sorts of weird requirements like having an email, getting a phone verification code, running software on a device that has an integrity attestation framework, etc. rather than trust them and punish bad behavior.
I want my government (and web services, but especially my government) to trust me and punish bad behavior, not treat me like an untrusted bot that needs to be managed and continuously verified.
> an understanding of digital identity that involves ownership of an email address and mobile phone rather than ability to sign a document.
Buying a SIM card in a great many countries today already requires showing state-approved ID and then signing a form that the shop clerk prints out. So, ownership of a mobile phone number does mean being able to sign a document. Are you concerned about SIM-jacking? I admit, I find the thrust of your post difficult to follow.
It absolutely does not, since a sim is just an object which can be acquired any number of ways than the intended one. They can also be faked and not even possessed at all, since all the server sees is some data. The server (the phone companies hardware) is not a notary public watching you sign something after verifying that your ID matches your person.
This is exactly the grossly naive assumption they are talking about people treating as though it had any substance at all when it has practically none.
The simple solution is democratization. If average Joe can’t have privacy, then President Joe shouldn’t either. And average Joe should have as much access to President Joe’s data as President Joe does to average Joe’s.
>Ultimately identity should be empowering not oppressive.
I agree completely. Are there any examples of that though, where it is empowering?
Talking about government (or any bureaucracy) there is no chance of empowerment, at least from the administrative arm. Software is written by lowest bidders for the convenience of administrators, to help them treat people like cattle. This can only change when we complain to real politicians who could potentially advocate for empowerment. As long as bureaucrats are in charge, it only gets worse.
Without identity posession is impossible - i.e. tying your name to contracts is impossible, thus owning objects or property becomes difficult. This is between individuals, but enforced by governement.
It becomes oppressive when you do not consent to giving your identity but it is taken and possibly used against you i.e. logging what you do online, having cctv video taken of you without consent - all things that can be used against you without explicit consent
The acceptance of a higher power that enforces contracts is not empowering. It is rational but still oppressive even if one argues that it is chosen rather than merely accepted.
The default state of being is that nothing is enforced besides what you can personally enforce. As a regular person this means you have no power, no property, no rights, nothing in the face of someone with a bigger stick.
The rule of law empowers all but the absolute strongest to also own property and gather resources while protecting them from arbitrary violence.
We merely replaced bigger sticks with bigger balances because physical strength is not the end all for our society. There is nothing empowering in the arrangement. Supposedly the purpose is to limit power to ensure a more level playing field, but it does not work that way in the end anyway.
Take science for example. Identity is empowering because you can say, "look, I synthetized unobtainium, my name is on the paper", which shows your value as a scientist and so you will be respected and trusted with important experiments. It is also oppressive because if you don't have your name on a paper, you will be considered a worthless scientist.
If you have something attached to your name that gives you some power, we can also say that not having that something takes that power away from you. Or its counterpart: if something attached to your name causes you to be oppressed, not having that something empowers you.
> At least is make common the idea that an identity is a cryptographic secret and not a "possession of an email and phone". We really really need to dig out of this "email address identifies you" hole.
A cryptographic secret is not an identity; at best, it is a way to authenticate access to an identity. A "secret" can be lost or stolen; it is at least something I am not stuck with for life--unlike a biometric--but only if I have a way to change it, which implies that the identity is NOT the cryptographic secret.
I'd actually argue that an email address is much more similar to the concept of an identity--as long as we accept an expensive definition that doesn't requiring an identity to be "a human" or anything silly like that, such that a faceless "role" can also serve as an identity--than a cryptographic secret.
To be clear: I agree that it sucks that so many people want to make authentication happen via credentials you can't truly possess (such as an email address, as at the end of the day you can't even truly possess a domain name) or control (a problematic category which to me includes biometrics); but trying to fix that is unrelated to the concept of "identity".
Half the internet thinks troll means "person who disagrees with me or (dis)likes thing i (dis)like" - I'm distrustful of people who paint them as a big problem on the internet.
Is this how the western Social Credit system begins?
Trolls, as in troll farms, astroturfing, organized influence campaigns, etc. are absolutely a serious problem for any society that pretends to care about democracy.
Especially in the LLM era, where the marginal cost of adding another artificial "voice" approaches $0.
If the troll farms are responsible for online attacks on our civil liberties, why would they use themselves, troll farms, as the justification? I rather think the attacks on our freedom are being made by real people who are misguided about the tradeoffs between letting people speak when they are wrong, and making it easy for officials covering up misconduct.
What about Apple's FaceID? Secure, private, decentralized, and and verifies sentience (well atleast intactness of your physical head), location and identity.
The location element of the trilemma doesn't seem as important as the other two, and is a violation of privacy. Let it be hard to discover.
A social site that discovers everyone's location, and reminds them it knows where they are, would probably drastically cut down on the trolling and abuse, but at what cost.
The philosophy of identity is quite interesting to me. I've been thinking about it for a long time.
I like the idea of proof of personhood, and proof of location. These are both concepts that I have explored extensively. I looked at retina scans, and found it they're both not unique enough, and not secure enough to act as a basis for a currency.
My proof of location was based on the 500 mile email story. Many millions of round trips between two devices could allow someone to prove on a blockchain that two secret keys were stored arbitrarily close to one another for some amount of time.
The paper mentions key signing parties which I believe must be part of any web of trust that proves personhood. Proving uniqueness is more difficult. There's nothing that would stop me from having several different identities verified at different key signing parties, then using those to generate even more.
Using genealogy seems to be the only way to truly verify a unique identity if the system becomes widespread enough that parents often verify their children at birth. Then the age of the identity matches the age of the individual.
Farther into the future, the possibility of creating multiple copies of myself makes it much more difficult to tie an identity to a particular group of cells. There's nothing truly unique about a copy, especially if we have the ability to change our DNA.
Philosophically, I started to wonder if there's anything particularly special about any arrangement of cells. Maybe we should all just do our best, share resources, and create a society where abundance makes keeping track of all this stuff unnecessary.
> Philosophically, I started to wonder if there's anything particularly special about any arrangement of cells. Maybe we should all just do our best, share resources, and create a society where abundance makes keeping track of all this stuff unnecessary.
We are an arrangement of cells betrayed by the human character. If all we wanted was to live leisurely this could have been easily achieved after the industrial revolution. But it's our nature to be ambitious, to achieve things, to want more, to compete, to be greedy and 1up each other. Most people are never satisfied regardless of how big their house is or what they do for a living. So I don't think an idle society of abundance can exist as long as people are competing for the largest piece of the pie.
Or perhaps its a small minority of people who have access to vast wealth and control through a combination of luck, ruthlessness, and glitches in our social, economic, and political systems, who then maintain their power by refusing to makes those systems more equitable. There are plenty of studies of Indigenous peoples whose societies are much more equitable than ours, which shows that the problems we face now are neither inevitable, nor solely a result of 'human nature', if such a universal concept even exists.
Or perhaps it's the market-based economy, supply and demand - a feedback loop that, by its very nature, optimizes away any average surplus a society accumulates, in the same way a DC filter will remove energy from a signal until it averages to zero.
I'm increasingly convinced this is a direct consequence, a flip side of our economic system - i.e. the very fact that people can freely compete means prices will go down to near production costs, as competition minimizes everyone's margins - but at a larger scale, it means the market will grow and change to consume any surplus, which, since nothing here is mathematically perfectly uniform, necessitates the existence of poverty and affluence - income inequality that can have arbitrary amplitude, but averages to zero.
> There are plenty of studies of Indigenous peoples whose societies are much more equitable than ours
Please name three. As far as I've read, all the various indigenous tribes past and present tend to have the same problems as every other group of people, including modern civilization; they're just earlier on the problem complexity/sophistication ladder. That, and they tend to have pretty horrible alternatives to what we've learned to solve via taxation and the rule of law.
I've started to suspect this kind of Darwinian individualism - which probably evolves in a lot of places - is guaranteed to lead to extinction, and is likely a critical part of the Great Filter.
When a species evolves to the point it has planet-shaping capabilities it needs to be able to think about consequences on a planetary scale - not just in terms of short term individual benefits.
Competitive individualism completely fails at scale.
It's possible there are other Darwinian solutions - maybe life that evolves as a single cooperative colony organism - which avoid this trap.
The corollary is that if we ever meet aliens, they're not going to have much in common with us.
54 comments
[ 6.1 ms ] story [ 115 ms ] threadthis is an 'online only' problem
It's unclear to me at what point we should stop saying that lots of people are using words incorrectly, and start saying that lots of dictionaries are sticking with outmoded definitions, but looking at the past, transitions certainly occur sometimes.
In this particular case, there's no danger of misunderstanding, so I agree that this was just a nitpick about dictionary definitions versus a fairly common usage.
"Sapience" on the other hand, is essentially "human-level intelligence, consciousness, and self-awareness" (from homo sapiens)
https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_...
And I cannot say for sure, but the formal proof of 4.4 basically summarizes the same points pointed out in 4.3.
Most of these are not inherently mathematical problems but a social one.
> Verifying sentience is a fuzzy concept. While they can be bound together momentarily as we see in [66 ], the binding is very easily decoupled.The verified user might choose to sell off their uniqueness identifier at time period 𝑡 + 1 if the verification which binds sentience with uniqueness ends at 𝑡.
Basically, people can sell identities
----
What really concerns me though, is how much and how often this paper discusses DRM, or in their own words, a "trust anchor"
> With the assumed threat model in our case, the lack of inherent trust in the user only compounds the unreliability of the model without any trust anchor.
> Assuming a proof of location is for a mobile device, rather than a particular human being, then associating the proof of uniqueness obtained under such a condition, i.e., without the involvement of a trust anchor, is unreliable.
I know that the authors aren't directly calling for more centralized trust. But given recent development at Google, we all know how the readers would think
> Basically, people can sell identities
I can see why Sam Altman believes iris scans are the future, it's definitely much more cumbersome to 'sell off' your iris. Especially if it needs to be rescanned on a daily basis or sooner.
It's a use-after-free for real life!
Calm down Rust people...nothing we can do here.
I felt that this was suggesting that Meta/Google/Platforms, need to start creating Account Recovery Offices.
If your account is compromised, then recovery means you have to prove you are the legitimate owner of the account. You are the real person, not the adversary.
Right now, this must be happening online, with some mix of captcha/image/ ID / Device verification.
But one part of this trilemma is that you need location as well to determine uniqueness.
Maybe you can get it from IP, but the stronger form would be in person verification.
——-
This is so big brother. Prove to me you are who you say to be.
In some sense I think the authors' hypothesis is a good thing, ie that you can never fully verify someone online. It prevents wholesale algorithmic management of people, which is really what governments and companies would like to do, and forced some level of human contact or at least intervention. I expect it's inevitable that they'll find a way to offload the problem onto the citizen, for the most part they already have, but I'm personally glad it's impossible to assign me some kind of infallible identifier that will let me be The Castle style abused remotely and without recourse.
For instance: I'm really really worried that governments are going to default into an understanding of digital identity that involves ownership of an email address and mobile phone rather than ability to sign a document.
Or: I'm really annoyed that software services and web apps have clauses like "you can't use scripts or automation software to access our API" when a browser is just that. And they should really be enforcing rate limits and punishing abusive behavior whether the user clicked a button in a browser or a script did.
These type of things are not rooted in a fundamental understanding of identity, they're sloppy stop-gaps. Despite all its faults, this is one of the reasons I'm super excited about WebAuthN. At least is make common the idea that an identity is a cryptographic secret and not a "possession of an email and phone". We really really need to dig out of this "email address identifies you" hole.
Anyway it's exciting to see people discuss the topic more formally. It gives me hope that we can ultimately get to a better understanding of digital identity and not be trying to solve impossible problems by chasing an impossibly perfect solution that verifies all 3 tenets that actually doesn't exist and making a big mess of things because nobody stopped to ask or understand the scope of what we should be trying to do.
Ultimately identity should be empowering not oppressive. And right now it feels more like services oppress people into all sorts of weird requirements like having an email, getting a phone verification code, running software on a device that has an integrity attestation framework, etc. rather than trust them and punish bad behavior.
I want my government (and web services, but especially my government) to trust me and punish bad behavior, not treat me like an untrusted bot that needs to be managed and continuously verified.
Buying a SIM card in a great many countries today already requires showing state-approved ID and then signing a form that the shop clerk prints out. So, ownership of a mobile phone number does mean being able to sign a document. Are you concerned about SIM-jacking? I admit, I find the thrust of your post difficult to follow.
This is exactly the grossly naive assumption they are talking about people treating as though it had any substance at all when it has practically none.
I agree completely. Are there any examples of that though, where it is empowering?
Talking about government (or any bureaucracy) there is no chance of empowerment, at least from the administrative arm. Software is written by lowest bidders for the convenience of administrators, to help them treat people like cattle. This can only change when we complain to real politicians who could potentially advocate for empowerment. As long as bureaucrats are in charge, it only gets worse.
It becomes oppressive when you do not consent to giving your identity but it is taken and possibly used against you i.e. logging what you do online, having cctv video taken of you without consent - all things that can be used against you without explicit consent
The default state of being is that nothing is enforced besides what you can personally enforce. As a regular person this means you have no power, no property, no rights, nothing in the face of someone with a bigger stick.
The rule of law empowers all but the absolute strongest to also own property and gather resources while protecting them from arbitrary violence.
Take science for example. Identity is empowering because you can say, "look, I synthetized unobtainium, my name is on the paper", which shows your value as a scientist and so you will be respected and trusted with important experiments. It is also oppressive because if you don't have your name on a paper, you will be considered a worthless scientist.
If you have something attached to your name that gives you some power, we can also say that not having that something takes that power away from you. Or its counterpart: if something attached to your name causes you to be oppressed, not having that something empowers you.
In most airports of the world, a US passport is quite an empowering identity.
A cryptographic secret is not an identity; at best, it is a way to authenticate access to an identity. A "secret" can be lost or stolen; it is at least something I am not stuck with for life--unlike a biometric--but only if I have a way to change it, which implies that the identity is NOT the cryptographic secret.
I'd actually argue that an email address is much more similar to the concept of an identity--as long as we accept an expensive definition that doesn't requiring an identity to be "a human" or anything silly like that, such that a faceless "role" can also serve as an identity--than a cryptographic secret.
To be clear: I agree that it sucks that so many people want to make authentication happen via credentials you can't truly possess (such as an email address, as at the end of the day you can't even truly possess a domain name) or control (a problematic category which to me includes biometrics); but trying to fix that is unrelated to the concept of "identity".
- identity is who you are (relationships, memories, secrets)
- identity is recognition and discernment (others need to confidently assess that you are you, and not a counterfeit)
- identity is correlation management (whether managed by others or ourselves, we have a right to privacy and selective disclosure)
Is this how the western Social Credit system begins?
Especially in the LLM era, where the marginal cost of adding another artificial "voice" approaches $0.
They're going to be pushing for WEI/Attestation/Requiring easily doxxable accounts using this a bogeyman, aren't they?
> They're going to be pushing for WEI/Attestation/Requiring easily doxxable accounts using this a bogeyman, aren't they?
It's not a bogeyman. It's a real, current, serious problem.
I know language changes over time. This was clearly a change for the worse.
A social site that discovers everyone's location, and reminds them it knows where they are, would probably drastically cut down on the trolling and abuse, but at what cost.
I like the idea of proof of personhood, and proof of location. These are both concepts that I have explored extensively. I looked at retina scans, and found it they're both not unique enough, and not secure enough to act as a basis for a currency.
My proof of location was based on the 500 mile email story. Many millions of round trips between two devices could allow someone to prove on a blockchain that two secret keys were stored arbitrarily close to one another for some amount of time.
The paper mentions key signing parties which I believe must be part of any web of trust that proves personhood. Proving uniqueness is more difficult. There's nothing that would stop me from having several different identities verified at different key signing parties, then using those to generate even more.
Using genealogy seems to be the only way to truly verify a unique identity if the system becomes widespread enough that parents often verify their children at birth. Then the age of the identity matches the age of the individual.
Farther into the future, the possibility of creating multiple copies of myself makes it much more difficult to tie an identity to a particular group of cells. There's nothing truly unique about a copy, especially if we have the ability to change our DNA.
Philosophically, I started to wonder if there's anything particularly special about any arrangement of cells. Maybe we should all just do our best, share resources, and create a society where abundance makes keeping track of all this stuff unnecessary.
We are an arrangement of cells betrayed by the human character. If all we wanted was to live leisurely this could have been easily achieved after the industrial revolution. But it's our nature to be ambitious, to achieve things, to want more, to compete, to be greedy and 1up each other. Most people are never satisfied regardless of how big their house is or what they do for a living. So I don't think an idle society of abundance can exist as long as people are competing for the largest piece of the pie.
I'm increasingly convinced this is a direct consequence, a flip side of our economic system - i.e. the very fact that people can freely compete means prices will go down to near production costs, as competition minimizes everyone's margins - but at a larger scale, it means the market will grow and change to consume any surplus, which, since nothing here is mathematically perfectly uniform, necessitates the existence of poverty and affluence - income inequality that can have arbitrary amplitude, but averages to zero.
> There are plenty of studies of Indigenous peoples whose societies are much more equitable than ours
Please name three. As far as I've read, all the various indigenous tribes past and present tend to have the same problems as every other group of people, including modern civilization; they're just earlier on the problem complexity/sophistication ladder. That, and they tend to have pretty horrible alternatives to what we've learned to solve via taxation and the rule of law.
When a species evolves to the point it has planet-shaping capabilities it needs to be able to think about consequences on a planetary scale - not just in terms of short term individual benefits.
Competitive individualism completely fails at scale.
It's possible there are other Darwinian solutions - maybe life that evolves as a single cooperative colony organism - which avoid this trap.
The corollary is that if we ever meet aliens, they're not going to have much in common with us.