I've thought this for many years, but I've never come up with so succinct and perfect a way of expressing it. Anonymity prevents free speech by making it easy to just drown any legitimate speech or valuable information in a torrent of bullshit.
This was always somewhat true due to the prevalence of human trolls and propagandists, but the ability to generate content automatically makes the problem orders of magnitude worse.
In the world of deep fakes and stuff like GPT-3 (which can be used for what amounts to deep fakes for text), any information without clear provenance is worse than worthless. It's a form of pollution whose result is to confuse, misdirect, misinform/disinform, and indoctrinate, and it's impossible to use it for anything legitimate since it's completely unverifiable and could easily be fake.
Right now it's possible to detect deep fakes, but it probably won't be long until the technology is good enough that this is very hard.
Then there's the fact that a lot of people won't even check. People just believe claims in stupid memes and forward them.
Example: a photo of a supposed Denver International Airport mural depicting kids in masks with different flags on them was going around a while back. The claim was that this showed some kind of foreknowledge of COVID-19 going all the way back to the 1990s. Turns out the photo was of a recent painting that was not from DIA. It was totally fake. I've still seen it making the rounds.
I've been predicting that deep fakes will make an appearance this October in the Presidential election. I think there's a really good chance. It'll be at the last minute so nobody has time to perform any analysis or fact check anything.
You're right, but in the context here, there isn't any significant difference. A pseudonymous machine account that is indistinguishable to a human carries the same type of risks to online discourse as an anonymous account.
Code is a very different thing. It's possible to run code and objectively see whether it does what it says or not.
Political speech and whistleblower type stuff is very different. Without a really deep investigation it's impossible to verify anything.
Let's say someone dumps an anonymous video showing Donald Trump or Joe Biden molesting a child. How would you verify that?
For a CS analogy: would you trust anonymous elliptic curves provided with no background or provenance? How can you prove they're not somehow weak in a way known only to the generator? This is why some people are suspicious of the NIST elliptic curves, though in that case they've been around for so long and used to protect so many high value targets they're believed to be probably safe. They've had what amounts to a huge bounty on them for quite a while.
But there already exist authentic videos of Joe Biden sniffing children without their consent. Doesn't seem to harm him any.
I guess we'll see if this is even an issue. Lying by making shit up is not as effective as lying by omission, and the mainstream press are _experts_ at the latter.
> In the world of deep fakes and stuff like GPT-3 (which can be used for what amounts to deep fakes for text)
Exactly what is GPT-3 doing in the example? You can already write whatever you want, better than GPT-3 can. You can attribute it however you like. People have been attributing undesirable positions to their political opponents and personal enemies for thousands of years. Where does GPT-3 come into it?
Writing propaganda is labor intensive. GPT-3 could do it well enough and almost for free. Think automated troll farms and spam that can be generated in unlimited quantity and that is close enough to real text that it can't be filtered.
Basically it does for spam what the machine gun did for open field warfare.
Meme warfare is not a new idea, but appears growing ever more sophisticated. Implication being, the value of attention and critical thinking is increasing exponentially as people struggle to navigate the information explosion online and to discern what really matters. Curious how people will evolve.
Zero-knowledge proofs solve this problem. Hell, even simple private key cryptography solves this problem. Distribute private keys to anonymous individuals who somehow prove their eligibility and only allow key holders to participate in the community. Easy peasy.
Just to challenge the easy part (or is this a challenge to peasy), can you please expand on "somehow"?
What is your strategy here to ensure that only "real" people can get access to the private keys (and not real people proxying for an automaton)? How do you maintain this assurance for a long running period of time?
Web of trust (or something substantially similar) solves this on an algorithmic level. The problem is that, to the best of my knowledge, we lack friendly and accessible UI tooling to support such a scheme.
Naive web of trust algorithms are likely to threaten anonymity in their own way though. Zero knowledge proofs might be able to facilitate a solution to that.
> and not real people proxying for an automaton
You can't prevent this under any scheme, only detect it after the fact. An authenticated individual will always be able to deliberately take actions on behalf of someone else.
> You can't prevent this under any scheme, only detect it after the fact.
So do I understand correctly that if someone posts 100 messages "anonymously", and in the last one they accidentally reveal themselves, all the other 99 messages are traceable to that same person?
The part of my comment you quoted was referring to preventing people from disingenuously proxying for another entity, not to maintaining their own personal anonymity.
Regarding accidentally revealing oneself, that's always a risk. If you create an anonymous HN identity but then inadvertently link it back to yourself then all previous posts under that account are immediately traceable as well.
Web of trust itself is easy and already widely used by many free software communities as a decentralized means of securing reputation based identities (anonymous or otherwise). The challenge is in allowing an established identity (established either via reputation, government ID, in person interaction, whatever) to perform an anonymous but trusted (ie authenticated) interaction with the network. In other words, to implement a web of trust scheme that supports anonymized but otherwise verified interactions via zero knowledge proofs.
Heh, that's definitely the messier part of this. It all depends on your threat model and the scale of the community you're running. For a small group of trusted friends, one could distribute keys in person and enforce time-based key expiry. For larger groups of strangers... I'm not sure. Reelin's web of trust idea seems like a good start. The only ideas coming to my mind right now require burning down the community and starting over whenever compromised.
This is definitely an interesting and difficult challenge! What ideas do you have in mind?
That solves the anonymity requirement part and I agree. But even if we made sure all account holders are humans, still we don’t have a way to test if the content was truly written by the account holder or created by a bot. Manual human labor requirement creates a rate limiting which is removed by automated content creation in the name of the human.
Worse yet, as long as the bot content drives engagement and bot action on ads can be prevented, platforms would have little objection to this. Reddit would benefit from us thinking we are talking to humans while interacting with bot content if it keeps us on the site.
Even worse yet, being controversial or scandalous tends to generate more engagement than being agreeable, which has been so far only exploited at the recommendation layer, now will poison the content layer itself.
I wonder if we will look back and see that we have witnessed the creation of the nuclear weapons of information warfare.
Oh, definitely, but that will get very expensive very quickly, especially with a self-updating difficulty mechanism. I don't see botnets being much of a problem outside of one-off posts (or maybe a few more for state actors).
> > Manual human labor requirement creates a rate limiting which is removed by automated content creation in the name of the human.
> What if we impose artificial rate limiting with posting fees or some kind of proof-of-work requirement?
These hypothetical anonymized but authenticated interactions are backed (via ZKP or otherwise) by a trusted identity (which itself might be pseudonymous, government verified, whatever). As such, why not track and publish the (approximate) rate at which attestation interactions are undertaken by a given identity (per some very coarse grained unit of time)? This rate could then be linked to the otherwise anonymized actions upon attestation.
Then people or platform moderators would be free to make their own judgments about what constitutes a reasonable (ie trustworthy) rate of interaction within a given context.
(This is a form of information leakage, so in certain scenarios it could allow deanonymization or at least linking anonymous interactions together. Extreme care would have to be taken when selecting the degree of approximation and the duration of unit time.)
> What if we impose artificial rate limiting with posting fees or some kind of proof-of-work requirement?
This is going to be a tough one. Someone with access to GPT-3 is going to have access to enough funds to post extensively. Also, muting poorer people is not a good idea.
Proof of work seems more promising, but you'll have trouble finding something that can't be automated. Even with captchas - when you're solving them, you're effectively training Googles network to solve their own challenges.
It strikes me that none of these are really novel problems. ML reduces the cost but otherwise the issues already exist. The current solution seems to be largely reputational - either interacting with identities that you personally trust or using a platform whose moderation you have some amount of faith in.
Scale has a quality all of its own. While I disapprove of government and business propaganda, I’m a bit more worried by the idea that any nut who takes a personal dislike to me or any group I just happen to be in, could single-handedly wage a propaganda war against me as competently and effectively as any of the government-led campaigns which led to the disappointingly large list of post-WW2 genocides.
Not only the scale but also the multi-agent nature of the problem makes it hard to predict.
As an example, humans killing each other is an ancient problem. But humans killing each other at scale with nuclear weapons is a much different problem. We are able to keep that threat at bay by the virtue of only nation states having the power of killing at this scale; number of agents that need to coordinate not defecting (that understand mutually assured destruction) is a handful. Now imagine if some new technology made it possible to make nuclear weapons with materials found in a kitchen. Then we have a coordination problem that requires planet level cooperation with billions of agents. It only takes one defecting to cause disproportionate harm.
By the same token, information warfare is not a new problem. Lying is an ancient invention. News outlets being biased makes it easy to guess those biases and somewhat balance them out. We expect states to engage in some degree of propaganda. But human created but AI curated content (i.e. recommendation algorithms) is already driving us a bit nuts; we are having a hard time making a cohorent narrative of what’s going on. AI created and curated content intermixed with human content will completely destroy our sensemaking capacity for the benefit of a few who can deploy this technology.
Imagine a per-voter propaganda model that drives AI created social media interactions with them, that can precisely tell what issues would you need to change your mind on to make you change your vote, which could be polar opposite to what the next voter is going to hear. Imagine this was deployed by multiple nations whenever there is an election in a country. This would still at the level of nation-states having nuclear weapons. I can’t imagine what the consequences of a more diffuse access to this tech would look like.
> I wonder if we will look back and see that we have witnessed the creation of the nuclear weapons of information warfare.
I wouldn't be surprised. What's the point of reading anything on the internet, or arguing with someone on a forum? Gmail-style canned replies were the beginning, now the only way I can be sure that I'm talking to a real human will be by meeting him or her face-to-face (and we all know how easy that is right now.)
One problem is that you want anonymity and not pseudonymity. I.e. when you post A and later B, you don't want that these posts are traceable to the same origin even if the origin remains unknown.
Right, again, get rid of public keys with zero-knowledge proofs, and you've solved this problem. The only way to screw up is by publicly disclosing which messages are yours. Although, I guess a large part of the problem is that some users might accidentally do this implicitly, and many may do it just by writing in some unique way.
> the marginal cost of distributing astroturfed propaganda online has firmly hit zero.
This is just not true. GPT-3 costs money to train, use, customize, and maintain. I would guess that paying people is still the more cost effective way to astroturf.
Also, if the argument is we have to ban anonymity, I strongly disagree. Obviously people shouldn't carelessly trust anonymous sources, but that's just common sense. Banning anonymity would prevent people anonymously sharing insight into companies, whistleblowing, talking freely without threatening their employment (e.g. SSC), etc. It would increase the chances of physical threat due to online action, and it would provide the people in power with far to much sway over speech.
The concern is that low-quality anonymous sources will become so common that, even if anonymity isn't formally banned, most people will have to proactively filter anonymous writing the same way they filter anonymous phone calls today.
I've been advocating for traceability by default. Then if a site wants to strip identity from comments they can. But then we have a clear distinction between places where you're not anonymous and those where you are. People can then choose who they want to interact with and listen to.
Although you quoted it, I think you missed the word "marginal": https://en.wikipedia.org/wiki/Marginal_cost. Yes, training, customizing, and maintaining GPT-3 for your use is going to be expensive, initially much more expensive than using humans. But the incremental cost for the second million usages is going to be very inexpensive compared to using humans, and as technology progresses, can probably be approximated as "free".
> Also, if the argument is we have to ban anonymity, I strongly disagree.
The author isn't saying that we have to, but that if we do not, the effectiveness of any one individual's free speech is going to greatly diminish. Do you disagree with this as well? If so, it would be good to outline your counterargument. If not, then you need to explain why the preservation of anonymity is worth more than the lost utility of free speech.
> if we do not, the effectiveness of any one individual's free speech is going to greatly diminish
False dichotomy. A solution which preserves anonymity while counteracting abuse should be sought after.
> why the preservation of anonymity is worth more than the lost utility of free speech
The loss of anonymity itself endangers free speech. This isn't terribly controversial - the US Supreme Court has consistently ruled this way. (https://www.eff.org/issues/anonymity)
I know what marginal cost means. Yeah, there are a good portion of GPT-3 costs that are fixed costs. But also I would wager there's a large portion of costs that are roughly proportional to the scope of your astroturfing campaign. It's not zero marginal cost, and even though the marginal cost might be smaller than hiring people, the fixed costs are probably higher.
I disagree with the premise that GPT-3 will greatly decrease the power of individual speech; I just don't think it will be as impactful as some people think, and I believe the burden of proof is on those people. It's probably going to have a marginal effect, though. But also I thought I gave some reasons for protecting anonymity in my first comment, and the last paragraph hints at the problem with banning anonymity: it necessarily centralizes control over the Internet.
> > the marginal cost of distributing astroturfed propaganda online has firmly hit zero.
> This is just not true. GPT-3 costs money to train, use, customize, and maintain.
Train/customize are fixed rather than marginal costs for a propaganda operation, maintenance is dubiously necessary, use is strictly nonzero marginal cost, but also not particularly significant.
I disagree that training and customizing have zero marginal costs necessarily. I think for a larger scale astroturfing campaign to be effective, you're going to have to put more money and effort in customization/network-training in order to avoid detection and to make sure your campaign is effective. I probably should have clarified that's what I meant in my original comment.
Also, I wonder how cheap use will actually be. Even storing 175B parameters seems like it would cost a lot to me. I haven't looked too much into it, though.
Anonymous - in what context? For most purposes, a pseudonymous account is anonymous. Not that many people can doxx my account here at HN, but I'm not even trying to be anonymous here.
Anonymity towards a state actor or similar is very different problem and as is shown by TOR, it's a rather hard problem to solve even without considering GPT-3.
More speech doesn't prevent anyone from speaking. You're mistaking a negative liberty, the idea of free speech, for a positive liberty, the idea that everyone should have an equal amount of attention.
I think it's in the internet boy (but maybe just some other interview) that Aaron Swartz says that unlike before, where the difficulty was being able to broadcast an idea (to the masses), now it's about being listened to. We can all spin up a website or a YouTube channel, but who's gonna visit it, subscribe to it. I found that contemplation fascinating ^^
Well, if you can generate enough crap speech you could possibly create a world that is filter by default, which I think would tend to trend towards status quo.
It's one of those discoveries of "the end of the X as we know it", but actually have been existing for quite a long time already.
The author somehow misses the fact that governments has been engaging small number of opinionmaking professionals (journalists, advertisers, writers, celebs etc.) to create the feeling of majority support since the time when majority opinion became important. State controlled TV, and radio existed for decades, and they always talk on behalf of "majority" which supports gov't.
And in a more democratic country a small number of people who do news, or control social platforms can do the same out of solidarity, shared economic interests, or just because.
To say that thousands generated Twitter accounts are more powerful than Twitter itself, or even more than TV channels used to be just 30 years ago is obviuos exaggeration.
We have been living with this for quite a long time already, and while it's often works, one can't do it for a long time, because there will be inevitable cultural response: people stop believing, or just paying attention - examples are all over the recent history.
Moreover, the claim that GPT-3 is a good tool for astroturfing is unproven so far and I am extremely skeptical. I don't think the supply of trollish garbage has even been the barrier to troll-speech shutting down constructive speech. I don't think even the limit on the ability of individual, "real" people to spout Trollish garbage has been the limit.
In the GPT-3 text I've read and the GPT-2 systems I've played with, what I read is a system that "weaves" a group of sentences that seem "well written and plausible" on the surface (maybe more well and more plausible than a lot of trolls). But any detailed look seems to show each reason step as both implausible and "weird", senseless. It's impressive but it's got the "talking dog effect" - people are going to be impressed by a talking dog even if the dog's statements aren't impressive otherwise.
Moreover, I think most would-be astroturfers have very specific aims. "Because of value X shared by group Y, group Y should hate group Z and like group Q", "Politician X violates strong social taboo Y" and similar formulas. Fifty people in boiler room in a third world country can be hired to make variations on the basic "payload" the astroturfer specifies for less than one silicon valley data scientist customizing GPT-3 and they have less chance of producing GPT-3 weirdness IMO.
Some even larger-scale hybrid system might automate the process further. But for the purposes of the astroturfer, it's hardly necessary.
> But any detailed look seems to show each reason step as both implausible and "weird", senseless.
Trouble here is that we're having this discussion in one of the most unusually concerned-with-logic spaces on the web. Your average facebook user isn't doing a detailed analysis of logical steps in a theory. They see something say "wind power bad" and that's that. Wind power bad.
If you read my whole comment, yes, trolls are indeed just "wind power bad" and you don't need GPT-3 for that. You've had that forever.
GPT-3 has this quality that yes, it's very impressive but much less useful for a purpose than such impressiveness suggests. Like much of current AI, imo. Certainly there are exception, valuable uses, but still.
> Moreover, the claim that GPT-3 is a good tool for astroturfing is unproven so far and I am extremely skeptical.
Fully agreed. These great "I've only clicked next ten times" examples are nice and all, but usually are still going nowhere. On the other hand, GPT-3 _is_ pretty good, even when it needs some form of human curation. At some GPT-n we _will_ have this problem (at least with a high probability), so the point still stands.
> I don't think even the limit on the ability of individual, "real" people to spout Trollish garbage has been the limit.
It's been happening since the 70s. Go back and watch the Church Committee congressional hearings. The CIA admitted to having key players in the magazine industry all around the US. When asked if they were involved in TV, they refused to answer outside of closed session.
It is absolutely reasonable to assume much of the major TV and radio narratives we take in the US today are directly influenced by the government. It's also influenced heavily by Big Tech and industry players, to the point where we're not truly getting any unfiltered narrative, compared to just following random actual people on the ground on Twitter.
This is the classic "Manufacturing Consent" narrative/conspiracy theory, at least when "engaging" implies (sometimes) "pays".
You may not mean it that way, but rather "talks to/adjusts its messaging". In that sense, it's quite obviously true, but just... not quite as sinister?
Some people have more influence because they people have come to enjoy, trust, or otherwise seek it. Sometimes, this dynamic is formalized by putting them into position with in-built distribution advantages, such as TV personality or editorial writer.
There is nothing wrong with that. "Opinion-having" is a skill like any other, and it would be strange to make it the one category where quality does not lead to more widespread consumption.
The US is also divided in a way not seen for generations. I don't quite see how one can look at the state of politics today and complain about a lack of choices, or imply that "they are all the same". If there is some cabal of mighty though-leaders, they certainly failed quite spectacular in 2016, when the Republican establishment was opposed to the current president at least as much as the Democratic party.
Indeed, you acknowledge the possibility that opinion-makers aren't on-board for rather mundane, individual reasons with the reference to "more democratic countries". I posit that even in the US, a majority does, in principle, support "the system". The left-wing critics of capitalism I know would be perfectly happy with something like the "Nordic Model" of social democracy, which is a set of policies entirely consistent with the current political structure of the US.
I never expected to see "On the internet, nobody knows your a dog" become a new moral panic.
Here's the pragmatic solution that doesn't involve entrusting so much power to governments: communicate with people you know. If you choose to communicate with strangers, be aware that they may be lying about anything they say.
This isn't new and it's not even a technological issue. This is how everyone in a minority position feels. be that an expert, a religious minority, a sexual minority, or anyone else who faces a crowd. In 1931 it was 100 Authors against Einstein.
I seriously wonder why it took some people twitter and gpt-3 to figure out that "more speech is the solution to bad speech" is actually complete nonsense that people who work at Facebook repeat because it increases their stock value.
The author isn't even really looking for free speech, they're just re-using the term because it's apparently the only acceptable adjective in front of 'speech'. What the author is figuring out is that free expression doesn't necessary produce truth or justice.
Being heard is about quality, not quantity. If a hundred nobodies argue on the internet, nobody is listening anyway. It does not matter if they are bots.
If a hundred nobodies leave a reply to some statement by a person that matters, it does not cancel out that statement.
Free speech is necessary, but not sufficient, to find truth and pursue justice.
> Free speech is necessary, but not sufficient, to find truth and pursue justice.
I wholeheartedly agree! However, regarding the rest of what you said. Propaganda can and does influence public opinion; numerous verified historical examples exist. It was already an issue that needed to be solved (IMO) long before GPT-3 was created.
Advanced ML techniques seem likely to further exacerbate a problem that already exists.
Curiously, the author "Jay Riverlong" doesn't appear to exist prior to a month ago, right around the time when GPT-3 came out. For a "professional poker player" who "travelled extensively throughout South America and Europe", it is interesting. Could be a pseudonym, but it's hard to tell whether the author is a real person.
Interesting. FWIW, the profile picture could very well be from thispersondoesnotexist.com, a cropped face smiling at the camera with the typical blur behind, plus there's a single picture available of the author. Picked carefully, the telltale signs of the source are non-obvious (unintended swirls and warps, peculiarities in the hair and around the mouth). Not knowing if it is a fake person makes it interesting to dissect, but also very awkward if it turns out to be a real Jay so I won't dwell on it. I will however mention the suspicious blur/pixelation at the chin and below the right ear.. which makes me inclined to agree with the thesis (sorry if I'm wrong M. Riverlong). Not sure what the link with GPT3 would be tho.
Based on the variety of tweets and blog posts this seems to me like it's being done by a real person, but I am guessing this Jay is a fake persona and made to try and make the point that it's easy to create a fake person. Having a digital presence so thin and a profile pic so like GAN creations definitely is quite strong evidence of this, so if this is true their point is kind of weak given how easy it is to spot the suspicious signs.
Unless GPT-3 made huge advances since the last samples were publish, this seems like it was done by a real person.
Still, it has something of the "GPT-3" style and I could believe the persona is someone taking GPT-3/GPT-2 output and editing it until makes sense or just aping the style.
Their profile pic on Twitter also looks very much like the stuff This Person Does Not Exist generates. I have a feeling this might be a social experiment on how easy it is to generate a persona using AI.
Surprisingly confused. Some kernel of truth in here.
It's not really free speech that's at issue here is it?
It's discourse populated by other human beings, or the "right" to be clearly heard. That's already contested, without AI. No transparent attribution.
In journalism, anonymous reports are verified to some extent by the journalists. You're basically relying on the newspaper's reputation. This might work in other cases, where you have some known reputable third party that can vouch for the anonymous account, at least to verify that they're a real person and not a sock puppet.
Wikipedia does the opposite, where most editors are anonymous and untrusted worker bees, but they are supposed to cite reliable sources. That can work too. Teach a GPT-3 successor to cite things correctly and you might have something useful, sort of like a search engine that can combine data from multiple places.
But if all you have is an anonymous account, it's basically a rumor and you shouldn't trust it. Isn't it rather weird to expect strangers on the Internet to believe what we say without knowing anything about us?
87 comments
[ 5.1 ms ] story [ 266 ms ] threadThis was always somewhat true due to the prevalence of human trolls and propagandists, but the ability to generate content automatically makes the problem orders of magnitude worse.
In the world of deep fakes and stuff like GPT-3 (which can be used for what amounts to deep fakes for text), any information without clear provenance is worse than worthless. It's a form of pollution whose result is to confuse, misdirect, misinform/disinform, and indoctrinate, and it's impossible to use it for anything legitimate since it's completely unverifiable and could easily be fake.
Right now it's possible to detect deep fakes, but it probably won't be long until the technology is good enough that this is very hard.
Then there's the fact that a lot of people won't even check. People just believe claims in stupid memes and forward them.
Example: a photo of a supposed Denver International Airport mural depicting kids in masks with different flags on them was going around a while back. The claim was that this showed some kind of foreknowledge of COVID-19 going all the way back to the 1990s. Turns out the photo was of a recent painting that was not from DIA. It was totally fake. I've still seen it making the rounds.
I've been predicting that deep fakes will make an appearance this October in the Presidential election. I think there's a really good chance. It'll be at the last minute so nobody has time to perform any analysis or fact check anything.
Political speech and whistleblower type stuff is very different. Without a really deep investigation it's impossible to verify anything.
Let's say someone dumps an anonymous video showing Donald Trump or Joe Biden molesting a child. How would you verify that?
For a CS analogy: would you trust anonymous elliptic curves provided with no background or provenance? How can you prove they're not somehow weak in a way known only to the generator? This is why some people are suspicious of the NIST elliptic curves, though in that case they've been around for so long and used to protect so many high value targets they're believed to be probably safe. They've had what amounts to a huge bounty on them for quite a while.
I guess we'll see if this is even an issue. Lying by making shit up is not as effective as lying by omission, and the mainstream press are _experts_ at the latter.
Exactly what is GPT-3 doing in the example? You can already write whatever you want, better than GPT-3 can. You can attribute it however you like. People have been attributing undesirable positions to their political opponents and personal enemies for thousands of years. Where does GPT-3 come into it?
Basically it does for spam what the machine gun did for open field warfare.
What is your strategy here to ensure that only "real" people can get access to the private keys (and not real people proxying for an automaton)? How do you maintain this assurance for a long running period of time?
Naive web of trust algorithms are likely to threaten anonymity in their own way though. Zero knowledge proofs might be able to facilitate a solution to that.
> and not real people proxying for an automaton
You can't prevent this under any scheme, only detect it after the fact. An authenticated individual will always be able to deliberately take actions on behalf of someone else.
So do I understand correctly that if someone posts 100 messages "anonymously", and in the last one they accidentally reveal themselves, all the other 99 messages are traceable to that same person?
Regarding accidentally revealing oneself, that's always a risk. If you create an anonymous HN identity but then inadvertently link it back to yourself then all previous posts under that account are immediately traceable as well.
Web of trust itself is easy and already widely used by many free software communities as a decentralized means of securing reputation based identities (anonymous or otherwise). The challenge is in allowing an established identity (established either via reputation, government ID, in person interaction, whatever) to perform an anonymous but trusted (ie authenticated) interaction with the network. In other words, to implement a web of trust scheme that supports anonymized but otherwise verified interactions via zero knowledge proofs.
This is definitely an interesting and difficult challenge! What ideas do you have in mind?
Worse yet, as long as the bot content drives engagement and bot action on ads can be prevented, platforms would have little objection to this. Reddit would benefit from us thinking we are talking to humans while interacting with bot content if it keeps us on the site.
Even worse yet, being controversial or scandalous tends to generate more engagement than being agreeable, which has been so far only exploited at the recommendation layer, now will poison the content layer itself.
I wonder if we will look back and see that we have witnessed the creation of the nuclear weapons of information warfare.
What if we impose artificial rate limiting with posting fees or some kind of proof-of-work requirement?
> I wonder if we will look back and see that we have witnessed the creation of the nuclear weapons of information warfare.
That's a terrifying thought.
> What if we impose artificial rate limiting with posting fees or some kind of proof-of-work requirement?
These hypothetical anonymized but authenticated interactions are backed (via ZKP or otherwise) by a trusted identity (which itself might be pseudonymous, government verified, whatever). As such, why not track and publish the (approximate) rate at which attestation interactions are undertaken by a given identity (per some very coarse grained unit of time)? This rate could then be linked to the otherwise anonymized actions upon attestation.
Then people or platform moderators would be free to make their own judgments about what constitutes a reasonable (ie trustworthy) rate of interaction within a given context.
(This is a form of information leakage, so in certain scenarios it could allow deanonymization or at least linking anonymous interactions together. Extreme care would have to be taken when selecting the degree of approximation and the duration of unit time.)
This is going to be a tough one. Someone with access to GPT-3 is going to have access to enough funds to post extensively. Also, muting poorer people is not a good idea.
Proof of work seems more promising, but you'll have trouble finding something that can't be automated. Even with captchas - when you're solving them, you're effectively training Googles network to solve their own challenges.
As an example, humans killing each other is an ancient problem. But humans killing each other at scale with nuclear weapons is a much different problem. We are able to keep that threat at bay by the virtue of only nation states having the power of killing at this scale; number of agents that need to coordinate not defecting (that understand mutually assured destruction) is a handful. Now imagine if some new technology made it possible to make nuclear weapons with materials found in a kitchen. Then we have a coordination problem that requires planet level cooperation with billions of agents. It only takes one defecting to cause disproportionate harm.
By the same token, information warfare is not a new problem. Lying is an ancient invention. News outlets being biased makes it easy to guess those biases and somewhat balance them out. We expect states to engage in some degree of propaganda. But human created but AI curated content (i.e. recommendation algorithms) is already driving us a bit nuts; we are having a hard time making a cohorent narrative of what’s going on. AI created and curated content intermixed with human content will completely destroy our sensemaking capacity for the benefit of a few who can deploy this technology.
Imagine a per-voter propaganda model that drives AI created social media interactions with them, that can precisely tell what issues would you need to change your mind on to make you change your vote, which could be polar opposite to what the next voter is going to hear. Imagine this was deployed by multiple nations whenever there is an election in a country. This would still at the level of nation-states having nuclear weapons. I can’t imagine what the consequences of a more diffuse access to this tech would look like.
I wouldn't be surprised. What's the point of reading anything on the internet, or arguing with someone on a forum? Gmail-style canned replies were the beginning, now the only way I can be sure that I'm talking to a real human will be by meeting him or her face-to-face (and we all know how easy that is right now.)
Any thoughts on a solution?
You could have your public persona that you write yourself in your own unique style, but use GPT-3 for things you don't want attributed to you.
This is just not true. GPT-3 costs money to train, use, customize, and maintain. I would guess that paying people is still the more cost effective way to astroturf.
Also, if the argument is we have to ban anonymity, I strongly disagree. Obviously people shouldn't carelessly trust anonymous sources, but that's just common sense. Banning anonymity would prevent people anonymously sharing insight into companies, whistleblowing, talking freely without threatening their employment (e.g. SSC), etc. It would increase the chances of physical threat due to online action, and it would provide the people in power with far to much sway over speech.
Although you quoted it, I think you missed the word "marginal": https://en.wikipedia.org/wiki/Marginal_cost. Yes, training, customizing, and maintaining GPT-3 for your use is going to be expensive, initially much more expensive than using humans. But the incremental cost for the second million usages is going to be very inexpensive compared to using humans, and as technology progresses, can probably be approximated as "free".
> Also, if the argument is we have to ban anonymity, I strongly disagree.
The author isn't saying that we have to, but that if we do not, the effectiveness of any one individual's free speech is going to greatly diminish. Do you disagree with this as well? If so, it would be good to outline your counterargument. If not, then you need to explain why the preservation of anonymity is worth more than the lost utility of free speech.
> if we do not, the effectiveness of any one individual's free speech is going to greatly diminish
False dichotomy. A solution which preserves anonymity while counteracting abuse should be sought after.
> why the preservation of anonymity is worth more than the lost utility of free speech
The loss of anonymity itself endangers free speech. This isn't terribly controversial - the US Supreme Court has consistently ruled this way. (https://www.eff.org/issues/anonymity)
I disagree with the premise that GPT-3 will greatly decrease the power of individual speech; I just don't think it will be as impactful as some people think, and I believe the burden of proof is on those people. It's probably going to have a marginal effect, though. But also I thought I gave some reasons for protecting anonymity in my first comment, and the last paragraph hints at the problem with banning anonymity: it necessarily centralizes control over the Internet.
> This is just not true. GPT-3 costs money to train, use, customize, and maintain.
Train/customize are fixed rather than marginal costs for a propaganda operation, maintenance is dubiously necessary, use is strictly nonzero marginal cost, but also not particularly significant.
Also, I wonder how cheap use will actually be. Even storing 175B parameters seems like it would cost a lot to me. I haven't looked too much into it, though.
Anonymity towards a state actor or similar is very different problem and as is shown by TOR, it's a rather hard problem to solve even without considering GPT-3.
The author somehow misses the fact that governments has been engaging small number of opinionmaking professionals (journalists, advertisers, writers, celebs etc.) to create the feeling of majority support since the time when majority opinion became important. State controlled TV, and radio existed for decades, and they always talk on behalf of "majority" which supports gov't.
And in a more democratic country a small number of people who do news, or control social platforms can do the same out of solidarity, shared economic interests, or just because.
To say that thousands generated Twitter accounts are more powerful than Twitter itself, or even more than TV channels used to be just 30 years ago is obviuos exaggeration.
We have been living with this for quite a long time already, and while it's often works, one can't do it for a long time, because there will be inevitable cultural response: people stop believing, or just paying attention - examples are all over the recent history.
Moreover, the claim that GPT-3 is a good tool for astroturfing is unproven so far and I am extremely skeptical. I don't think the supply of trollish garbage has even been the barrier to troll-speech shutting down constructive speech. I don't think even the limit on the ability of individual, "real" people to spout Trollish garbage has been the limit.
In the GPT-3 text I've read and the GPT-2 systems I've played with, what I read is a system that "weaves" a group of sentences that seem "well written and plausible" on the surface (maybe more well and more plausible than a lot of trolls). But any detailed look seems to show each reason step as both implausible and "weird", senseless. It's impressive but it's got the "talking dog effect" - people are going to be impressed by a talking dog even if the dog's statements aren't impressive otherwise.
Moreover, I think most would-be astroturfers have very specific aims. "Because of value X shared by group Y, group Y should hate group Z and like group Q", "Politician X violates strong social taboo Y" and similar formulas. Fifty people in boiler room in a third world country can be hired to make variations on the basic "payload" the astroturfer specifies for less than one silicon valley data scientist customizing GPT-3 and they have less chance of producing GPT-3 weirdness IMO.
Some even larger-scale hybrid system might automate the process further. But for the purposes of the astroturfer, it's hardly necessary.
Trouble here is that we're having this discussion in one of the most unusually concerned-with-logic spaces on the web. Your average facebook user isn't doing a detailed analysis of logical steps in a theory. They see something say "wind power bad" and that's that. Wind power bad.
GPT-3 has this quality that yes, it's very impressive but much less useful for a purpose than such impressiveness suggests. Like much of current AI, imo. Certainly there are exception, valuable uses, but still.
Fully agreed. These great "I've only clicked next ten times" examples are nice and all, but usually are still going nowhere. On the other hand, GPT-3 _is_ pretty good, even when it needs some form of human curation. At some GPT-n we _will_ have this problem (at least with a high probability), so the point still stands.
> I don't think even the limit on the ability of individual, "real" people to spout Trollish garbage has been the limit.
That's probably true, though.
It is absolutely reasonable to assume much of the major TV and radio narratives we take in the US today are directly influenced by the government. It's also influenced heavily by Big Tech and industry players, to the point where we're not truly getting any unfiltered narrative, compared to just following random actual people on the ground on Twitter.
You may not mean it that way, but rather "talks to/adjusts its messaging". In that sense, it's quite obviously true, but just... not quite as sinister?
Some people have more influence because they people have come to enjoy, trust, or otherwise seek it. Sometimes, this dynamic is formalized by putting them into position with in-built distribution advantages, such as TV personality or editorial writer.
There is nothing wrong with that. "Opinion-having" is a skill like any other, and it would be strange to make it the one category where quality does not lead to more widespread consumption.
The US is also divided in a way not seen for generations. I don't quite see how one can look at the state of politics today and complain about a lack of choices, or imply that "they are all the same". If there is some cabal of mighty though-leaders, they certainly failed quite spectacular in 2016, when the Republican establishment was opposed to the current president at least as much as the Democratic party.
Indeed, you acknowledge the possibility that opinion-makers aren't on-board for rather mundane, individual reasons with the reference to "more democratic countries". I posit that even in the US, a majority does, in principle, support "the system". The left-wing critics of capitalism I know would be perfectly happy with something like the "Nordic Model" of social democracy, which is a set of policies entirely consistent with the current political structure of the US.
Here's the pragmatic solution that doesn't involve entrusting so much power to governments: communicate with people you know. If you choose to communicate with strangers, be aware that they may be lying about anything they say.
I seriously wonder why it took some people twitter and gpt-3 to figure out that "more speech is the solution to bad speech" is actually complete nonsense that people who work at Facebook repeat because it increases their stock value.
The author isn't even really looking for free speech, they're just re-using the term because it's apparently the only acceptable adjective in front of 'speech'. What the author is figuring out is that free expression doesn't necessary produce truth or justice.
If a hundred nobodies leave a reply to some statement by a person that matters, it does not cancel out that statement.
Free speech is necessary, but not sufficient, to find truth and pursue justice.
I wholeheartedly agree! However, regarding the rest of what you said. Propaganda can and does influence public opinion; numerous verified historical examples exist. It was already an issue that needed to be solved (IMO) long before GPT-3 was created.
Advanced ML techniques seem likely to further exacerbate a problem that already exists.
https://news.ycombinator.com/threads?id=riverlong
https://github.com/jayriverlong
https://twitter.com/jayriverlong
https://jayriverlong.github.io/about/
edit: I have pinged him on Twitter. Let's see what happens. https://twitter.com/breandan/status/1287148374077186059
Still, it has something of the "GPT-3" style and I could believe the persona is someone taking GPT-3/GPT-2 output and editing it until makes sense or just aping the style.
I don't see what has changed in that regard.
Perhaps no one ever read your banner supporting X in the first place...
It's not really free speech that's at issue here is it? It's discourse populated by other human beings, or the "right" to be clearly heard. That's already contested, without AI. No transparent attribution.
Wikipedia does the opposite, where most editors are anonymous and untrusted worker bees, but they are supposed to cite reliable sources. That can work too. Teach a GPT-3 successor to cite things correctly and you might have something useful, sort of like a search engine that can combine data from multiple places.
But if all you have is an anonymous account, it's basically a rumor and you shouldn't trust it. Isn't it rather weird to expect strangers on the Internet to believe what we say without knowing anything about us?
2A224B042AC1EA9BF37F0FD43C36AF022C2ADEAB