218 comments

[ 6.8 ms ] story [ 241 ms ] thread
Don't talk bad about your government, folks. We're going to be entering a new age of technological tyranny.
@ezekg, GovAI has detected unlawful talk posted from your account. Your CBDC account is locked for 48 hours.
Further violations will lead to the balances of your close friends and family being adjusted by -20%, and the balances of acquaintances being adjusted by -5%. Help protect against the threat of misinformation and safeguard your Balance for up to 28 days by reporting anything that you think could lead to harm. Remember, We're All In This Together.
> We're All In This Together.

Sent chills down my spine.

Move along citizen.
@ezekg, GovAI has detected that your post violates community guidelines. Your COVID pass is RED for 72 hours to protect yourself and others.
The government appears more worried about managing dissent than anything else, like what actual threats people are talking about.
So we'll have AI that can mask writer's identity in about three months.
Text to text AI, it should be relatively easy to do too
Basically exists already, right? A lot of college kids are using it for their writing assignments.
It'll be AIs all the way down.
If precision isn't super important you can already run your text through machine translation into another language and then back again. Spammy content sites already seem to do that to avoid copyright detection.
But it'll erase all my anachronistic grammatical and style preferences! Then what's even the point of writing?

(it's rôle and &c., goddamnit)

They demonized China for doing things like that

Now they'll copy China

I find it very funny

"How dare they do it before us"

USA next decade:

"Posting bad things on twitter reduces your credit score"

Credit scores are not determined by the government. If a credit ratings agency took that step, it would harm them because Twitter posts are unlikely to represent a meaningful variable when predicting someone's creditworthiness.

Don't confuse that with "social credit" systems, whereby China prevents you from riding trains if you say something naughty.

(citation needed - that’s not actually implemented in china at scale, though it’s a convenient talking point in the west)
It is called 社会信用体系 .

Try and keep up.

https://en.m.wikipedia.org/wiki/Social_Credit_System

Try and read.

Where exactly does it say it’s ever progressed beyond trials and announcements ie implemented at scale - oh it doesn’t

thank you. the most specific citation I could find in your link was this, regarding the 80% rollout statistic:

>As of December 2020, more than 80 percent of all the provinces, autonomous regions, and municipal cities had issued or were preparing to issue local credit laws and regulations.

> that's not actually implemented

That's my understanding as well.

> it's a convenient talking point in the west

When online comments get worked up about The Social Credit System, a key thing I believe they're trying to do is spread awareness of how disturbing it is that a government is even considering such a thing that, as we understand it, is closely related to being a core technology in an authoritarian dystopia.

While it's not implemented at scale, the unnerving fact is that govt policy makers did a careful enough take on a social credit system to decide that it was worthwhile investing (probably non-trivial amounts of ) money and resources into exploring it and did eventually reach a point where they were a handful of steps short of wide-scale implementation.

Ok. If we’re talking speculative planning, the US also considers bringing privatized credit score system under government control instead, the very thing OP is criticizing. I have no problem with criticizing authorities but the us vs them look how much worse it is in red china god bless our freedoms here angle is tired bootlicking too
Or that VP of Apple quoting from a movie called “Arthur” at an auto show resulting in him being fired.

It has begun right here in the United States and we just are oblivious to these new dastardly form of social credits.

Saying "they" are oppressing an Apple VP is an interesting contrast to the next post over which says Apple (FAANG) is part of "they".
Except that it was the bevy of Apple VPs putting pressure on this ouster.
government, FANG, it's all the same, the same group of lobby talking and coordinating with each other, hiring CIA agents and bunch of 'friends'

https://mronline.org/2022/07/27/national-security-search-eng...

that's why the US wants to ban TikTok asap, because they don't want china to be able to do what they are doing for decades too

> it would harm them because Twitter posts are unlikely to represent a meaningful variable when predicting someone's creditworthiness.

people get fired and arrested already for posting stuff on twitter, in both the US and Europe, so no, it's not just just a "twitter moderation" thing

Ask yourself why they are allowed to exist and still operate despite unable to grow and are loosing money for years, talk about anti-competitive practices, unless it's in reality a government body in disguise

> government, FANG, it's all the same

If you believe that, then you also support efforts to force big tech to respect freedom of political speech, yes?

i support nothing, i only am an observer and analyst
(comment deleted)
Developing a technology that can do that doesn't infringe on anyone's liberties.

Failing to develop that technology, leaving it to China or some other authoritarian state to do, would be more likely to harm liberties, wouldn't it?

Seems like you're just "damned if you do, damned if you don't"-ing, no offense.

> Developing a technology that can do that doesn't infringe on anyone's liberties.

Why are you making this up?

> They demonized China for doing things like that

> Now they'll copy China

Rinse and repeat.

Plans? I assumed lots of people were already working on this. There's already a lot of training data out there, and I suspect most users can be identified by use of a handful of uncommon trigrams and sentence stats. I know you can recognize things I've written at work because they have real em dashes—people rarely type with those.
NSA already does this. "Stylometry" I believe is the term. Perhaps they used heuristics and algorithms so far? I would NLP was good enough for this years ago.
Back in 2015 you might have some linguists work with data scientists to do feature engineering and use those as inputs to an LR model. I suppose you can let a deep learning model feature engineer for you, but either way, you'll get to some of the same heuristics you're thinking of.
I taught this as a routine class project in 2018.
What training data did you use in that project?
Didn't they de-annonymize Satoshi through analyzing his "stylometry" too?
Two of the people who [likely] made up "Satoshi"s core team are now deceased, RIP (Hal Finney [ALS] & Len Sussman [Suicide]).
Both of these have been widely debunked as Satoshi candidates, especially the latter.
Someone unintentionally did something similar with HN comments to find users who sound most similar to you/eachother and people were finding their throwaway and alt accounts.
Yes I recall that: 'Find Your Hacker News Doppelgänger':

https://news.ycombinator.com/item?id=27568709

Interesting. There was commentary about finding anonymous/throwaway accounts, but on mine, I did not find my anonymous account (although I use that very rarely). The accounts that turned up seemed to be all real, and in a couple cases I could guess what might have made the checker match us (e.g., mentions of MFAs or Apple //e or similar politics), but not all. I didn’t notice any linguistic similarities.
Yeah back then I felt mine didn't produce good matches either.
Neat experiment, though it took me fewer than 30 seconds to rule out my nearest "doppelganger". Too many patterns there I've never used.
That's interesting. I wonder if they will periodically update the data set to reflect new posts.
The next step after is to build an AI to convert your style of writing to another person's style.
Running all writing through AI to re-write everything in a different style while conveying the same information might work. Create a way to apply it to all online writing and you have something like a writing VPN.
Exactly. You barely need "AI" for this. Changing style enough to put some sand in a sophisticated text identifier could be as simple as introducing some spelling and grammar variations. It could easily be commoditized (isn't grammarly doing this already minus the privacy part?). Of all cat and mouse games LE and intelligence are playing, for this one I'm betting on the mouse.
Just write all your 'sensitive' texts in leetspeak, use very few words, and very short words and it should be fine. It's like the typing equivalent of cutting letters out of magazines.
I proposed this as an automated process for our provinces social services/inquiries email. The idea being that the text would be harder to use as a method of discrimination.
(comment deleted)
About 15 years ago, at JHU, I heard about an algorithm that detected a writer's gender with more than 90% of accuracy, and the NLP professor considered that problem solved.
I forget that they're called em dashes -- but I also love to use them to offset what I say from other people.
They've broken more than one Python script of mine. That'll make you declare everything UTF-8.
If you have a compose key, (–) is compose + (-) (-) (.)

Or compose + (-) (-) (-) for (—) which is wider depending on the font.

I briefly considered making one. My idea was simple — build a Markov chain for each person plus the text with the unknown author, do a dot product of the intersection of the all the chains, pick the author with the best match. Never got around to it. Perhaps this weekend?
I use em dashes — though with space — and also ellipses… Also 2×4s (that are actually 1½×3½), where 2 ≠ 4 above −270°C, and footnotes¹ and all that.

¹ There.

I'm a big counter intelligence man, so I love deliberately inserting randomness into potential identification metrics.

In the words of martha stewart, it's a good thing

You probably don't want to associate yourself with a convicted criminal as a deflection technique.
We both have unfortunate usernames when it comes to that
I'm supportive of helping our convicts reintegrate into our society as healthy contributors and seeing they receive the support they need to find their niche in our diverse economy and social landscape.
I wonder if there is a way to do that automatically.
rng shuffling of style and library usages, paradigms. and naming convention. camelCase vs NOCAMELCASE based on a coinflip, etc. leverage Lint static analyzer tools to enforce prior to submission.

lot of fun ways to stick it to those bulk surveillance data lakes that may be the targets of such training sets

You can probably spot me at work because I use really simple words.

I write obviate then I re-read and replace it with “removes the need to do $foo.

Some writers at work forget that the goal of the written word is to communicate to the reader not impress them with word salad.

I regard it the same as clever code (note: not the same as complex code) - written for the author not the reader.

Your example doesn't match the principles you highlighted. Obviate->remove is replacing a perfect-fit with a generic alternative. "Obviate" was neither a word salad nor complex, it was straight up (marginally) better. This was a tradeoff to appeal to people whose vocabulary is narrow, not improve comprehension.
People reading what I write are frequently users of English as a second language.

Expecting most of the English people to know obviate is pushing it, for someone using English as a second language it’s just adding complexity.

It’s about aiming where you expect the average reader to be.

Because you know the word don’t fall into the trap of assuming everyone else does.

Treating everyone like an idiot is just doing you and them a disservice. The word exists for a reason, use it.
Many times it's the other way around.

Obviate has several cognates in romance languages, so it is straightforward.

Now, the phrasal verbs that are so obvious to a native speaker, these are the real head-scratchers.

Or street slang. It may sound like dumbing it down to you, but it actually adds complexity for a non-native.

Indeed.

"Register": I immediately know what it means and it takes tenths of a second for me to make the decision to click (or not to).

"Sign up": despite having a CEFR C2 level of English and using it at work every day, I still need to think for a few seconds if this is a registration or a login (cf. "sign in").

For a native speaker I suppose the latter is (marginally?) easier, but for a non-native, it's much harder, not even close.

Your examples are very illustrative, but I would argue that both phrases suffer from being too generic. I wish that there wasn’t such a push for “one word” CTAs. Examples of alternatives I’d advocate for depending on context:

“Create an account”

“Subscribe to mailing list”

“Submit contest entry”

As a native speaker, it's very unpleasant to write like that. I will if necessary for effective communication, but it's like wearing a straitjacket. It can be awful to read, too. Rarer synonyms and the phrasal verbs (both commonly avoided in a simple style) are sometimes necessary to avoid excessive repetition.

It is one of the downsides of being a native speaker. I write pretty much like how I speak. It's hard to use it like a foreign language, but that's what technical international English feels like to me, in a way. In my experience, fluent second language learners master that style more easily than native speakers do.

Isn't that how communication works? Understanding your audience and "appealing" to them does improve comprehension for them.
Yeah, using words you're sure your audience knows just is communication, to me.

That said, there is of course a gray zone here.

> your audience

There lies the spring coil. The less it is established what your audience is, the more you shift from "calibrated expression" to a form of expression which is as absolute and universal as possible. (See my previous post here, at https://news.ycombinator.com/item?id=33045225 .) If you have a message for an audience, you will try and condition its expression according the audience; the more the audience is abstract, the more the expression will be unconditioned, as if intended for an ideal audience.

> This was a tradeoff to appeal to people whose vocabulary is narrow, not improve comprehension.

”Comprehension” is not some audience-independent property of a text. Comprehension is what happens when a text is well-calibrated for an audience. If your audience is unlikely to know the word ”obviate” (which seems true of many engineering settings, where the audience is international), you absolutely improve comprehension by replacing it with words the audience is more likely to know.

I agree with you that ”obviate” sounds better, and in a setting like a blog post, monograph, or even an hn comment, that’s probably what I’d use. But in e.g. a work email for colleagues in another country, you reduce the risk of miscommunication by following OP’s suggestion.

"Don't use a five-dollar word when a fifty-cent word will do."

I would have sworn that was from Elements of Style but apparently it's a Mark Twain quote.

Arguably it's that kind of thinking that's destroyed society. Dumbing things down to the lowest common denominator.
I agree with the sentiment, but in some cases it is not dumbing down but making it clearer.

For me, the gold standard of this is Carl Sagan writings. Beautiful ideas, explained in the clearest language possible.

> destroy[ing] society

More precisely: not just having people less intellectually exercised¹, but also and especially creating some sort of "demand" that "things have to be simple". Non-recognition of complexity is also one of the cracks allowing populism. Your duty is to "spend one further thought", while some instances seem to defend an idea of an "as-if-constitutional right to reduced consideration".

(¹ I wrote the other day, «I also believe that "Now drop and give me twenty" will return us fitter personnel than "Please, be fed from a straw"»)

Don't agree. Like the commenter upthread, I have for some years had the feeling that my language is bloated with subordinate clauses, and that I use too many ten-dollar words. It's easy to use flowery language to conceal from yourself the shallowness of your understanding. So I get out my Xacto knife, and cut out the crap. Sometimes.
When I compare modern thinkers to those of say 150 years ago, I find that modern thinkers not only use a simpler style, the ideas are better too.
To quote the late philosopher Dr. Rick Roderick, "Deeply rooted in our culture is anti-intellectualism; our fear of eggheads. The work of intellectuals has always been separated off from the work of ordinary people - you have to be freed from the constraints of manual labor. When I was a dishwasher, I didn't have time to do this. Any time I was involved in manual labor, I didn't really have the time to do this intellectual work."

Most people do not have time, nor desire to do additional intellectual work after slaving away doing whatever it is they do for income.

“The difference between the right word and the almost right word is the difference between lightning and the lightning bug.” Mark Twain
"If I had more time, I would have written a shorter letter"
If I had had more time, you mean.
William Faulkner:

"Ernest Hemingway: he has no courage, has never crawled out on a limb. He has never been known to use a word that might cause the reader to check with a dictionary to see if it is properly used."

Ernest Hemingway:

“Poor Faulkner. Does he really think big emotions come from big words? He thinks I don’t know the ten-dollar words. I know them all right. But there are older and simpler and better words, and those are the ones I use."

> not impress them with word salad

No. Look, I had some "fight" in these pages two days ago because somebody complained of my style.

It is the farthest from legitimate to suppose that one's intention is to impress - that is accusing people of childishness, and you need to have solid grounds to suppose that. There are higher chances that one wants to be _precise_.

That would be precision in the expression of some content, to have such expression match as perfectly as possible a defined structure of thoughts you elaborated. You may want to describe the facts precisely, and/or the mental processes that those facts provoke.

Also some image subtly (and not so subtly) emerged, days ago, of "language as a cage": but if you want to express exactly some "/that/", it is the language that will have to provide the means - as opposed to a horrifying idea of "expressing what language allows".

There are contexts in which it is of paramount importance that the reader understands the message (e.g. instructions) - pay attention towards using the most digestible expression then. But when you want your content to be expressed as the defined sculpture of structures of ideas, while many different expressions are possible they are pretty much "what they are, because they are what they should be".

Nit pick. Fair point in some ways but it depends on your audience. If the latter consists primarily of folk with an extensive vocabulary that enables them to detect the nuance that might be intended by the use of a supposedly unusual word (like obviate) then they won't know what you mean by 'word salad' unless you're using some word inappropriately. Seven letters versus twenty three characters. No competition. If obviate is what you mean then use it. For a general audience however, you're absolutely right.

Occurs to me there's another point in your favor as expressed in Orwell's version of a well-known passage from the Old Testament 'Ecclesiastes' into modern English. He makes the point brilliantly. Aesthetics does come into it!

https://www.orwellfoundation.com/the-orwell-foundation/orwel...

yeah. I've read about this many years ago, and it had scared me enough to develop an alternative writing style for my squeaky clean real life persona

I wouldn't doubt for a second that NSA is already using shit like this for their global passive adversary thing

You can pick me out because I use double hyphens -- the poor man's em dash.
Triple hyphens --- the LaTeX man's em-dash.
On my phone the failure is manifest — it linebreaks between the hyphens.
The basic version would be university code detectors that try to match similarities between users code.

That isn't detecting who wrote it, but certainly is the first step.

How robust is it though? If you scale to thousands or tens of thousands of people. If it depends on trigrams and uncommon characters it can be countered by incredibly simple measures.
> most users can be identified

For some value of "identified". However good it is, it can't definitively nail someone as the author of a piece of text.

Wouldn't it be more accurate to say it links anonymous writings together? If you don't have any writings under your real name there's nothing it can do except indicate two pseudonyms belong to the same person.
You’re assuming they don’t have copies of every email you ever sent…
I wonder how things like Gmail's smart auto-complete would affect these efforts.
I'm sure it will not be used in malicious way
The best protection against this type of de-anonymization is to take measures now, while you still have time, to prevent it. It is possible to change the style of one's writing by using a language model which alters the original text in order to create a new piece with a different style. For example, to translate your text into the grandiose and flowing diction of a bygone era, you might consider the project below.

[1] https://github.com/lwneal/victorianhackernews

There is a case to be made, not just for natural language but code.

AFAIK there is quite a bit of examples from security labs where malware authors aren't necessarily identified but at least fingerprinted based on naming conventions, patterns they use across multiple projects etc...

That sort of fingerprinting could expand to correlating someone's anonymous software projects to other examples of code elsewhere (ex: if they contribute to source available stuff).

re: the example project you mention specifically, it does feel like using tools like that almost as a linter for natural language would be a fingerprint in itself.

EDIT: As far as OPSEC goes, a fun tidbit. A friend of mine identified a PR I submitted anonymously to them, simply because of the style of PR comments I made.

I suspect this is why TrueCrypt shut down.

A paper got released in 2015 that claimed 94% accuracy in identifying authors.

I’m sure it would have been quite easy for the NSA to figure out who Satoshi Nakamoto was too considering PRISM was also scooping up everyone’s email.

I find the examples given in the README to be quite tame for Victorian English. Compare it with the ending lines of A Tale of Two Cities:

"It is a far, far better thing that I do, than I have ever done; it is a far, far better rest that I go to than I have ever known.",

or this from Pride and Prejudice:

"However little known the feelings or views of such a man may be on his first entering a neighbourhood, this truth is so well fixed in the minds of the surrounding families, that he is considered as the rightful property of some one or other of their daughters."

Tools like this probably fool traditional stylometry, but what about de-anonymization tools that find similar ideas, not writing style? Perhaps most people have boring common ideas they got from others, but the sort of people the US Government is most interested in are likely quirkier than most.
Then they'd literally just go after people because they think they might harbor certain ideas?

"sir you are under arrest for maybe thinking about banning dairy production, which is at conflict with national security. We found an anonymous text online that has the same idea."

Why even bother to come up with a pretext then?

To provide comfort to the masses that it was fair.

It would be trivial to mask text and change irrelevant opinions to be anonymous again. Though the real problem is related to posing as others.

Reminds me of ilillliillilill. There's a reason people use names like this.

Aren't GANs all about creating both the generator and the discriminator? Seems to me you can also build the "reverser" quite easily.
I someday hope for style transfer for language, so I can write a paragraph and convert it to Michael Creighton's writing style.
Kennyblanken opens up the fifteen inches of aluminum that is his aging Macintosh Book Pro, produced by a company laughably named after a piece of fruit and formerly headed by a brilliant but somewhat psychopathetic man, now long pushing up the daisies, and taps out the commands to login to his account on the antiquated and increasingly inaccurately named hackernews, bathing in the yellow-orange theme as he stretches across the pale green couch in his small apartment in the big city. Outside, a 2007 Neopolitan Flyer bus roars by, half full with tired commuters.

He expertly massages the worn keys to type out a lengthy and witty response about how he'd imitate the style of a famous novelist whose prose exists mostly to result in the sacrifice of as many innocent trees as possible - demonstrating how to stretch "I'd use Dan Brown's writing style" into nearly two full paragraphs of text.

To be a proper Dan Brown, you'd need to start with "Internet Commentator KennyBlanken" (or some other prefixed descriptor.)

cf [1]

"I think what enabled the first word to tip me off that I was about to spend a number of hours in the company of one of the worst prose stylists in the history of literature was this. Putting curriculum vitae details into complex modifiers on proper names or definite descriptions is what you do in journalistic stories about deaths; you just don't do it in describing an event in a narrative."

[1] http://itre.cis.upenn.edu/~myl/languagelog/archives/000844.h...

(comment deleted)
> export OPENAI_API_KEY=hunter2

lel

What we need is a speaking style anonymizer. Like a language translator except it translates your text into some kind of stylistic uniformness.

We're flexible on that format. Anything legible and relatively easy to translate into. Call it ANONSPEAK.

It will probably be, aesthetically, horrible.

Seems very doable given the state of google translate.

Trouble is if you are a revolutionary leader of some kind you are probably going to be saying new things that no-one else talks about - which renders both anonspeak and the ai detection kind of redundant.

I guess the application for this then is in the interim to stop people or online groups becoming revolutionary by tracking and deradicalising them with targeted manipulation.

There exists such a practice, and it is known as academic publishing.

Every verb is done in passive voice, punctuation is added - wherever possible - to make sentences appear more complex than they need be, and of course there is an effervescent use of sesquipedalian terms where shorter similes would otherwise suffice.

Didn't they already use something like this to supposedly unmask Satoshi Nakamoto?
Nakamoto Satoshi has not been unmasked despite the very short list of people capable/interested in creating what he made.

Stylometric analysis did suggest a single person on that list. The easier thing for governments to do at the time would have been to just spin up a node in the first year and look at the IP addresses.

He had no desire to become known back then and likely never will. It's only more dangerous now compared to the threat before of being locked up like the LibertyCoin guy (who just got released a year ago).

NS is happy to stay in the shadows, nearly everyone respects that decision especially in a world of crypto scams and ponzis. Surprised they never linked the domain name purchase to him though.

This is why I write my shit under my own legal name, even going to notarize this account at some point. Yeah throwaway yeah. Anonymous speech. Oh yeah darknet, Tor, cryptography, like yes sometimes, but it's a game of cat and mouse, it's purely a question of cost.

Furthermore I consider games like poker or Magic the Gathering unplayable, that is the extent to which there is literally absolutely no privacy.

But don't mind me, just got lobotomized is all.

There's a great book on this kind of thing, Author Unknown: On the Trail of Anonymous, by Don Foster. This was written way back in 2000.

People have been doing this for decades.

> People have been doing this for decades.

The key point here is that it's AI-driven and at scale -- in other words, mass surveillance.

Personally, I see this as part of the US IC's mission, despite the potential domestic detriment.

Moreover, a stylometry analysis will reveal when I started to limit myself to one "moreover" for every two or three chapters.

     Voynich manuscript ---> US Govt AI stylometry machine ---> 42
Content marketers in the digital marketing space commonly put blog posts through "spinners" that take your text and modify it through replacing words/phrases with similar equivalents. This lets you take one article and turning it into 5-10+ unique ones, even though they still discuss the same things. It would be a shame if a service like this was marketed towards those interested in privacy, it would probably break this entire system...
I’ve found plenty of articles that seem to be run through these spinners, but hand made corrections are likely to be necessary (unless it can be automated with ML for example) as you can almost always tell that something is odd based on context lacking word choices
And thats exactly what newer programs do, look at AppSumo and its practically all of them. The older gen simply used a giant dictionary, then picked a random option from the list of acceptable choices.
Ok, I wasn’t familiar with the newer gen of spinners
The thing is, AI is a good mask for a "backend process that you don't need to explain how it works". Assuming that the US government already has private conversations on multiple content and messaging platforms, this AI will provide the perfect excuse to connecting a blog post with a given id in a process.
My first cynical take was that this will be used for "hunch laundering." There could be no indication that user A is an alias for user B, other than someone's hunch, but getting a computer to say that they match might be good enough to get a warrant when someone's hunch wouldn't be. It would be similar to having drug sniffing dogs affirm their handlers' feelings.
If I want to write anonymously, I cycle my text through Google Translate multiple times and keep all the grammatical errors intact. So, English > Italian, and then Italian > French, then back to English.

I also pass it into Hemingway[0] first to make my text lean and non-superfluous.

[0] https://hemingwayapp.com/

Are you satisfied with the level it preserves your "voice"? Humor, style, metaphors, etc?
Fail, now Google and its friends and AI models have a copy of all your original text as well as the new text.
^ In case anyone is interested in learning more, the first part of what vmoore is doing is termed back-translation.
We can get around this by writing something and sending it through GPT-3 for “style correction”.
Creepy factor aside, a similar tool for attribution would be very useful for content creators (or copyright holders) currently worried about stable diffusion.
I built something like that more than a decade ago to identify alter-egos in an online game from in-game chat. It was reasonably successful and I thought about commercial applications for it, but ultimately decided most things that could be used for are creepy or evil.

I remember hearing DARPA was actively seeking research in the field around that time. In principle, I'm not absolutely against my software being part of the chain of events that leads to the decision to kill someone, but I don't trust the US government (or, realistically, anybody else) to independently verify an identification made by such a system.

I'd be surprised if the three letter agencies aren't using something at least as good as what I wrote by now.

How sophisticated was your system? It sounds like you used cutting edge NLP techniques at the time.
I'd describe it as fairly simple. It was just a classifier where each account name was a category: there was no fancy NLP. It used a single feature type and an algorithm from a well-known family. I don't want to say what either was lest I further proliferate the technique.

I cross checked using statistically improbable words, which helped confirm or exclude weak matches.

How about using this to find bot accounts?
I'm old so I've been planning on this for awhile. I have a folder that contains all my personal data: photos and videos, journal, all my saved social media posts, all my emails and all my anonymous handles leading to everything I've ever written online. My thinking is an AI could create a reasonable facsimile of myself that my descendants could have a conversation with. I think it'd be better than a autobiography since Joyce Carol Oates convinced me by something she tweeted that no one reads autobiographies, not even close family, unless you are famous.
If I had a bot that replicated my great great ancestor I’d probably get bored quickly and then try to prod it into revealing it’s deeply outdated and inappropriate social views
You're right, the novelty would wear off quickly, but it doesn't hurt anything for me to organize a data set about myself just in case
"Hey Joe, run that article of yours through the anonymizing AI first!"