Wow, this is really interesting. The research in this area, as the paper points out, is almost completely garbage. The Botometer case is especially shocking:
> The
total numbers of Twitter followers they analyzed during
the election campaign was 838,026, which, assuming the
claimed “social bot” prevalence of 9.9%, would correspond
to roughly 83k “social bots”. However, the paper does not
provide any examples of “social bot” accounts and the raw
data was not shared. When we contacted the authors, they
were unable to provide us with a single credible example of
a “social bot”.
> To summarize the result, not a single one of the accounts
in the list was a “bot” in any meaningful sense of the
word, certainly not a “social bot”. In other words, every
single “bot” in our sample was a false alarm; the precision
(positive predictive value) of Botometer on this task is 0%.
Assuming that Botometer does not perform worse than a
random number generator (as we did not check the accounts
rated as human), we can also conclude that the best guess for
the number of “social bots” following the political parties in
Germany is not 83,000, as claimed by Keller and Klinger, but
zero.
It seems that is reasonable to claim that Botometer has not proven its effectiveness. But that does not prove the opposite.
That a police department is unable to distinguish thieves from the rest of citizens do not prove that there are no thieves but just that the police department is incompetent.
Quite a few comments seem to be mixing up a couple of things here:
1. Do spam bots exist
2. Do "social bots" exist
For (1) the answer is clearly yes and nobody is claiming otherwise. However, academics don't see to care about spambots. That's a problem industrial researchers care about but not the sort of people who publish 'scientific' papers.
For (2) the correct answer is "there is no reliable evidence to support this". The term social bot is of course vague, but these papers are usually defining it as bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas. And when you go looking for evidence that such bots exist you end up finding what Gallwitz & Kreil found - that not just some of the research is bad, but all of it is. Where are the papers proving this problem actually exists in a clearly convincing way? There aren't any.
There's a very deep problem here which crops up in lots of other contexts e.g. why is there so much research into psionic abilities, why is COVID modelling so unreliable, why do scientific journals keep publishing auto-generated gibberish as "peer reviewed science" [1], why do so many papers in so many fields fail to replicate incl. not just social sciences but fields of critical importance like oncology? The root causes are the same, namely, academic institutions don't actually seem to care if research uses the scientific method or not. So you get things like Botometer as featured in this article, where it's just some ML model that's producing noise. It classified half of US Congress as bots and nobody cared. Thousands of papers have been published on the back of this model and zero have been retracted. It's because in that world what matters is publishing things that are interesting, not publishing things that are true.
> For (2) the correct answer is "there is no reliable evidence to support this". The term social bot is of course vague, but these papers are usually defining it as bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas. And when you go looking for evidence that such bots exist you end up finding what Gallwitz & Kreil found - that not just some of the research is bad, but all of it is. Where are the papers proving this problem actually exists in a clearly convincing way? There aren't any.
I'm not sure clear on your distinction between "spam bots" and "social bots," but clearly there is reliable evidence to support "bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas," exist, like plans to procure accounts for them.
> Suppliers should provide accounts on overseas social platforms to purchasers at any time. The platforms include Twitter, Facebook, etc., and the supplier should provide about 300 accounts per month on each platform. Note: Content for this project is highly time sensitive. Suppliers should be able to supply accounts at any time.
I'd imagine those accounts would be deployed and managed in an automated fashion. Is someone automating accounts to send out propaganda operating a "spam bot" or a "social bot"?
The article is paywalled so I can't tell if it's reliable or not. My prior expectation is that it's not reliable, because:
a. The NYT has been a prime pusher of conspiracy theories about Russian foreign influence on US elections, none of which turned out to be reliable. Claims of Chinese foreign influence are very similar.
b. The media have published a great number of stories on supposed Russian/foreign conspiracies to use "social bots" and on investigation they are invariably relying on third party research of the sort that this paper shows aren't reliable.
c. The New York Times has become extremely biased in the last decade, especially post 2016 and especially on anything related to why people vote the way they do.
I would say though, that 10 accounts per day doesn't seem like enough to have much impact on anything at the scale of social networks. The claim the paper is making is twofold:
1. The research asserting the existence of large scale social bots is all flawed and doesn't really show that.
2. Even if there are people trying to do such things, there's no evidence to suggest it would or could work.
Given the saturation level media coverage of this topic and the widespread ideological belief in certain political quarters that you can rewrite people's politics with hashtags, I'm sure there must be some people, somewhere, trying it out. But if there are, the academic papers claiming to have spotted them aren't really doing so.
> The article is paywalled so I can't tell if it's reliable or not. My prior expectation is that it's not reliable, because:
It's reliable, unless you're willing to go so far as to say the NYT would lie about Chinese government procurement documents: "The documents, which were part of a request for bids from contractors, offer a rare glimpse into how China’s vast bureaucracy works to spread propaganda and to sculpt opinion on global social media. They were taken offline after The Times contacted the Chinese government about them."
> a. The NYT has been a prime pusher of conspiracy theories about Russian foreign influence on US elections, none of which turned out to be reliable. Claims of Chinese foreign influence are very similar.
What outlets, specifically, do you think are more reliable on that and other topics?
> b. The media have published a great number of stories on supposed Russian/foreign conspiracies to use "social bots" and on investigation they are invariably relying on third party research of the sort that this paper shows aren't reliable.
On what basis can you personally judge the relative merits of research papers in this area?
I don't know if they would lie in this case or not. Probably not about the exact wording but when it comes to context, well, they've certainly published bizarre and untrue claims in the past on other topics.
Re: other outlets. I don't think any outlets are reliable on those topics because the whole idea this is really happening at scale seems to be made up.
Re: on what basis. Having read them. See the paper this thread is about. Also, having worked on bot fighting professionally before at Google.
>>> a. The NYT has been a prime pusher of conspiracy theories about Russian foreign influence on US elections, none of which turned out to be reliable. Claims of Chinese foreign influence are very similar.
>> What outlets, specifically, do you think are more reliable on that and other topics?
> Re: other outlets. I don't think any outlets are reliable on those topics because the whole idea this is really happening at scale seems to be made up.
So the other outlet is really just your own personal bias?
Honestly, I get the impression that you have a preconceived notion that social media manipulation isn't important (i.e. either it doesn't exist, but if it does it doesn't happen often, and if it does happen often, it doesn't work, etc.), and are moving goalposts and/or engaging in confirmation bias to reject anything that contradicts that notion. For instance, I don't think a serious outlet would claim you can "rewrite people's politics with hashtags" and the precise scale or effectiveness isn't super important (e.g. if someone's shooting at you, in important ways it doesn't matter exactly how many shots they take or if they actually hit you or not).
I'm also not clear on the distinction between "social bots", "spam bots", and a human manually operating many social media accounts like a puppeteer. I'm quite prepared to believe "social bots" don't actually exist (e.g. some kind of semi-autonomous "AI"-driven thing), but I'm quite sure the latter two do exist for political manipulation (not D to R reprogramming, more amplifying existing divisions and sowing confusion).
My 'personal bias' is better phrased as 'direct personal experience of investigating these claims in depth'. What makes you so sure these social bot campaigns really exist? Because a journalist told you?
"I don't think a serious outlet would claim you can "rewrite people's politics with hashtags"
That is exactly the claim made by some social bot research papers. Until you spend some time reading the literature I don't think it's a great idea to form strong opinions about this field.
In particular the section where he starts talking about causality inversion addresses this.
"the precise scale or effectiveness isn't super important"
Taxpayers are paying for this bad research because granting bodies and academic administrators incorrectly believe that it's a large and important problem. They believe that because they can't reliably detect pseudo-scientific research, which seems pretty important to me. Our money should be spent on researching things that are both real and important.
"Social bots" is a very poorly defined term (this is one of the issues with the research field as a whole) but generally means automated accounts that pretend to be real people, but which aren't, and which are engaging on social media to influence politics. A lot of claims about social bots imply Russia has developed general AI far in advance of anything in the west and is keeping it a secret just so they can troll Twitter, which is a bizarre and absurd claim on its face - probably why it's rarely stated outright, just implied.
"Spam bot" is easier to define because there's no implication of intelligence. Just getting a link to your webshop out there is sufficient.
Tools that help firms and people manage a single Twitter/FB account aren't a part of either definition.
I think it's funny how the language of the terminally online political partisans is mirrored. NPC/Russian Bot, SJW/Chud, Libtard/Nazi, Snowflake/Fragile, etc.. round and round it goes.
Those names are reflective of various ways you can dismiss people without considering their positions — or more charitably, categorize and dismiss people espousing positions you've already considered and dismissed. It shows that those techniques are non-ideological, independent of the views of the person employing them. Use of language like that of a sign that if you're looking for substantive discussion of ideas, you're in the wrong place.
Those terms have differing reasons to why they are said, they aren't mirrored, just grouped.
They don't really have to be partisan terms either. Some are dangerous and dehumanize the other side, not just insult. Some are just stupid. Some are rude but accurate.
-
NPC = Person who isn't thinking for themselves, follows the group mindlessly.
Russian Bot = Term used to dehumanize and attribute real dissent to a disinformation campaign.
SJW = Social justice warrior. An overly PC person who goes on social media cancel culture brigades in the name of fixing social injustices.
Chud = Foreign or strange in Slavic or slang for "unattractive"? Seems to be an insult at appearance, not intelligence.
Libtard = An insult at liberals/leftists directed at their mental capability.
Nazi = Term used to dehumanize. Usually implies the person is racist.
Snowflake = Someone who gets offended easily
Fragile = Never heard this one used, usage is older than snowflake, maybe just used now because one side doesn't want to use the same term?
>Russian Bot = Term used to dehumanize and attribute real dissent to a disinformation campaign.
I'm gathering from the context of the thread that this is used to describe real people, not actual bot accounts. After all, who would tweet at a spam bot?
NPC is used to dehumanize real dissent. I don't see how calling someone an NPC isn't calling them a bot since both are saying that your opinions were programmed by someone else and that your opinions can be dismissed.
Your characterization says more about which side of the debate you're on.
NPC could be used whichever side you are on, it's an insult for someone that follows group think.
Russian bot can only be used on the side that uses Russia as a scapegoat for any political event.
When you call someone an NPC you are saying they aren't thinking for themselves, but the issue isn't questioned as foreign propaganda, just as parroting talking points without thinking.
When you call someone a Russian bot, you are saying either that's not a real person or what they are saying is Russian propaganda.
You haven't pointed out any substantial difference there.
You've just pointed out that it is an insult that only goes one direction, and you're objecting to that probably because you're the target.
NPC, SJW, white knighting, etc are all equally as insulting and meant to shut down discussion and dehumanize. You aren't any better, no matter how much you think you are. That's just your own contribution to polarization.
Chud came from the movie (https://en.wikipedia.org/wiki/C.H.U.D. ), it entered the political "discourse" through the Chapo Trap House and became a generic word liberals use for conservatives/trump supporters
Interesting, that is what urban dictionary listed (C.H.U.D. definition). From that movie reference it still just seems like a general insult to appearance / culture (basement dweller). Unless there is more to how Chapo Trap House used it..
The CIA used white papers during the Cold War for psyops against adversarial nations. This is just one of many examples to show that feds are doing it domestically as well.
No, quite the opposite. The contention that 'psyops' were being conducted using bots is now in serious doubt because of this new research.
And far from such 'psyops' being the act of the American federal government, as you suggest, instead it was argued that the Russian government was responsible.
And the accusation that there was collusion with Russians to achieve this was the central driving force behind the impeachment and near-removal of the U.S. president.
It's completely irrelevant and nonsensical to respond to this research by claiming it proves "the feds" are "doing psyops" domestically.
I thought the accusations of collusion were more about one-sided release of information from hacked servers? and that the bots accusation (which is not disproved here afaict?) was more directed at the russians _in addition_ to the hacking accusations. but i always understood that the collusion was strictly about the hacking. correct me if i'm wrong.
So first the evil Russian government (with a gdp smaller than several US states) was behind the bots that were reported by white papers funded domestically. Now that the bots are seen as fake, it was the evil Russians again that faked the domestically funded reports! It's amazing how competent their government is at espionage and how grossly incompetent ours is on every level, according to you.
>To understand how these powerful mechanisms function, OCCRP member center Slidstvo.Info sent a journalist undercover to work at a Ukrainian troll farm in advance of this summer’s parliamentary elections.
The paper does not claim "bots aren't real." It claims:
> The field of “social bot” research is fundamentally flawed. While “social bot” researchers have received an enormous amount of public attention, their methods are highly dubious. And, as we and others have demonstrated, they fail miserably and consistently when evaluated under real-world conditions. Studies claiming to investigate the prevalence, properties, or influence of “social bots” have, in reality, just investigated false positives and artifacts of the flawed detection methods employed.
The paper found that the vast majority of Twitter users classified as bots were actually human. They analyzed data sets previously used by three organizations responsible for the majority of available papers on the topic.
The authors of the paper concluded that, in the future, any claims that withhold the list of account handles classified as bots should be ignored because the claims can’t be independently verified.
So they didn’t rebuke just one study but datasets underlying most studies on the topic, they analyzed Twitter rather than Reddit, and they did not claim that bots don’t exist. If I am mistaken, someone who’s analyzed the paper in more depth than I did can correct me.
I think this is specific about the notion of "social bots", i.e. bots that are mostly indistinguishable from humans or could even interact with them in a realistic way. Obviously everybody can write a spambot, or click a "post this on twitter" button (which is 90% of what people mistake for "bot posts", anyway).
Their research shows nearly all social bot papers are wrong, not just a single paper. I've summarized the paper and talk here, along with some more of my own inputs (I used to work on bot fighting):
One of the points I make is that social bot research is a prolific academic field. There are nearly 10,000 papers indexed on Google Scholar on the topic and that's not all of them. And basically all of them are fantasy. Clearly, bots do exist, but the techniques in these papers are not able to detect them reliably and the authors don't seem to care.
They show some papers use a bad bot detector with even worse (and unrecommended) settings. Worse, they explicitly skip bot accounts that twitter confirmed, and thus evaluate on accounts twitter didn't think were bots. That kind of mistep means it fails to show botometer is a good/bad classifier (even if we all know that), so until they fix their methodology, their misreproduction is bad scientific grounds for disqualifying all papers using it.
This is why we have peer review and you can pick what level of peer you trust your work at. I like the intent of the paper. But, I wouldn't have let it pass peer review as-is.
I do agree social scientists deserve better tools because botometer is more of a crude footgun at this point. A lot has happened since botometer, and what may have made some sense 5-10 years ago doesn't make sense today. They could have written a scientific paper showing this and I'd 100% believe it, but instead they used garbage methodolgy and people are inferring whatever they want from that.
Could you point out where they say they ignored all the accounts shut down by Twitter? In most cases they can't even do this because the sets of accounts identified as 'bots' by othe researchers aren't made public. If you look at their methodology for proving Botometer is flawed, it's based on taking sets of verified human Twitter accounts like US Congress, none of which were shut down by Twitter, and then showing high FP rates (like 50%).
Some of the papers mentioned in the analysis don't use Botornot, but are still deeply flawed. I should point out here that I'm cited by the authors (Hearn2017) and the paper I looked at doesn't use Botometer at all, nor did I exclude accounts identified by Twitter for the simple reason that no specific accounts were named by the researchers at all.
"I like the intent of the paper. But, I wouldn't have let it pass peer review as-is."
Then you will be pleased to know that despite peer reviewers signing off on thousands of incorrect, evidence-free papers talking about social bots, this paper - which provides ample proof of its claims - remains a preprint because none of the journals interested in social bots are willing to publish it. What a surprise.
You sound like someone who is or was a social scientist? I think you should take it from a guy who did get paid to fight bots at Google: none of this research matters to or is read by the people whom it's supposed to benefit. Industry ignores it or occasionally points out that it's flawed, as Twitter did here: https://blog.twitter.com/en_us/topics/company/2020/bot-or-no...
"What about tools like Botometer & Bot Sentinel? ... This is an extremely limited approach
... These tools do not account for these common Twitter use cases, how far we’ve come, and how things have evolved. As a result nuance can be lost. The outcome? Binary judgments of who’s a “bot or not”, which have real potential to poison our public discourse — particularly when they are pushed out through the media."
Section 3.2: the 10%+ of already-deleted accounts were removed. You can often actually infer why an account was removed, but they didn't.
Furthermore, it is questionable to me to know 10% were already confirmed removed and then state the reproducers found all of the remaining were great.
If I went deeper, I bet even more issues. As is, this is already enough to remove any word like "all" and various weasel word phrasings.
---
Ad-hominem shouldn't matter. But:
Professionally, I work with top gov agencies, top 3 cloud providers, F500s, startups, misinfo teams, etc. to provide pluggable core tech for their core graph intelligence pipelines (Graphistry), such as for sigint tasks. Botnets, hackers, account take overs, account abuse, etc - threat hunting, threat research, AML teams, digital human trafficking. As a #data4good effort, we help non-profits & researchers on the same things, such as via our free tier & some of our own hands-on help. We're just a tiny startup, but helped with massive international takedowns, flagged one of the Jan 6 indictees back in December (that FB and friends didn't take down util after more pressure), regularly help folks publish breaking news, etc.
Before, I was a well-cited academic (Test of Time award, Best of Years, etc), and work spanned early R&D for engineering tools your colleagues likely use today, and empirical & social research stuff for being able to judge work like this. Which I am, harshly. It looked like an easy win of a paper if they were dilligent, but ended up needlessly weasely.
---
I agree with you on the tools weaknesses and big companies ignoring researchers, but for different reasons than researchers being wrong.
The Twitter response is stock corporate CYA, so I don't take that at face value.
Instead, consistent with what I've written, tools like botometer are largely irrelevant to modern practitioners here because reasons like:
- Abuse teams at free social platforms are at battle with themselves and their surrounding organization & CEOs. The Facebook whistleblower accounts may particularly resonate well with you on why being factually right/wrong has little to do with what platforms do.
- Botometer is dumb. Twitter went as far as acquiring the startup of one of the top GNN researchers: In contrast, botometer is from the dinosaur era.
- Botometer is data-starved. Data is thin (no IPs, ...) and at low volumes (capped APIs). When you have APIs, app use logs, etc., same as someone self-hosting even the tiniest and most boring online store, you can do much better informed analyses. It's like a crypto app fraud & compliance team who only looks at blockchain data to decide good bad account/transaction, and ignores all the app log data & threat intel data.
"Section 3.2: the 10%+ of already-deleted accounts were removed."
You've been implying that this is a methodological flaw of the paper we're talking about but Section 3.2 is an attempt to replicate someone else's study, one which claims Botometer isn't flawed. As they clearly point out the study wasn't replicable because the original authors didn't record/supply the data used to do the original classification, just numeric user IDs. Non-replicable research is exactly the kind of problem they're calling out here!
Gallwitz and Kreil have pointed out by this paragraph that the way the accounts were pre-filtered will already avoid Botometer's biggest weaknesses. Yet, despite this fact, "A surprisingly high number of the 121 alleged bots in our sample are in reality individuals with academic or professional credentials, many of them directly related to the topic of vaccines" and "A number of the 121 alleged bots are, in reality, the official Twitter accounts of health-related organisations", and "some of the alleged vaccine bots are, in reality, Twitter accounts related to agricultural topics and pets".
Section 3.2 points out that the study isn't replicable, which is by itself sufficient to discard it. Yet even when this problem is ignored and a replication is attempted anyway, a convincing takedown is easy to mount. It shows clearly that the authors never bothered to look at their "true positives" and is thus a great example of the problem in action.
If you think it's easy to do a much better rebuttal to all these bad papers than these two German authors managed, great, let's see it? It sounds like you would be pointing out flaws in a widely used competitor, after all. I agree completely that Botometer is dumb and data starved, but where we seem to disagree is that you think it's possible to give social 'scientists' better tools. I don't think it is. Users consent to sharing their data with services when they tweet, post, etc but they don't consent to sharing it with random academics. The whole field of academic social bot research is DOA because of this problem, yet, academic institutions like universities, journals and granting bodies systematically cannot accept this. Anything they produce will always be a data starved PRNG.
... Also: Congressional accounts are generally managed, often by multiple people and with automation tools. They don't post like normal people.
Half of them being flagged for inauthentic behavior is arguably NOT a false positive. I doubt the authors checked whether it is really Bernie Sanders or a gaggle of twenty-somethings working with some DNC tooling.
This article is good to see but frustrating as similarly weak in execution due to cherry picking and selection bias. People will mistake the review's sloppiness and issues with some papers with "social bots don't exist and detection is mostly false positives". It's not like Facebook is spending ~$1B every year here to ban 0 accounts.
- Manipulated accounts are prevalent: People sell manipulation tools, and once you identify the tool fingerprints, you can often precisely identify the accounts using them. For example, one of our team members published on NBC etc on a 'cyborg' (mixed use) tool used by Qanon + Trump's social media manager which, initially, self-identified itself via a unique user agent tag. Those resulted in high volume same-text tweets & fake retweets as they'd blast out via members who gave personal account access, which is huge for scales like MAGA Twitter. We have plenty of examples like this, and reading the paper gives a weasel-worded impression that it's not happening.
- The paper explicitly did not check accounts Twitter already removed for likewise suspicion of being bots. Note that Twitter is generally slower and and more conservative here than a researcher -- they have money at stake that really likes bots -- so when even Twitter agrees it's a TOS-abusing bot, that's generally a good sign. Ex: One of our collaborators was reporting on Saudi bots, and it took a year for Twitter to take them down, namely around the time Twitter acknowledged the Saudi gov had corrupted one of their employees and they announced a sweep. Skipping those accounts for validation is massive selection bias. Bans take awhile, and frustratingly generally too late as only well after the accounts are used for social media manipulation. But looking at old papers and ignoring the confirmed findings and then claiming no bots is bad science.
- The original botometer paper was years ago, which matched a different era in Twitter use, Twitter moderation, and attacker patterns. The paper seems similar to evaluating a HFT algorithm from 10 years ago in the current stock market, and worse, with bad settings, and being surprised they lost money. You wouldnt invest your savings that way. Likewise, who would use a 5yr old virus scanner? It's a more boring paper when you phrase it this way.
There are good takeaways from this paper for folks in the field, but they give a quite warped impression if you don't have the context for these
Ex: Simple rules and features like in botometer can work, but settings matter (drift, ...)
Ex: While we and others are moving to graph neural networks and other continuing advances in account/event analytics, we need to make sure these tools are accessible else social scientists will be stuck with old algorithms, crufty implementations, and confusing settings. (GNNs are super exciting in general, see our recent review on industry adoption of more modern graph intelligence w/ gradientflow .)
Ex: Twitter's rules for accessing+ publishing data (... you largely can't) hurts infodemic research, forcing deep hacks in what should have been a simpler reproduction study like this
Ex: Based on reception and writing of this paper, the term bots may be confusing the public on what social media manipulation looks like
Edit: They could have used clean methodology and written a paper "The Botometer tool is increasingly unreliable; X% of papers relying on it are likely incorrect". I think they should, because it's probably higher than people suspect. But it's not what they did, so I wouldn't conclude much from it until they do. The (mis)reproduction study is not even close to the standards my (peer-reviewed) peers regularly work for.
> Those resulted in high volume same-text tweets & fake retweets as they'd blast out via members who gave personal account access, which is huge for scales like MAGA Twitter.
But these are not bots, this is "huge" maga twitter coordinating messaging that they all individually support.
No, during the time windows of media manipulations, the accounts are controlled by the puppet master, not the account owner. The puppet master picks a message and a rolling schedule, and as in a malware botnet, the bot-side software is listening and autoresponds. You can google the tool's UI panels & instructions, it's pretty clear. Funny enough, you may have heard of Roger Stone, and with all of the Trump indictments and later QAnon scrutiny, some of the details we hadn't known at the time here came out.
Hence why I wrote 'cyborg': The accounts are bots during the inauthentic media manipulation, and human otherwise. Twitter is uncomfortable banning these accounts despite TOS violations because they know these are sellable eyeballs. Pretty insidious!
Where you are right is there is a slippery slope. A more benign and hypothetical version would be where the user is emailed text to retweet, and in support, people manually do something. That's a grassroots campaign by people. A dark variant happens in reality here: It's quite common for misinformers to build up their twitter feed into a misinfo parallel universe, and with Twitter happily auto-generating a wall full of misinfo recommendations for them, to then like/retweet each in support without actually reading any.
I will say a lot of the examples of false-positives for bots are true-negatives for BotSentinel. Many of the handles are deactivated or suspended, but here are some examples of some live ones.
But another question is, do we care if these bots are actually automated, or whether they are individuals that behave like them. Manually tweeting a single, political hashtag 854 times in a six day period is a spammer, regardless if they are a human. The result is little different than if they had programmed a bot to do so. Another was spamming using an IFTT trigger.
If I was on the twitter team, I'd have no problem if that person was flagged as a bot.
Of course we care. Claiming an account is a bot is very different to claiming it's a real but disruptive person.
Claims of botting are ultimately always claims of a conspiracy (by the bot master). Many of these conspiracies are merely commercial in nature and so we don't normally use the term, but social bot claims are insidious because they're invariably assertions of political or geopolitical conspiracy. If it's not true it's deeply corrosive to real personal relationships, trust in electoral integrity, international relations and so on.
Even putting the wider social effects of a false classification to one side, a disruptive person is not necessarily misrepresenting anything and can possibly be made to follow the rules of the community with temporary timeouts, notification messages, etc. They may not have realized they were being disruptive. A spammer already knows they're breaking the rules of the platform and don't care, so the countermeasures need to be different.
Twitter themselves have pointed this out quite eloquently, and made it clear that they think BotSentinel is no better than Botometer:
"What about tools like Botometer & Bot Sentinel? .... Binary judgments of who’s a bot or not have real potential to poison our public discourse — particularly when they are pushed out through the media ... the narrative on what’s actually going on is increasingly behind the curve."
I'm not associated with the Bot Sentinel project, but I think one of their major strengths is how they communicate their results. See the description for "disruptive" accounts, the class before "problematic":
> Accounts that are classified as disruptive, often engage in malicious tweet activity. Some disruptive accounts harass other accounts and use offensive language. Disruptive accounts periodically share misinformation and can frequently spam hashtags. Inauthentic accounts and toxic trolls regularly receive a disruptive rating, so we recommend you exhibit caution when interacting with disruptive accounts.
And from their FAQ
> I am a human being, so why am I rated disruptive or problematic?
> Bot Sentinel doesn’t just classify bots, we developed Bot Sentinel to classify several types of accounts including accounts operated by human beings.
All this to say, a great effort is made to ensure people understand that what is being classified is behaviour, not human-ness.
To your point re: "...social bot claims are insidious because they're invariably assertions of political or geopolitical conspiracy.
Surely, many humans behind accounts classified as "disruptive" or "problematic" are statistically not going to be physically in some warehouse or being paid as part of some state-sponsored psy-op.
But it's hard to extricate the political and propagandist nature of groups of, often loosely coordinated, individuals who consistently and totally disrupt non-political discussion with literal thousands of messages. It wouldn't be accepted in any physical public space at the scale and frequency it is done online.
> "A spammer already knows they're breaking the rules of the platform..."
I would contest this point, and further would say that ignorance of the rules is not much of an excuse. Bans are often preceded by a suspension, which provide users with an explanation of the offending behaviour.
I'm not sure how things like Bot Sentinel do anything but educate users on whether their behaviours are disruptive or not. Good faith actors will take that as constructive advice to change, bad faith actors will wear it as a badge of honour.
> Binary judgments of who’s a bot or not have real potential to poison our public discourse
Bot Sentinel does not claim to identify bots full-stop. I think this is a bit of willful ignorance on Twitter's part. The bit about "the narrative on what’s actually going on is increasingly behind the curve" is something I feel is particularly rich, because Twitter is financially incentivised to be themselves "behind the curve" when it comes to moderating behaviour on their platform, as disruptive accounts often drive engagement.
The entire "online manipulation" research field is deeply compromised and unserious. Because there are enormous political ramifications to it. You can't just come out with a rigorous & factual study of the "Russiagate" hoax or Julian Assange defamation campaign. That would be career suicide if it ended up anywhere but a right-wing or Russian blog.
One of the most glaring flaws of the research is the lack of effort put into verifying attribution. Some suspicious accounts are pushing barely-literate propaganda in support of Country A? Case closed: it's a propaganda campaign by the government of Country A. These "studies" virtually never test whether the suspicious behavior was orchestrated by an enemy of Country A to generate bad press, that is, propaganda against Country A.
This still happens even after the Vault 7 leaks revealed that the CIA actively plants "false flag" strings in Iranian, Russian, Chinese languages within CIA malware.
There is also the economic remifications, Facebook, Google, Amazon, MS, All depend upon quantity of Users and quantity of Users time. If there is no distinction between bots and real users, then bots directly increase thier revenue.
There is a threshold where platform manipulation can destroy the value of the platform, so these companies definitely have an interest in keeping it in check.
In places where labor is cheap, it could be more effective to employ the equivalent of a "call center" to have a large number of people animate a whole bunch of Facebook and Twitter accounts.
63 comments
[ 0.17 ms ] story [ 136 ms ] thread> The total numbers of Twitter followers they analyzed during the election campaign was 838,026, which, assuming the claimed “social bot” prevalence of 9.9%, would correspond to roughly 83k “social bots”. However, the paper does not provide any examples of “social bot” accounts and the raw data was not shared. When we contacted the authors, they were unable to provide us with a single credible example of a “social bot”.
> To summarize the result, not a single one of the accounts in the list was a “bot” in any meaningful sense of the word, certainly not a “social bot”. In other words, every single “bot” in our sample was a false alarm; the precision (positive predictive value) of Botometer on this task is 0%. Assuming that Botometer does not perform worse than a random number generator (as we did not check the accounts rated as human), we can also conclude that the best guess for the number of “social bots” following the political parties in Germany is not 83,000, as claimed by Keller and Klinger, but zero.
That a police department is unable to distinguish thieves from the rest of citizens do not prove that there are no thieves but just that the police department is incompetent.
1. Do spam bots exist
2. Do "social bots" exist
For (1) the answer is clearly yes and nobody is claiming otherwise. However, academics don't see to care about spambots. That's a problem industrial researchers care about but not the sort of people who publish 'scientific' papers.
For (2) the correct answer is "there is no reliable evidence to support this". The term social bot is of course vague, but these papers are usually defining it as bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas. And when you go looking for evidence that such bots exist you end up finding what Gallwitz & Kreil found - that not just some of the research is bad, but all of it is. Where are the papers proving this problem actually exists in a clearly convincing way? There aren't any.
There's a very deep problem here which crops up in lots of other contexts e.g. why is there so much research into psionic abilities, why is COVID modelling so unreliable, why do scientific journals keep publishing auto-generated gibberish as "peer reviewed science" [1], why do so many papers in so many fields fail to replicate incl. not just social sciences but fields of critical importance like oncology? The root causes are the same, namely, academic institutions don't actually seem to care if research uses the scientific method or not. So you get things like Botometer as featured in this article, where it's just some ML model that's producing noise. It classified half of US Congress as bots and nobody cared. Thousands of papers have been published on the back of this model and zero have been retracted. It's because in that world what matters is publishing things that are interesting, not publishing things that are true.
[1] https://dailysceptic.org/2021/10/03/436-randomly-generated-p...
Society is currently designed to force the have-nots to LARP gladiatorial combat, not enrich each other’s lives.
> For (2) the correct answer is "there is no reliable evidence to support this". The term social bot is of course vague, but these papers are usually defining it as bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas. And when you go looking for evidence that such bots exist you end up finding what Gallwitz & Kreil found - that not just some of the research is bad, but all of it is. Where are the papers proving this problem actually exists in a clearly convincing way? There aren't any.
I'm not sure clear on your distinction between "spam bots" and "social bots," but clearly there is reliable evidence to support "bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas," exist, like plans to procure accounts for them.
https://www.nytimes.com/interactive/2021/12/20/technology/ch...
> Suppliers should provide accounts on overseas social platforms to purchasers at any time. The platforms include Twitter, Facebook, etc., and the supplier should provide about 300 accounts per month on each platform. Note: Content for this project is highly time sensitive. Suppliers should be able to supply accounts at any time.
I'd imagine those accounts would be deployed and managed in an automated fashion. Is someone automating accounts to send out propaganda operating a "spam bot" or a "social bot"?
a. The NYT has been a prime pusher of conspiracy theories about Russian foreign influence on US elections, none of which turned out to be reliable. Claims of Chinese foreign influence are very similar.
b. The media have published a great number of stories on supposed Russian/foreign conspiracies to use "social bots" and on investigation they are invariably relying on third party research of the sort that this paper shows aren't reliable.
c. The New York Times has become extremely biased in the last decade, especially post 2016 and especially on anything related to why people vote the way they do.
I would say though, that 10 accounts per day doesn't seem like enough to have much impact on anything at the scale of social networks. The claim the paper is making is twofold:
1. The research asserting the existence of large scale social bots is all flawed and doesn't really show that.
2. Even if there are people trying to do such things, there's no evidence to suggest it would or could work.
Given the saturation level media coverage of this topic and the widespread ideological belief in certain political quarters that you can rewrite people's politics with hashtags, I'm sure there must be some people, somewhere, trying it out. But if there are, the academic papers claiming to have spotted them aren't really doing so.
It's reliable, unless you're willing to go so far as to say the NYT would lie about Chinese government procurement documents: "The documents, which were part of a request for bids from contractors, offer a rare glimpse into how China’s vast bureaucracy works to spread propaganda and to sculpt opinion on global social media. They were taken offline after The Times contacted the Chinese government about them."
> a. The NYT has been a prime pusher of conspiracy theories about Russian foreign influence on US elections, none of which turned out to be reliable. Claims of Chinese foreign influence are very similar.
What outlets, specifically, do you think are more reliable on that and other topics?
> b. The media have published a great number of stories on supposed Russian/foreign conspiracies to use "social bots" and on investigation they are invariably relying on third party research of the sort that this paper shows aren't reliable.
On what basis can you personally judge the relative merits of research papers in this area?
Re: other outlets. I don't think any outlets are reliable on those topics because the whole idea this is really happening at scale seems to be made up.
Re: on what basis. Having read them. See the paper this thread is about. Also, having worked on bot fighting professionally before at Google.
>> What outlets, specifically, do you think are more reliable on that and other topics?
> Re: other outlets. I don't think any outlets are reliable on those topics because the whole idea this is really happening at scale seems to be made up.
So the other outlet is really just your own personal bias?
Honestly, I get the impression that you have a preconceived notion that social media manipulation isn't important (i.e. either it doesn't exist, but if it does it doesn't happen often, and if it does happen often, it doesn't work, etc.), and are moving goalposts and/or engaging in confirmation bias to reject anything that contradicts that notion. For instance, I don't think a serious outlet would claim you can "rewrite people's politics with hashtags" and the precise scale or effectiveness isn't super important (e.g. if someone's shooting at you, in important ways it doesn't matter exactly how many shots they take or if they actually hit you or not).
I'm also not clear on the distinction between "social bots", "spam bots", and a human manually operating many social media accounts like a puppeteer. I'm quite prepared to believe "social bots" don't actually exist (e.g. some kind of semi-autonomous "AI"-driven thing), but I'm quite sure the latter two do exist for political manipulation (not D to R reprogramming, more amplifying existing divisions and sowing confusion).
"I don't think a serious outlet would claim you can "rewrite people's politics with hashtags"
That is exactly the claim made by some social bot research papers. Until you spend some time reading the literature I don't think it's a great idea to form strong opinions about this field.
Try watching the talk version of this paper: https://archive.org/details/hopeconf2020/20200726_2000_Peopl...
In particular the section where he starts talking about causality inversion addresses this.
"the precise scale or effectiveness isn't super important"
Taxpayers are paying for this bad research because granting bodies and academic administrators incorrectly believe that it's a large and important problem. They believe that because they can't reliably detect pseudo-scientific research, which seems pretty important to me. Our money should be spent on researching things that are both real and important.
"Social bots" is a very poorly defined term (this is one of the issues with the research field as a whole) but generally means automated accounts that pretend to be real people, but which aren't, and which are engaging on social media to influence politics. A lot of claims about social bots imply Russia has developed general AI far in advance of anything in the west and is keeping it a secret just so they can troll Twitter, which is a bizarre and absurd claim on its face - probably why it's rarely stated outright, just implied.
"Spam bot" is easier to define because there's no implication of intelligence. Just getting a link to your webshop out there is sufficient.
Tools that help firms and people manage a single Twitter/FB account aren't a part of either definition.
>these papers are usually defining it as bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas.
Sorry but the issue smells recursive
They don't really have to be partisan terms either. Some are dangerous and dehumanize the other side, not just insult. Some are just stupid. Some are rude but accurate.
-
NPC = Person who isn't thinking for themselves, follows the group mindlessly.
Russian Bot = Term used to dehumanize and attribute real dissent to a disinformation campaign.
SJW = Social justice warrior. An overly PC person who goes on social media cancel culture brigades in the name of fixing social injustices.
Chud = Foreign or strange in Slavic or slang for "unattractive"? Seems to be an insult at appearance, not intelligence.
Libtard = An insult at liberals/leftists directed at their mental capability.
Nazi = Term used to dehumanize. Usually implies the person is racist.
Snowflake = Someone who gets offended easily
Fragile = Never heard this one used, usage is older than snowflake, maybe just used now because one side doesn't want to use the same term?
I'm gathering from the context of the thread that this is used to describe real people, not actual bot accounts. After all, who would tweet at a spam bot?
Your characterization says more about which side of the debate you're on.
Russian bot can only be used on the side that uses Russia as a scapegoat for any political event.
When you call someone an NPC you are saying they aren't thinking for themselves, but the issue isn't questioned as foreign propaganda, just as parroting talking points without thinking.
When you call someone a Russian bot, you are saying either that's not a real person or what they are saying is Russian propaganda.
You've just pointed out that it is an insult that only goes one direction, and you're objecting to that probably because you're the target.
NPC, SJW, white knighting, etc are all equally as insulting and meant to shut down discussion and dehumanize. You aren't any better, no matter how much you think you are. That's just your own contribution to polarization.
May be easier to consume than reading a scientific paper...
and there you can watch it without tracking :-)
https://media.ccc.de/v/34c3-9268-social_bots_fake_news_und_f...
https://youtu.be/6jNWl5d_DOk
And far from such 'psyops' being the act of the American federal government, as you suggest, instead it was argued that the Russian government was responsible.
And the accusation that there was collusion with Russians to achieve this was the central driving force behind the impeachment and near-removal of the U.S. president.
It's completely irrelevant and nonsensical to respond to this research by claiming it proves "the feds" are "doing psyops" domestically.
You misunderstood my comment because I never suggested that. I don't know where you got that from.
[0] https://www.technologyreview.com/2021/09/16/1035851/facebook...
>Meet The Activist Who Uncovered The Russian Troll Factory Named In The Mueller Probe
>Internet activist Lyudmila Savchuk spent two months working undercover at the troll factory in 2015
https://www.npr.org/sections/parallels/2018/03/15/594062887/... (2018)
>Inside a Ukrainian Troll Farm
>To understand how these powerful mechanisms function, OCCRP member center Slidstvo.Info sent a journalist undercover to work at a Ukrainian troll farm in advance of this summer’s parliamentary elections.
https://www.occrp.org/en/investigations/inside-a-ukrainian-t... (2019)
>Undercover reporter reveals life in a Polish troll farm
>Katarzyna Pruszkiewicz spent six months running fake social media accounts at self-described ‘ePR firm’ in Wrocław
https://www.theguardian.com/world/2019/nov/01/undercover-rep... (2019)
It basically says "we rebuked one study, therefore bots aren't real."
Anyone who has been on Reddit for the last year has seen government run botnets posting spintext by the hundreds.
Thinking there are no bots is wildly naive
> The field of “social bot” research is fundamentally flawed. While “social bot” researchers have received an enormous amount of public attention, their methods are highly dubious. And, as we and others have demonstrated, they fail miserably and consistently when evaluated under real-world conditions. Studies claiming to investigate the prevalence, properties, or influence of “social bots” have, in reality, just investigated false positives and artifacts of the flawed detection methods employed.
The authors of the paper concluded that, in the future, any claims that withhold the list of account handles classified as bots should be ignored because the claims can’t be independently verified.
So they didn’t rebuke just one study but datasets underlying most studies on the topic, they analyzed Twitter rather than Reddit, and they did not claim that bots don’t exist. If I am mistaken, someone who’s analyzed the paper in more depth than I did can correct me.
https://blog.plan99.net/fake-science-part-ii-bots-that-are-n...
One of the points I make is that social bot research is a prolific academic field. There are nearly 10,000 papers indexed on Google Scholar on the topic and that's not all of them. And basically all of them are fantasy. Clearly, bots do exist, but the techniques in these papers are not able to detect them reliably and the authors don't seem to care.
They show some papers use a bad bot detector with even worse (and unrecommended) settings. Worse, they explicitly skip bot accounts that twitter confirmed, and thus evaluate on accounts twitter didn't think were bots. That kind of mistep means it fails to show botometer is a good/bad classifier (even if we all know that), so until they fix their methodology, their misreproduction is bad scientific grounds for disqualifying all papers using it.
This is why we have peer review and you can pick what level of peer you trust your work at. I like the intent of the paper. But, I wouldn't have let it pass peer review as-is.
I do agree social scientists deserve better tools because botometer is more of a crude footgun at this point. A lot has happened since botometer, and what may have made some sense 5-10 years ago doesn't make sense today. They could have written a scientific paper showing this and I'd 100% believe it, but instead they used garbage methodolgy and people are inferring whatever they want from that.
Some of the papers mentioned in the analysis don't use Botornot, but are still deeply flawed. I should point out here that I'm cited by the authors (Hearn2017) and the paper I looked at doesn't use Botometer at all, nor did I exclude accounts identified by Twitter for the simple reason that no specific accounts were named by the researchers at all.
"I like the intent of the paper. But, I wouldn't have let it pass peer review as-is."
Then you will be pleased to know that despite peer reviewers signing off on thousands of incorrect, evidence-free papers talking about social bots, this paper - which provides ample proof of its claims - remains a preprint because none of the journals interested in social bots are willing to publish it. What a surprise.
You sound like someone who is or was a social scientist? I think you should take it from a guy who did get paid to fight bots at Google: none of this research matters to or is read by the people whom it's supposed to benefit. Industry ignores it or occasionally points out that it's flawed, as Twitter did here: https://blog.twitter.com/en_us/topics/company/2020/bot-or-no...
"What about tools like Botometer & Bot Sentinel? ... This is an extremely limited approach ... These tools do not account for these common Twitter use cases, how far we’ve come, and how things have evolved. As a result nuance can be lost. The outcome? Binary judgments of who’s a “bot or not”, which have real potential to poison our public discourse — particularly when they are pushed out through the media."
Furthermore, it is questionable to me to know 10% were already confirmed removed and then state the reproducers found all of the remaining were great.
If I went deeper, I bet even more issues. As is, this is already enough to remove any word like "all" and various weasel word phrasings.
---
Ad-hominem shouldn't matter. But:
Professionally, I work with top gov agencies, top 3 cloud providers, F500s, startups, misinfo teams, etc. to provide pluggable core tech for their core graph intelligence pipelines (Graphistry), such as for sigint tasks. Botnets, hackers, account take overs, account abuse, etc - threat hunting, threat research, AML teams, digital human trafficking. As a #data4good effort, we help non-profits & researchers on the same things, such as via our free tier & some of our own hands-on help. We're just a tiny startup, but helped with massive international takedowns, flagged one of the Jan 6 indictees back in December (that FB and friends didn't take down util after more pressure), regularly help folks publish breaking news, etc.
Before, I was a well-cited academic (Test of Time award, Best of Years, etc), and work spanned early R&D for engineering tools your colleagues likely use today, and empirical & social research stuff for being able to judge work like this. Which I am, harshly. It looked like an easy win of a paper if they were dilligent, but ended up needlessly weasely.
---
I agree with you on the tools weaknesses and big companies ignoring researchers, but for different reasons than researchers being wrong.
The Twitter response is stock corporate CYA, so I don't take that at face value.
Instead, consistent with what I've written, tools like botometer are largely irrelevant to modern practitioners here because reasons like:
- Abuse teams at free social platforms are at battle with themselves and their surrounding organization & CEOs. The Facebook whistleblower accounts may particularly resonate well with you on why being factually right/wrong has little to do with what platforms do.
- Botometer is dumb. Twitter went as far as acquiring the startup of one of the top GNN researchers: In contrast, botometer is from the dinosaur era.
- Botometer is data-starved. Data is thin (no IPs, ...) and at low volumes (capped APIs). When you have APIs, app use logs, etc., same as someone self-hosting even the tiniest and most boring online store, you can do much better informed analyses. It's like a crypto app fraud & compliance team who only looks at blockchain data to decide good bad account/transaction, and ignores all the app log data & threat intel data.
You've been implying that this is a methodological flaw of the paper we're talking about but Section 3.2 is an attempt to replicate someone else's study, one which claims Botometer isn't flawed. As they clearly point out the study wasn't replicable because the original authors didn't record/supply the data used to do the original classification, just numeric user IDs. Non-replicable research is exactly the kind of problem they're calling out here!
Gallwitz and Kreil have pointed out by this paragraph that the way the accounts were pre-filtered will already avoid Botometer's biggest weaknesses. Yet, despite this fact, "A surprisingly high number of the 121 alleged bots in our sample are in reality individuals with academic or professional credentials, many of them directly related to the topic of vaccines" and "A number of the 121 alleged bots are, in reality, the official Twitter accounts of health-related organisations", and "some of the alleged vaccine bots are, in reality, Twitter accounts related to agricultural topics and pets".
Section 3.2 points out that the study isn't replicable, which is by itself sufficient to discard it. Yet even when this problem is ignored and a replication is attempted anyway, a convincing takedown is easy to mount. It shows clearly that the authors never bothered to look at their "true positives" and is thus a great example of the problem in action.
If you think it's easy to do a much better rebuttal to all these bad papers than these two German authors managed, great, let's see it? It sounds like you would be pointing out flaws in a widely used competitor, after all. I agree completely that Botometer is dumb and data starved, but where we seem to disagree is that you think it's possible to give social 'scientists' better tools. I don't think it is. Users consent to sharing their data with services when they tweet, post, etc but they don't consent to sharing it with random academics. The whole field of academic social bot research is DOA because of this problem, yet, academic institutions like universities, journals and granting bodies systematically cannot accept this. Anything they produce will always be a data starved PRNG.
Half of them being flagged for inauthentic behavior is arguably NOT a false positive. I doubt the authors checked whether it is really Bernie Sanders or a gaggle of twenty-somethings working with some DNC tooling.
This paper is frustrating.
Nearly all is only something that you would say if you had no familiarity with the field
- Manipulated accounts are prevalent: People sell manipulation tools, and once you identify the tool fingerprints, you can often precisely identify the accounts using them. For example, one of our team members published on NBC etc on a 'cyborg' (mixed use) tool used by Qanon + Trump's social media manager which, initially, self-identified itself via a unique user agent tag. Those resulted in high volume same-text tweets & fake retweets as they'd blast out via members who gave personal account access, which is huge for scales like MAGA Twitter. We have plenty of examples like this, and reading the paper gives a weasel-worded impression that it's not happening.
- The paper explicitly did not check accounts Twitter already removed for likewise suspicion of being bots. Note that Twitter is generally slower and and more conservative here than a researcher -- they have money at stake that really likes bots -- so when even Twitter agrees it's a TOS-abusing bot, that's generally a good sign. Ex: One of our collaborators was reporting on Saudi bots, and it took a year for Twitter to take them down, namely around the time Twitter acknowledged the Saudi gov had corrupted one of their employees and they announced a sweep. Skipping those accounts for validation is massive selection bias. Bans take awhile, and frustratingly generally too late as only well after the accounts are used for social media manipulation. But looking at old papers and ignoring the confirmed findings and then claiming no bots is bad science.
- The original botometer paper was years ago, which matched a different era in Twitter use, Twitter moderation, and attacker patterns. The paper seems similar to evaluating a HFT algorithm from 10 years ago in the current stock market, and worse, with bad settings, and being surprised they lost money. You wouldnt invest your savings that way. Likewise, who would use a 5yr old virus scanner? It's a more boring paper when you phrase it this way.
There are good takeaways from this paper for folks in the field, but they give a quite warped impression if you don't have the context for these
Ex: Simple rules and features like in botometer can work, but settings matter (drift, ...)
Ex: While we and others are moving to graph neural networks and other continuing advances in account/event analytics, we need to make sure these tools are accessible else social scientists will be stuck with old algorithms, crufty implementations, and confusing settings. (GNNs are super exciting in general, see our recent review on industry adoption of more modern graph intelligence w/ gradientflow .)
Ex: Twitter's rules for accessing+ publishing data (... you largely can't) hurts infodemic research, forcing deep hacks in what should have been a simpler reproduction study like this
Ex: Based on reception and writing of this paper, the term bots may be confusing the public on what social media manipulation looks like
Edit: They could have used clean methodology and written a paper "The Botometer tool is increasingly unreliable; X% of papers relying on it are likely incorrect". I think they should, because it's probably higher than people suspect. But it's not what they did, so I wouldn't conclude much from it until they do. The (mis)reproduction study is not even close to the standards my (peer-reviewed) peers regularly work for.
But these are not bots, this is "huge" maga twitter coordinating messaging that they all individually support.
Hence why I wrote 'cyborg': The accounts are bots during the inauthentic media manipulation, and human otherwise. Twitter is uncomfortable banning these accounts despite TOS violations because they know these are sellable eyeballs. Pretty insidious!
Where you are right is there is a slippery slope. A more benign and hypothetical version would be where the user is emailed text to retweet, and in support, people manually do something. That's a grassroots campaign by people. A dark variant happens in reality here: It's quite common for misinformers to build up their twitter feed into a misinfo parallel universe, and with Twitter happily auto-generating a wall full of misinfo recommendations for them, to then like/retweet each in support without actually reading any.
Although... wasn't the original Turing test actually involving whether a man would beat a bot in pretending to be a woman?
Edit: yes, it was about gender detection: https://en.m.wikipedia.org/wiki/Turing_test#:~:text=The%20Tu....
- @ErikLuczak https://botsentinel.com/profile/1077274080984862720
- @pdmonna https://botsentinel.com/profile/1200756272942829575
- @CTO1ChipNagel https://botsentinel.com/profile/1411284846
Some accounts get a "disruptive" score
- @cj_panirman https://botsentinel.com/profile/719017787805577220
But another question is, do we care if these bots are actually automated, or whether they are individuals that behave like them. Manually tweeting a single, political hashtag 854 times in a six day period is a spammer, regardless if they are a human. The result is little different than if they had programmed a bot to do so. Another was spamming using an IFTT trigger.
If I was on the twitter team, I'd have no problem if that person was flagged as a bot.
Claims of botting are ultimately always claims of a conspiracy (by the bot master). Many of these conspiracies are merely commercial in nature and so we don't normally use the term, but social bot claims are insidious because they're invariably assertions of political or geopolitical conspiracy. If it's not true it's deeply corrosive to real personal relationships, trust in electoral integrity, international relations and so on.
Even putting the wider social effects of a false classification to one side, a disruptive person is not necessarily misrepresenting anything and can possibly be made to follow the rules of the community with temporary timeouts, notification messages, etc. They may not have realized they were being disruptive. A spammer already knows they're breaking the rules of the platform and don't care, so the countermeasures need to be different.
Twitter themselves have pointed this out quite eloquently, and made it clear that they think BotSentinel is no better than Botometer:
"What about tools like Botometer & Bot Sentinel? .... Binary judgments of who’s a bot or not have real potential to poison our public discourse — particularly when they are pushed out through the media ... the narrative on what’s actually going on is increasingly behind the curve."
> Accounts that are classified as disruptive, often engage in malicious tweet activity. Some disruptive accounts harass other accounts and use offensive language. Disruptive accounts periodically share misinformation and can frequently spam hashtags. Inauthentic accounts and toxic trolls regularly receive a disruptive rating, so we recommend you exhibit caution when interacting with disruptive accounts.
And from their FAQ
> I am a human being, so why am I rated disruptive or problematic?
> Bot Sentinel doesn’t just classify bots, we developed Bot Sentinel to classify several types of accounts including accounts operated by human beings.
All this to say, a great effort is made to ensure people understand that what is being classified is behaviour, not human-ness.
To your point re: "...social bot claims are insidious because they're invariably assertions of political or geopolitical conspiracy.
Surely, many humans behind accounts classified as "disruptive" or "problematic" are statistically not going to be physically in some warehouse or being paid as part of some state-sponsored psy-op.
But it's hard to extricate the political and propagandist nature of groups of, often loosely coordinated, individuals who consistently and totally disrupt non-political discussion with literal thousands of messages. It wouldn't be accepted in any physical public space at the scale and frequency it is done online.
> "A spammer already knows they're breaking the rules of the platform..."
I would contest this point, and further would say that ignorance of the rules is not much of an excuse. Bans are often preceded by a suspension, which provide users with an explanation of the offending behaviour.
I'm not sure how things like Bot Sentinel do anything but educate users on whether their behaviours are disruptive or not. Good faith actors will take that as constructive advice to change, bad faith actors will wear it as a badge of honour.
> Binary judgments of who’s a bot or not have real potential to poison our public discourse
Bot Sentinel does not claim to identify bots full-stop. I think this is a bit of willful ignorance on Twitter's part. The bit about "the narrative on what’s actually going on is increasingly behind the curve" is something I feel is particularly rich, because Twitter is financially incentivised to be themselves "behind the curve" when it comes to moderating behaviour on their platform, as disruptive accounts often drive engagement.
One of the most glaring flaws of the research is the lack of effort put into verifying attribution. Some suspicious accounts are pushing barely-literate propaganda in support of Country A? Case closed: it's a propaganda campaign by the government of Country A. These "studies" virtually never test whether the suspicious behavior was orchestrated by an enemy of Country A to generate bad press, that is, propaganda against Country A.
This still happens even after the Vault 7 leaks revealed that the CIA actively plants "false flag" strings in Iranian, Russian, Chinese languages within CIA malware.
No bots or special tooling needed.