I know a few people who do Twitter bot detection as sort of a hobby, and I would say you have to be skeptic of these numbers from both directions.
Obvious bots are, well, obvious. Novice or careless implementations lead to accounts that tweet at the same time or based on the same trigger words. Tweets made on a routine basis are the easiest to detect (e.g. Tweets at 5PM CST every day)
But things go much deeper. There are even companies that now develop bots tools essentially, increasing the randomness or human-like qualities of bots. These are much harder to detect with the standard methods.
There are other proxies, like not getting organic replies or having no friends. But the sophisticated bot networks have already figured this out. You have to realize that sometimes human labor even goes into the effort, where someone will maintain hundreds of fake accounts, and attempt to make them look organic.
From these points, it's easy to underestimate the number of actual bot accounts.
On the other hand, these same methods don't usually control well for a certain type of Twitter user. There are actually some really obsessive twitter users who will spend 12+ hours a day active. They typically produce a ton of content, and a lot of replies. I notice this on celebrity/political accounts especially. (Apparently people are calling this the 'reply-guy' now)
Those accounts are susceptible to appearing as bots because they deviate far from what one might imagine the 'average' Twitter user looks like. Speaking anecdotally, I think this relates closest to politics, and hot topic issues (Hence the topic in this research). Take a Tweet from the US President for example. You will notice the same few dozen accounts reply to every Tweet of his. Some may be bots, but others are most definitely human.
But these often get classified as bots.
If I had to make a guess, I would say the bot detection models fail more often to account for the true number of bots.
At this point, it's an arms race. Obvious bots are becoming increasingly less obvious. Overall, I think bots are only going to get better, and it worries me for the fate of social networks. The thing most frightening is finding an obvious bot and seeing a large number of organic users interacting with it as if it were real.
Edit: One more thing I forgot to mention. An increasing number of bots are getting verified now. I have no idea how the verification process works, but apparently the botters have figured out how to game it successfully.
‘Reply-guy‘ is subjective, and a bit reductionist and has a somewhat negative connotation. It’s used to describe unwanted attention from a specific person or a specific type of person, which is also subjective. It goes further than simply replying a lot and generally being active on Twitter. Not to be a lexical prescriptivist, just aiming for clarity of description of usage and NPOV.
Appreciate it. I've just been hearing the term thrown around a lot lately in context of someone who replies to every single one of someone else's tweets.
> There are actually some really obsessive twitter users who will spend 12+ hours a day active. They typically produce a ton of content, and a lot of replies. I notice this on celebrity/political accounts especially. (Apparently people are calling this the 'reply-guy' now)
Lol there are definitely some real people that post as if they would not pass the Turing test! Repetitive replies, etc.
I went bot hunting a few weeks ago and it was incredibly insightful. What’s interesting is you can just drill in deeper and deeper by looking at the followers of bot accounts, which are very likely bots themselves. Many created in Feb and March 2020 with thousands of tweets already. I had a lot of fun thinking about the heuristics to detect a bot. Some common ones seemed to be they follow a bot, they have inflammatory bio, banner, or profile pictures. Many tweet really polarizing things.
Here is a thread I did about it. I actually tagged a suspect account and the troll started commenting in the thread, DMed me, reported random tweets, and a bunch of other stuff.
The first bot seems to launch ad hominem attacks on organic users via retweets with comments saying something terrible about them. When it doesn’t do that, it just retweets other accounts that do the same strategy.
>Some common ones seemed to be they follow a bot, they have inflammatory bio, banner, or profile pictures. Many tweet really polarizing things.
Perhaps many are real people who get banned often and create new accounts, hence the recent account created dates and inflammatory bio, banner, etc. I often hear about bots and their influence on the internet, but I don't seem to see them that often. Granted, I try not to spend a lot of time on social media, nor do I actively search for them. I suppose I have never seen a clear definition of what makes a bot a bot.
Sure. In the thread I elaborate that they typically have already cranked out thousands of tweets. Some seem more innocuous. Some seem to have already gained 50 followers in a month or two. That’s possible, but hard without a viral tweet. Some might be real users.
I think there is a space for an "ublock origin" type of extension that blocks bots.
It would first start with twitter, flagging "likely a bot". Then once it gets good enougth, just hide bot tweets.
If it becomes popular enough, you can extend it to reddit, imgur, insta, facebook, etc.
This will feed the system with a lot of data, espacially usernames, emails, accounts, and IP, plus a graph of all that, that we can cross reference to detect even more bots.
Now for the ultimate goal: hide comments from bots on shopping websites, starting with amazon.
For many of these accounts, I'm not sure it actually matters whether the account is a true bot with machine-generated text; a sockpuppet of someone with a lot of accounts; or just someone who spends all their time trolling, insulting and starting fights. No need to get into the Turing test.
"8 digits in account name" is such a reliable signal of low quality that I'd be happy to block all such accounts, for example.
People talk about "bubbles", but to me it's more like trying to insist that Origami Twitter and Arsonist Twitter share the same space - one is trying to make and the other to destroy, they're not compatible goals.
And there's a familiar real-world example of "please remove your fight-starting highly visible identifying markers so we can have a nice time in the same space": pubs with "no football colours" signs.
>"I went bot hunting a few weeks ago and it was incredibly insightful."
How does one go about "bot hunting"? Are there bot scanners? Is it ML? Genuinely curious. Are there any resources or links on the subject you could recommend?
"We do know that it looks like it's a propaganda machine, and it definitely matches the Russian and Chinese playbooks, but it would take a tremendous amount of resources to substantiate that," said Kathleen Carley, a professor of computer science at Carnegie Mellon University who is conducting a study into bot-generated coronavirus activity on Twitter that has yet to be published.
There is definitely a Russian disinformation campaign as well as a GOP disinformation campaign. With respect to Covid-19 these are designed to get Trump re-elected. I think the Russians also just want to sow discord as well per usual.
The pitch will be that it was the democratic governors who shut down the economy and not him.
This is part of the problem I have with the narrative that you must allow completely unfettered speech on any platform. That you should fight lies with the truth.
That sort of argument ends up going completely out the window when you have bot accounts spun up by the thousands designed to artificially boost certain narrative for the sake of profit or to sow dissent. Much like managing your lawn, you can't let the weeds grow and expect other plants to simply out-compete them when the weeds choke out everything else around them. Unfortunately that seems to be an inevitable problem on any platform that prioritizes userbase growth above all else.
I wonder how many of these are actively trying to influence the conversation, as opposed to just piling onto a trending hashtag with irrelevant material. As an example, I picked the currently trending hashtag #YouKnowYoureASidePieceWhen and immediately found this spam: https://twitter.com/DebtReliefLoans/status/12637033931193712...
As someone who spends way too much time on Twitter (been using it since 2008), I see these kinds of articles often and have hardly ever encountered these types of bots.
Additionally, I've had my own account locked and shadow/ghost banned on twitter for having the words coronavirus or COVID-19 in tweets. It took weeks of never tweeting those words to undo it.
I've even had my personal and business twitter accounts suspended all at once for "Violating Twitter rules of using multiple accounts for targeted harassment." They restored some of these accounts after filing an appeal and they said "We're sorry, but our systems mistakenly flagged your account." They never restored all of them and the ones restored have been locked permanently again because one account logged into the app used the C words again. One of the suspended accounts never to return was a 10K follower image bot for tech and old school computer images. Another was an account for my main business to announce news and server status updates.
Whether bots are mucking up Twitter or not, Twitter itself is a horrible platform and company. I can't wait for the service to disappear into oblivion.
>Twitter itself is a horrible platform and company. I can't wait for the service to disappear into oblivion.
You can always walk away from a service. If you are waiting for the next "Big Thing" to ditch your problems you will find that even this new shiny platform will (eventually) have the same or worse problems.
The internet is a rabble that needs taming. Services automate a lot of stuff to try to tame us, but nothing is perfect. Innocent users get banned.
If you are building a business on these services don't rely on just one. Build redundancy into your marketing, use many channels.
Although not a fan of exposing real identity on social media i wonder if there is a mechanism where twitter can acknowledge that an account is real based on some behind the scenes Zk-snark like validation.
This is bizarre to the point where there are a bunch of questions raised.
> Among the misinformation disseminated by bot accounts: tweeted conspiracy theories about hospitals being filled with mannequins or tweets that connected the spread of the coronavirus to 5G wireless towers
> We do know that it looks like it's a propaganda machine, and it definitely matches the Russian and Chinese playbooks
What exactly is mental state of the Chinese or Russian propaganda departments that they would be doing this? How could the Russians possibly benefit from US citizens aged 60+ dying off at 2-4 times the normal rate? How could the Russians possibly sow more division amongst Americans vs. the normal state of American politics? Just this week Pelosi called Trump morbidly obese, and she is the underdog of childish name-calling in that pairing. How could foreign agents trolling on twitter top that level of incivility?
And who are the idiots who are looking up random twitter accounts as their information source?
The foreign intelligence services don't care about left vs. right; they care about competent vs. incompetent, and destroying the American capacity for foresight and competent, proportional response. Every time a competent person says something correct the partisans don't like and gets fired, they win a little more.
There is no way that Russia could sink the USS Theodore Roosevelt; their own carrier practically sank on its own during a recent sortie. They're not responsible for the mass infection of its crew either - and it may not have been possible to prevent. But what they can do is make the firing of its captain a partisan issue, and of course ensure that the acting secretary of the Navy is operating in an environment where partisan loyalty to Trump is more important than empathy with the crew, so he gives a morale-destroying speech.
An early, solid lockdown would have been short and contained the virus. A relaxed, fragmented one ensures it remains in the population, damaging the economy longer.
Coronavirus isn't a biological weapon. Partisan stupidity can be weaponised, though.
That is a totally stupid strategy that relies on a series of connections that don't make any sense and could just as easily be good decisions for America that result in the country doing better than normal. And it isn't Russia making partisan issues partisan - it is the action of the partisans themselves.
It would be cheaper and more effective to bribe key officials in government. There was a conspiracy about Jeff Epstein running his paedophile ring in aid of ... Mossad or the Illuminati or whoever. It would be a bit more expensive and a lot more effective to make that conspiracy theory a reality rather than rely on bots to 'sow division' - whatever that is supposed to do that the politicians can't manage on their own.
Butina's involvement with the NRA is a perfect example of this; it's the ideal place to ramp up the partisanship and nobody can question its patriotism.
> could just as easily be good decisions for America that result in the country doing better than normal
But they're not. The point of chaos propaganda is to encourage people to make stupid decisions for the wrong reasons, to focus attention and energy on the wrong priorities, and to waste time and effort debunking nonsense. Lengthy investigations into misconduct that go nowhere are also a good waste of time that prevents governments from achieving anything.
It's not directed at specific outcomes, it's just starting as many fights as possible and getting people to bind their partisanship to ideas that are unsupported by evidence. Or supported by fake evidence from fake sources.
It's so cheap because you can use partisans as amplifiers. Just as Stalin could use American communists as "useful idiots", modern foreign propagandists can get sincere-but-misguided partisans to fight on their behalf.
41 comments
[ 3.1 ms ] story [ 94.0 ms ] threadObvious bots are, well, obvious. Novice or careless implementations lead to accounts that tweet at the same time or based on the same trigger words. Tweets made on a routine basis are the easiest to detect (e.g. Tweets at 5PM CST every day)
But things go much deeper. There are even companies that now develop bots tools essentially, increasing the randomness or human-like qualities of bots. These are much harder to detect with the standard methods.
There are other proxies, like not getting organic replies or having no friends. But the sophisticated bot networks have already figured this out. You have to realize that sometimes human labor even goes into the effort, where someone will maintain hundreds of fake accounts, and attempt to make them look organic.
From these points, it's easy to underestimate the number of actual bot accounts.
On the other hand, these same methods don't usually control well for a certain type of Twitter user. There are actually some really obsessive twitter users who will spend 12+ hours a day active. They typically produce a ton of content, and a lot of replies. I notice this on celebrity/political accounts especially. (Apparently people are calling this the 'reply-guy' now)
Those accounts are susceptible to appearing as bots because they deviate far from what one might imagine the 'average' Twitter user looks like. Speaking anecdotally, I think this relates closest to politics, and hot topic issues (Hence the topic in this research). Take a Tweet from the US President for example. You will notice the same few dozen accounts reply to every Tweet of his. Some may be bots, but others are most definitely human.
But these often get classified as bots.
If I had to make a guess, I would say the bot detection models fail more often to account for the true number of bots.
At this point, it's an arms race. Obvious bots are becoming increasingly less obvious. Overall, I think bots are only going to get better, and it worries me for the fate of social networks. The thing most frightening is finding an obvious bot and seeing a large number of organic users interacting with it as if it were real.
Edit: One more thing I forgot to mention. An increasing number of bots are getting verified now. I have no idea how the verification process works, but apparently the botters have figured out how to game it successfully.
https://twitter.com/sbarolo/status/1036685010869407744?s=20
https://en.wikipedia.org/wiki/Wikipedia:Neutral_point_of_vie...
Lol there are definitely some real people that post as if they would not pass the Turing test! Repetitive replies, etc.
I totally question the human qualities of whoever develops such tools.
Here is a thread I did about it. I actually tagged a suspect account and the troll started commenting in the thread, DMed me, reported random tweets, and a bunch of other stuff.
https://twitter.com/elamje/status/1259691257724579841?s=21
The first bot seems to launch ad hominem attacks on organic users via retweets with comments saying something terrible about them. When it doesn’t do that, it just retweets other accounts that do the same strategy.
Perhaps many are real people who get banned often and create new accounts, hence the recent account created dates and inflammatory bio, banner, etc. I often hear about bots and their influence on the internet, but I don't seem to see them that often. Granted, I try not to spend a lot of time on social media, nor do I actively search for them. I suppose I have never seen a clear definition of what makes a bot a bot.
The type of bots that are most concerning are the ones that you would not be able to readily identify as bots.
It would first start with twitter, flagging "likely a bot". Then once it gets good enougth, just hide bot tweets.
If it becomes popular enough, you can extend it to reddit, imgur, insta, facebook, etc.
This will feed the system with a lot of data, espacially usernames, emails, accounts, and IP, plus a graph of all that, that we can cross reference to detect even more bots.
Now for the ultimate goal: hide comments from bots on shopping websites, starting with amazon.
It would become a killer product.
"8 digits in account name" is such a reliable signal of low quality that I'd be happy to block all such accounts, for example.
People talk about "bubbles", but to me it's more like trying to insist that Origami Twitter and Arsonist Twitter share the same space - one is trying to make and the other to destroy, they're not compatible goals.
And there's a familiar real-world example of "please remove your fight-starting highly visible identifying markers so we can have a nice time in the same space": pubs with "no football colours" signs.
How does one go about "bot hunting"? Are there bot scanners? Is it ML? Genuinely curious. Are there any resources or links on the subject you could recommend?
"We do know that it looks like it's a propaganda machine, and it definitely matches the Russian and Chinese playbooks, but it would take a tremendous amount of resources to substantiate that," said Kathleen Carley, a professor of computer science at Carnegie Mellon University who is conducting a study into bot-generated coronavirus activity on Twitter that has yet to be published.
That sort of argument ends up going completely out the window when you have bot accounts spun up by the thousands designed to artificially boost certain narrative for the sake of profit or to sow dissent. Much like managing your lawn, you can't let the weeds grow and expect other plants to simply out-compete them when the weeds choke out everything else around them. Unfortunately that seems to be an inevitable problem on any platform that prioritizes userbase growth above all else.
https://www.lesswrong.com/posts/tscc3e5eujrsEeFN4/well-kept-...
Additionally, I've had my own account locked and shadow/ghost banned on twitter for having the words coronavirus or COVID-19 in tweets. It took weeks of never tweeting those words to undo it.
I've even had my personal and business twitter accounts suspended all at once for "Violating Twitter rules of using multiple accounts for targeted harassment." They restored some of these accounts after filing an appeal and they said "We're sorry, but our systems mistakenly flagged your account." They never restored all of them and the ones restored have been locked permanently again because one account logged into the app used the C words again. One of the suspended accounts never to return was a 10K follower image bot for tech and old school computer images. Another was an account for my main business to announce news and server status updates.
Whether bots are mucking up Twitter or not, Twitter itself is a horrible platform and company. I can't wait for the service to disappear into oblivion.
You can always walk away from a service. If you are waiting for the next "Big Thing" to ditch your problems you will find that even this new shiny platform will (eventually) have the same or worse problems.
The internet is a rabble that needs taming. Services automate a lot of stuff to try to tame us, but nothing is perfect. Innocent users get banned.
If you are building a business on these services don't rely on just one. Build redundancy into your marketing, use many channels.
it could also have a way for legitimate bots to be registered and easily identified as such
I'm all for it!
> Among the misinformation disseminated by bot accounts: tweeted conspiracy theories about hospitals being filled with mannequins or tweets that connected the spread of the coronavirus to 5G wireless towers
> We do know that it looks like it's a propaganda machine, and it definitely matches the Russian and Chinese playbooks
What exactly is mental state of the Chinese or Russian propaganda departments that they would be doing this? How could the Russians possibly benefit from US citizens aged 60+ dying off at 2-4 times the normal rate? How could the Russians possibly sow more division amongst Americans vs. the normal state of American politics? Just this week Pelosi called Trump morbidly obese, and she is the underdog of childish name-calling in that pairing. How could foreign agents trolling on twitter top that level of incivility?
And who are the idiots who are looking up random twitter accounts as their information source?
There is no way that Russia could sink the USS Theodore Roosevelt; their own carrier practically sank on its own during a recent sortie. They're not responsible for the mass infection of its crew either - and it may not have been possible to prevent. But what they can do is make the firing of its captain a partisan issue, and of course ensure that the acting secretary of the Navy is operating in an environment where partisan loyalty to Trump is more important than empathy with the crew, so he gives a morale-destroying speech.
An early, solid lockdown would have been short and contained the virus. A relaxed, fragmented one ensures it remains in the population, damaging the economy longer.
Coronavirus isn't a biological weapon. Partisan stupidity can be weaponised, though.
It would be cheaper and more effective to bribe key officials in government. There was a conspiracy about Jeff Epstein running his paedophile ring in aid of ... Mossad or the Illuminati or whoever. It would be a bit more expensive and a lot more effective to make that conspiracy theory a reality rather than rely on bots to 'sow division' - whatever that is supposed to do that the politicians can't manage on their own.
Undoubtedly that has been going on as well: https://en.wikipedia.org/wiki/Maria_Butina#Involvement_in_U.... plus some of the people on this fine list: https://en.wikipedia.org/wiki/Timeline_of_investigations_int...
Butina's involvement with the NRA is a perfect example of this; it's the ideal place to ramp up the partisanship and nobody can question its patriotism.
> could just as easily be good decisions for America that result in the country doing better than normal
But they're not. The point of chaos propaganda is to encourage people to make stupid decisions for the wrong reasons, to focus attention and energy on the wrong priorities, and to waste time and effort debunking nonsense. Lengthy investigations into misconduct that go nowhere are also a good waste of time that prevents governments from achieving anything.
It's not directed at specific outcomes, it's just starting as many fights as possible and getting people to bind their partisanship to ideas that are unsupported by evidence. Or supported by fake evidence from fake sources.
It's so cheap because you can use partisans as amplifiers. Just as Stalin could use American communists as "useful idiots", modern foreign propagandists can get sincere-but-misguided partisans to fight on their behalf.
They had the most success egging on far-right grievances, but they do the left too. California secession, etc.